https://pwiki.pic.es/api.php?action=feedcontributions&feedformat=atom&user=McaubetPublic PIC Wiki - User contributions [en]2026-05-02T09:05:47ZUser contributionsMediaWiki 1.35.14https://pwiki.pic.es/index.php?title=DCACHE_CHECK_POOL_METADATA&diff=231DCACHE CHECK POOL METADATA2017-10-06T13:29:56Z<p>Mcaubet: /* Special thanks */</p>
<hr />
<div>= For the MoD =<br />
* Open a ticket to incidents@support.pic.es with the alarm. <br />
* This alarm only affects the duration of the pool start up, nothing has to be done until the next working day where this can be fixed by dCache Administrators.<br />
* Same information as [[DCACHE CHECK POOL BTREE MEMORY]] as both alarms are strictly related.<br />
<br />
= For the Admin =<br />
<br />
* Berkely DB should be tunned according to the number of meta files that a pool contains. If this not correctly adapted, pool start up will may need several hours to be fully available. Gerd explained how to tune up a pool according to this number. Below I show a modified explanation of what he said, adapting the explanation to the ATLAS pools.<br />
<br />
* The default je.log.fileCacheSize is 100. This could be increased by creating a file je.properties and adding the 'je.log.fileCacheSize=100'.<br />
<br />
* Also, other relevant settings are:<br />
<br />
je.maxMemory=<br />
je.maxMemoryPercent=20<br />
<br />
* These are equivalent. The former sets the number of bytes used for the btree cache, while the latter defines it as a percentage of the max heap size. E.g. if dcache.java.memory.heap is set to 2048m, then 410 MB is used for the btree cache.<br />
<br />
* The question is then how to determine a good size for the cache. There are some hidden utilities you can use to do this.<br />
<br />
* Below I show an example for dc042_1:<br />
<br />
[root@dc042 meta]# java -cp /usr/share/dcache/classes/je-*.jar com.sleepycat.je.util.DbPrintLog -h . -S<br />
<DbPrintLog><br />
Log statistics:<br />
type total provisional total min max avg entries<br />
count count bytes bytes bytes bytes as % of log<br />
MapLN 1,324 0 12,617,573 76 11,985 9,529 0.3<br />
NameLN 4 0 141 31 38 35 0<br />
FileSummaryLN 231,140 0 55,644,337 24 49,701 240 1.4<br />
IN 84,083 1,080 296,942,059 45 5,709 3,531 7.7<br />
BIN 114,865 114,865 377,569,148 44 6,675 3,287 9.8<br />
DbTree 318 0 1,034,997 2,680 4,136 3,254 0<br />
Commit 3,376,012 0 114,784,408 34 34 34 3<br />
CkptStart 158 0 4,899 30 33 31 0<br />
CkptEnd 226 0 22,560 67 186 99 0<br />
Trace 2,506 0 525,597 48 402 209 0<br />
FileHeader 385 0 14,630 38 38 38 0<br />
DEL_LN_TX 748,055 0 51,409,527 57 69 68 1.3<br />
INS_LN_TX 619,747 0 192,298,301 96 1,443 310 5<br />
UPD_LN_TX 2,008,226 0 398,854,718 32 1,919 198 10.4<br />
UPD_LN 6,636,173 0 1,997,042,458 18 1,819 300 51.9<br />
NewBINDelta 720,540 720,540 348,480,254 55 1,782 483 9.1<br />
key bytes 10,012,201 387,254,450 1 61 38 (10.1)<br />
data bytes 9,264,146 2,036,632,128 1 1,848 219 (52.9)<br />
<br />
Total bytes in portion of log read: 3,847,245,607<br />
Total number of entries: 14,543,762<br />
<br />
Per checkpoint interval info:<br />
lnTxn ln mapLNTxn mapLN end to end end to start start to end maxLNReplay ckptEnd<br />
273 63,082 0 10 30,455,407 3,857,456,063 0 63,355 0x1f2f/0x6f2ef<br />
5,168 130,892 0 24 39,782,369 3,827,000,656 0 136,060 0x226b/0x3a0d0<br />
27,098 132,905 0 15 50,120,541 16,509,395 33,611,146 160,003 0x252c/0x577ad<br />
128 32,773 0 3 9,904,160 3,737,097,746 0 134,154 0x254c/0x4014d<br />
82 31,341 0 12 10,576,583 3,727,193,586 0 31,423 0x2567/0xccd94<br />
0 72 0 4 711,899 51,780 660,119 72 0x2567/0x17aa6f<br />
36,445 61,509 0 0 28,499,838 3,715,905,104 0 97,954 0x25e5/0xc66d<br />
2 108 0 6 1,564,919 56,871 1,508,048 110 0x25e5/0x18a764<br />
10,732 50,299 0 5 18,877,564 3,685,840,347 0 61,055 0x2678/0x786e0<br />
33 190 0 6 1,627,301 94,279 1,533,022 223 0x2678/0x205b85<br />
8,277 84,136 0 3 28,217,133 17,808,699 10,408,434 92,434 0x26be/0x52732<br />
5,401 25,394 0 7 9,824,151 3,637,118,349 0 61,029 0x26cf/0x27849<br />
49,026 92,572 0 3 41,592,807 3,627,294,198 0 141,598 0x2747/0x1ac630<br />
2 46 0 6 357,170 45,107 312,063 48 0x2747/0x203962<br />
20,496 39,154 0 3 18,591,447 3,585,344,221 0 59,654 0x27db/0xabb39<br />
0 154 0 6 1,187,463 83,676 1,103,787 154 0x27db/0x1cd9c0<br />
17,107 49,468 0 3 18,905,722 3,565,565,311 0 66,583 0x285c/0xc273a<br />
58 0 0 6 392,501 62,920 329,581 58 0x285c/0x12246f<br />
113,890 201,969 0 17 79,488,418 3,546,267,088 0 315,874 0x28ed/0xa5611<br />
22,147 98,579 0 8 40,657,515 36,319,235 4,338,280 120,726 0x2952/0x145e7c<br />
0 122 0 6 1,296,043 70,361 1,225,682 28,367 0x2952/0x282527<br />
19,570 65,944 0 5 27,570,660 25,251,492 2,319,168 85,520 0x2962/0x3138b<br />
37,714 28,889 0 3 21,505,514 19,337,884 2,167,630 66,613 0x297d/0x1a0c75<br />
3 332 0 6 2,437,155 124,630 2,312,525 479 0x297d/0x3f3c98<br />
41,458 62,406 0 3 26,385,897 3,373,311,783 0 103,904 0x29c7/0x81701<br />
0 46 0 6 1,404,642 46,045 1,358,597 46 0x29c7/0x1d85e3<br />
41,443 87,747 0 3 39,909,271 14,624,816 25,284,455 129,198 0x29fb/0x1c237a<br />
0 123 0 5 1,185,318 68,652 1,116,666 79,470 0x29fb/0x2e39a0<br />
9,199 12,698 0 4 9,927,451 3,304,426,655 0 21,899 0x2a00/0x2d1e3b<br />
8 0 0 5 118,374 35,364 83,010 8 0x2a00/0x2eeca1<br />
122,697 96,787 0 3 66,966,017 3,294,380,830 0 219,488 0x2a44/0xa122<br />
53,436 12,002 0 3 21,830,901 3,227,414,813 0 65,438 0x2a53/0x1c9117<br />
20 167 0 6 1,308,679 85,964 1,222,715 187 0x2a53/0x30891e<br />
18,552 38,805 0 6 17,501,977 3,204,275,233 0 57,381 0x2a58/0xa6b37<br />
9 105 0 4 678,297 87,871 590,426 114 0x2a58/0x14c4d0<br />
25,434 36,231 0 10 19,064,711 3,186,094,959 0 61,675 0x2a5e/0x67f57<br />
15 159 0 6 1,467,746 83,228 1,384,518 174 0x2a5e/0x1ce4b9<br />
12,236 14,099 0 3 8,375,785 3,165,562,502 0 26,358 0x2a63/0x41c22<br />
6 184 0 6 1,058,988 89,948 969,040 190 0x2a63/0x1444ce<br />
71,844 24,018 0 3 29,188,061 27,937,328 1,250,733 95,870 0x2aac/0x7e12b<br />
9,407 25,608 0 7 11,016,096 3,126,939,668 0 35,029 0x2aba/0x17624b<br />
20 110 0 6 748,510 82,108 666,402 130 0x2aba/0x22ce29<br />
64,997 49,072 0 15 27,822,548 27,438,951 383,597 114,142 0x2af3/0x1947d<br />
10,534 23,197 0 6 12,103,837 3,087,352,514 0 33,737 0x2b01/0x21ae9a<br />
122 0 0 32 879,716 489,740 389,976 122 0x2b01/0x2f1afe<br />
60,029 128,431 0 24 58,556,679 56,338,979 2,217,700 188,485 0x2b3c/0x191505<br />
15 52 0 6 530,374 53,795 476,579 231 0x2b3c/0x212ccb<br />
48,799 11,045 0 4 18,178,427 17,837,860 340,567 59,855 0x2b41/0x56146<br />
45,998 29,564 0 3 20,154,236 2,997,103,481 0 75,573 0x2b52/0x7bbc2<br />
39,570 60,757 0 15 30,273,541 2,976,949,245 0 100,327 0x2b8e/0xbe847<br />
31,631 33,780 0 13 22,348,448 16,815,306 5,533,142 65,411 0x2bad/0x2fbde7<br />
16,831 40,178 0 5 20,571,223 2,924,327,256 0 57,047 0x2bc6/0x38753e<br />
4,292 18,826 0 8 8,363,076 2,903,756,033 0 23,118 0x2be9/0x1f7b02<br />
0 254 0 6 2,043,974 119,900 1,924,074 254 0x2be9/0x3eab48<br />
30 19,902 0 4 7,727,051 2,893,348,983 0 19,954 0x2c00/0x1bfc93<br />
27,984 75,433 0 32 29,896,927 2,885,621,932 0 103,417 0x2c13/0x1a69f2<br />
40,477 36,001 0 10 19,737,838 17,591,303 2,146,535 76,478 0x2c2a/0x1669e0<br />
2 2 0 4 194,668 36,808 157,860 10 0x2c2a/0x19624c<br />
29,246 68,367 0 3 31,325,183 2,835,792,499 0 97,615 0x2c4c/0x2d9acb<br />
0 26 0 5 1,118,302 27,604 1,090,698 26 0x2c4c/0x3eab29<br />
4 19,763 0 3 7,562,665 2,803,349,014 0 19,775 0x2c53/0x197a52<br />
26 194 0 6 1,436,963 103,803 1,333,160 220 0x2c53/0x2f6775<br />
6,255 14,988 0 8 9,972,102 2,794,349,386 0 21,292 0x2c5f/0x2efa7b<br />
39,212 18,305 0 5 18,591,243 2,784,377,284 0 57,517 0x2c8f/0x197b86<br />
0 41 0 6 1,495,215 121,720 1,373,495 41 0x2c8f/0x304c35<br />
347 23,340 0 3 7,036,643 2,764,290,826 0 23,697 0x2c95/0x31498<br />
23,408 9,116 0 3 10,098,528 2,757,254,183 0 32,524 0x2ca4/0x49578<br />
45,631 27,183 0 3 19,972,142 2,747,155,655 0 72,814 0x2cad/0x428a6<br />
0 0 0 1 963,802 69 963,733 0 0x2cad/0x12dd80<br />
276 19,066 0 4 10,705,056 5,926,693 4,778,363 19,342 0x2cc4/0x1d9fa0<br />
0 25,292 0 2 10,568,825 7,122,257 3,446,568 25,292 0x2ccf/0x264d99<br />
722 25,106 0 3 9,376,076 2,704,945,830 0 26,568 0x2cd3/0x1cc865<br />
15 96 0 6 601,987 65,907 536,080 111 0x2cd3/0x25f7e8<br />
50,662 51,942 0 4 29,283,798 2,694,967,767 0 102,611 0x2d20/0x1b0a3e<br />
18,486 16,140 0 4 12,291,893 2,665,683,969 0 34,626 0x2d3a/0x3e02f3<br />
67,481 104,443 0 5 48,233,597 15,261,303 32,972,294 171,924 0x2d79/0x230ef0<br />
19,276 34,062 0 3 18,103,592 2,605,158,479 0 171,476 0x2deb/0x61f18<br />
0 225 0 6 607,824 37,538 570,286 225 0x2deb/0xf6568<br />
14,038 14,130 0 3 13,699,081 2,586,447,063 0 28,385 0x2e21/0x47d6f1<br />
4,147 13,212 0 3 5,363,836 2,572,747,982 0 17,359 0x2e40/0x118ed<br />
0 0 0 4 133,826 37,921 95,905 0 0x2e40/0x323af<br />
31,744 33,396 0 3 22,247,625 2,567,250,320 0 65,140 0x2e45/0x256f78<br />
9 0 0 6 290,601 55,855 234,746 9 0x2e45/0x29dea1<br />
16 25,020 0 4 10,475,420 2,544,712,094 0 25,039 0x2e5b/0x311fbd<br />
1,682 19,008 0 3 8,507,134 2,534,236,674 0 20,690 0x2e5f/0x1a583b<br />
12 0 0 6 354,107 41,047 313,060 12 0x2e5f/0x1fbf76<br />
6,849 54,863 0 3 20,176,980 2,525,375,433 0 61,719 0x2e7d/0x2272ca<br />
30,012 67,588 0 5 28,465,075 27,606,214 858,861 97,600 0x2eb6/0xb06fd<br />
14,048 16,912 0 2 9,587,676 2,476,733,378 0 66,881 0x2ee1/0x4bc59<br />
15,183 12,704 0 6 11,270,153 6,834,792 4,435,361 27,887 0x2ef4/0x181de2<br />
23,012 39,873 0 7 18,610,316 2,455,875,549 0 62,948 0x2f1a/0x2e96e<br />
25,472 49,151 0 6 22,681,935 2,437,265,233 0 74,623 0x2f5a/0x2bd5bd<br />
1,031 60,319 0 6 20,425,359 2,414,583,298 0 61,350 0x2f66/0x32534c<br />
48,185 83,225 0 4 42,656,335 28,813,330 13,843,005 131,410 0x2f9d/0x5adb9b<br />
16,859 32,139 0 3 18,804,637 12,310,056 6,494,581 74,600 0x2fa3/0x489e38<br />
29,050 79,391 0 8 47,784,742 37,879,192 9,905,550 108,511 0x300a/0x26d0de<br />
32 38,306 0 6 19,991,849 14,244,764 5,747,085 117,588 0x3026/0x26b107<br />
0 37 0 6 345,851 39,476 306,375 25,114 0x3026/0x2bf802<br />
17,725 34,199 0 3 19,155,357 2,264,574,525 0 51,955 0x303f/0x1f149f<br />
1,379 23,909 0 6 10,787,679 2,245,419,168 0 25,288 0x304f/0x2b197e<br />
3 12 0 5 297,677 34,135 263,542 15 0x304f/0x2fa44b<br />
17,968 47,908 0 2 17,014,500 2,234,333,812 0 65,891 0x306c/0x2162f<br />
45,151 19,691 0 3 20,592,703 18,998,392 1,594,311 64,842 0x3071/0xb216e<br />
18,246 13,219 0 3 14,900,440 2,196,726,609 0 31,514 0x308d/0x55e7c6<br />
26 12,008 0 5 7,554,365 4,234,132 3,320,233 12,034 0x30a1/0x309683<br />
35,556 69,762 0 20 31,161,221 2,174,271,804 0 105,338 0x30c2/0x424e88<br />
33,203 114,692 0 5 49,168,609 43,804,782 5,363,827 147,895 0x30fd/0x359ee9<br />
182 19,563 0 7 10,363,229 2,093,941,974 0 87,398 0x310c/0x3b29c6<br />
14 19,817 0 4 9,355,037 4,743,642 4,611,395 19,831 0x3110/0x315263<br />
46 37,708 0 8 20,678,814 10,884,622 9,794,192 41,674 0x315c/0x3bae01<br />
88 279 0 5 761,100 44,429 716,671 3,390 0x315c/0x474b0d<br />
40,709 61,572 0 4 26,837,961 2,052,783,794 0 102,607 0x3189/0x170b56<br />
42 0 0 6 814,957 77,642 737,315 42 0x3189/0x237ac3<br />
69,126 107,504 0 5 52,094,116 47,372,898 4,721,218 176,645 0x31b8/0x436ee7<br />
46 0 0 5 393,197 51,476 341,721 82,129 0x31b8/0x496ed4<br />
48,348 117,823 0 3 48,879,763 41,965,088 6,914,675 166,196 0x3201/0x3856e7<br />
16,349 39,063 0 7 17,875,388 14,606,658 3,268,730 62,982 0x3208/0x17eba3<br />
105 0 0 6 3,394,912 60,919 3,333,993 2,571 0x3208/0x4bb903<br />
217 17,310 0 7 5,776,110 1,902,493,500 0 17,617 0x3210/0xb4571<br />
36,657 27,268 0 7 22,160,771 4,617,088 17,543,683 63,925 0x3224/0x2c3df4<br />
24,153 42,022 0 1 22,019,603 1,874,556,619 0 114,113 0x3235/0x4b0f07<br />
16 18,174 0 7 7,706,321 1,852,537,016 0 18,190 0x324e/0x280f58<br />
120 24,379 0 7 10,458,169 1,844,830,695 0 24,499 0x3266/0x2f0d11<br />
14,236 34,916 0 3 17,998,326 12,723,415 5,274,911 49,152 0x327b/0x108207<br />
79 0 0 5 673,666 37,699 635,967 2,808 0x327b/0x1ac989<br />
6,624 22,958 0 3 11,999,224 1,815,700,534 0 29,650 0x3290/0x394b01<br />
19,965 40,810 0 8 17,269,398 1,803,701,310 0 60,775 0x32a7/0xfa097<br />
7,517 58,175 0 5 21,772,528 1,786,431,912 0 65,692 0x32de/0x2aac87<br />
9,415 18,600 0 6 10,786,946 1,764,659,384 0 28,015 0x32e1/0x36ae89<br />
51 0 0 6 622,591 52,177 570,414 51 0x32e1/0x402e88<br />
30,139 66,899 0 4 30,497,693 1,753,249,847 0 97,065 0x3315/0x47c6a5<br />
6 41,284 0 1 16,342,999 12,394,744 3,948,255 41,290 0x3320/0xff97c<br />
21,091 17,059 0 6 11,525,646 1,706,409,155 0 38,984 0x332b/0x27410a<br />
0 94 0 4 892,946 34,897 858,049 94 0x332b/0x34e11c<br />
3 52,035 0 3 18,160,444 1,693,990,563 0 52,132 0x336d/0x18cf58<br />
58 27,035 0 6 8,947,956 1,675,830,119 0 27,093 0x3376/0x8c1cc<br />
3 66 0 4 501,044 49,007 452,037 69 0x3376/0x106700<br />
27,134 44,130 0 4 22,994,532 1,666,381,119 0 71,330 0x339f/0x3e1864<br />
0 108 0 4 439,894 35,044 404,850 108 0x339f/0x44ceba<br />
17 51,506 0 3 20,284,195 1,642,946,693 0 51,631 0x33a6/0x4924dd<br />
28 16,826 0 2 9,954,053 1,622,662,498 0 16,854 0x33b6/0x487162<br />
2,079 43,231 0 2 20,850,856 13,682,453 7,168,403 45,310 0x33d2/0x556d0a<br />
2 30,027 0 3 14,418,091 8,924,719 5,493,372 30,743 0x33d9/0x40b5<br />
0 1,157 0 6 5,510,215 155,394 5,354,821 3,103 0x33d9/0x5454fc<br />
31,517 31,808 0 3 18,110,480 1,571,929,283 0 64,085 0x33ee/0x37800c<br />
30,598 27,193 0 4 18,121,300 12,201,650 5,919,650 57,791 0x3402/0x1ad560<br />
38,905 34,440 0 4 18,443,706 15,100,893 3,342,813 78,249 0x340c/0x3161a<br />
51,887 95,209 0 4 40,262,214 29,791,799 10,470,415 152,283 0x343b/0x71660<br />
1 28,686 0 7 10,081,465 1,476,991,583 0 142,625 0x344c/0x85499<br />
28,401 13,707 0 3 10,169,877 1,466,910,118 0 42,108 0x3457/0xaec2e<br />
5 16,531 0 6 14,562,193 4,402,715 10,159,478 16,536 0x345b/0x50893f<br />
0 15,920 0 7 9,514,322 1,442,178,048 0 19,604 0x3460/0x492011<br />
4 16,264 0 6 10,131,895 1,432,663,726 0 16,268 0x3465/0x4b2348<br />
30,231 28,986 0 4 18,938,103 14,093,138 4,844,965 59,217 0x347c/0x3aef3f<br />
14 0 0 5 512,006 34,432 477,574 120 0x347c/0x42bf45<br />
20,678 38,294 0 3 17,101,301 1,403,081,722 0 58,975 0x3497/0x16843a<br />
9,869 25,099 0 5 12,548,707 1,385,980,421 0 34,968 0x34b3/0x3d681d<br />
36,465 60,605 0 5 26,361,794 24,558,695 1,803,099 97,070 0x34d6/0x5e45f<br />
36 23,345 0 3 9,621,974 6,699,792 2,922,182 23,911 0x34d9/0x1fb5<br />
25,436 49,579 0 7 20,880,498 18,608,524 2,271,974 75,850 0x34df/0xd8f27<br />
32 1,485 0 6 4,197,555 413,166 3,784,389 1,535 0x34df/0x4d9bda<br />
31,535 59,968 0 5 25,248,912 24,728,797 520,115 91,849 0x3512/0x51cea<br />
1,601 24,378 0 6 10,268,411 8,200,696 2,067,715 26,160 0x351b/0x93565<br />
14,147 23,162 0 8 11,300,331 1,276,852,570 0 37,400 0x3525/0x1d0cd0<br />
14,204 20,662 0 3 13,299,117 1,265,552,239 0 34,866 0x3532/0x4f63fd<br />
17,514 24,728 0 3 16,797,576 12,513,187 4,284,389 42,242 0x3553/0x1e8685<br />
7,303 46,343 0 7 23,094,370 14,762,461 8,331,909 54,344 0x3580/0x4dbde7<br />
45,620 41,812 0 7 27,514,220 21,511,990 6,002,230 90,879 0x359c/0x27cfd3<br />
16,861 12,552 0 3 9,619,073 1,184,846,956 0 104,424 0x35a0/0x21ffd4<br />
0 271 0 6 3,731,928 93,043 3,638,885 271 0x35a0/0x5af1ac<br />
30,860 48,187 0 7 26,338,978 24,109,627 2,229,351 79,097 0x35ab/0x2314ce<br />
39,031 24,255 0 4 20,202,284 1,145,156,977 0 63,301 0x35c4/0x262afa<br />
2,576 46,002 0 6 19,184,048 5,580,647 13,603,401 48,578 0x35dc/0x19b7aa<br />
2 0 0 6 174,028 33,089 140,939 31,536 0x35dc/0x1c5f76<br />
111,528 77,161 0 7 62,341,581 50,549,489 11,792,092 188,690 0x360c/0x401a43<br />
28,697 25,910 0 6 18,246,350 13,855,848 4,390,502 56,636 0x3618/0x255811<br />
4,335 12,127 0 6 9,004,926 5,130,460 3,874,466 18,985 0x3621/0x16290f<br />
18,413 8,607 0 5 8,878,516 7,701,363 1,177,153 27,027 0x3626/0x50c43<br />
0 2,910 0 6 6,228,723 848,601 5,380,122 2,910 0x3626/0x641736<br />
28,784 22,689 0 3 21,119,867 14,626,185 6,493,682 52,178 0x362a/0x752db1<br />
41,676 62,528 0 4 34,112,988 29,303,765 4,809,223 105,238 0x363e/0x1b598d<br />
17 735 0 5 12,521,216 31,105 12,490,111 1,300 0x3643/0x41d20d<br />
7,802 34,494 0 7 33,127,169 7,514,191 25,612,978 43,003 0x3659/0x71898e<br />
0 8,734 0 5 3,377,408 900,015,281 0 25,130 0x365c/0xc7c0e<br />
114 9,858 0 7 15,782,161 650,578 15,131,583 9,972 0x365f/0x64b69f<br />
1 11,516 0 7 5,378,418 880,855,712 0 19,548 0x3664/0x1e3191<br />
7 22,005 0 7 8,922,863 6,691,187 2,231,676 22,012 0x3668/0xdc200<br />
13 13,409 0 6 15,952,421 360,887 15,591,534 14,391 0x3671/0x6895a5<br />
1,749 8,568 0 3 3,660,457 850,602,010 0 22,698 0x3676/0x7d9ce<br />
5,390 10,146 0 6 14,882,555 1,855,609 13,026,946 15,536 0x3688/0x525a49<br />
10,230 9,513 0 5 4,816,061 832,058,998 0 28,465 0x3691/0x34086<br />
14,698 31,907 0 7 41,080,361 36,925,183 4,155,178 46,605 0x36aa/0x13bcaf<br />
2 1,755 0 6 4,666,159 345,619 4,320,540 47,630 0x36aa/0x5aefde<br />
25,935 30,385 0 5 38,398,767 781,496,417 0 57,119 0x36c5/0x42810d<br />
9,478 14,696 0 7 21,102,013 743,097,650 0 24,174 0x36e1/0x5351ca<br />
65 12,746 0 6 5,610,587 3,858,013 1,752,574 12,811 0x36ed/0x1057a5<br />
5,806 9,341 0 6 10,353,968 2,690,013 7,663,955 15,343 0x36fc/0x15be55<br />
5 0 0 6 4,894,393 33,744 4,860,649 5,931 0x36fc/0x606d0e<br />
28,754 58,230 0 18 28,401,618 701,136,689 0 86,988 0x3708/0x480960<br />
0 15,897 0 6 10,890,931 3,098,266 7,792,665 15,897 0x370a/0x55a193<br />
126 10,927 0 6 26,068,972 2,214,107 23,854,865 16,100 0x372d/0x19a5ff<br />
9,837 16,155 0 6 11,096,409 6,346,816 4,749,593 29,696 0x3732/0x2a60d8<br />
128 15,250 0 6 8,591,606 3,747,801 4,843,805 16,046 0x3734/0x14e34e<br />
123 28,336 0 6 16,047,724 5,161,792 10,885,932 31,737 0x3736/0x712b3a<br />
24 8,534 0 5 3,423,605 1,552,888 1,870,717 19,982 0x3740/0xcd22f<br />
533 0 0 6 20,325,930 275,776 20,050,154 3,884 0x3742/0x11cb59<br />
0 0 0 4 72,242 36,230 36,012 0 0x3742/0x12e58b<br />
26,449 16,768 0 3 20,777,536 576,217,652 0 43,217 0x3756/0x1ec2cb<br />
11,493 23,349 0 3 21,836,979 13,435,005 8,401,974 34,842 0x375c/0x3aca7e<br />
309 32,239 0 6 17,558,766 9,588,731 7,970,035 56,790 0x3761/0x158a6c<br />
12 26,562 0 7 12,407,639 6,425,279 5,982,360 27,257 0x3779/0x3a4743<br />
6,845 41,908 0 7 33,072,537 22,775,381 10,297,156 51,339 0x379b/0x69295c<br />
6,778 29,990 0 6 25,664,453 17,110,934 8,553,519 41,294 0x37a8/0x2701a1<br />
5 16,509 0 9 19,648,397 8,979,547 10,668,850 44,387 0x37ae/0x21a42e<br />
116,569 137,423 0 12 129,250,694 72,386,145 56,864,549 267,033 0x37e2/0x163534<br />
13 17,546 0 6 28,042,727 34,685 28,008,042 187,464 0x37e7/0x90ee1b<br />
33,326 30,872 0 6 43,125,925 9,805,827 33,320,098 81,756 0x37ec/0x280a40<br />
3 5,332 0 5 19,873,538 37,653 19,835,885 30,103 0x37ef/0x261c42<br />
17 2,306 0 6 19,166,423 244,987 18,921,436 7,657 0x37f1/0x196419<br />
10,753 20,277 0 6 40,648,779 11,079,498 29,569,281 32,618 0x37f6/0x234a64<br />
10 22,876 0 6 18,391,375 164,634 18,226,741 49,034 0x37f8/0xabeb3<br />
30 34,377 0 7 32,006,385 1,775,081 30,231,304 56,545 0x37fb/0x295c24<br />
4 23,090 0 7 38,624,657 2,288,459 36,336,198 52,101 0x37ff/0x145fb5<br />
6 8,689 0 6 27,108,848 949,959 26,158,889 24,449 0x3801/0x80d8a5<br />
6 5,117 0 7 18,644,436 55,620 18,588,816 11,221 0x3803/0x6c2979<br />
837 0 0 6 6,661,670 288,422 6,373,248 5,956 0x3804/0x39391f<br />
5,863 6,917 0 0 3,705,888 3,705,888 0 12,780 0x3804/0x71c53f<br />
</DbPrintLog><br />
<br />
* This prints some statistics about the database. In particular the two rows called key bytes and data bytes are relevant. For the following step you need the values from the avg column - i.e. 38 and 219 in this case. You also need the value of “Total number of entries” (14543762 in this case). Now we take those values and put them into this command:<br />
<br />
[root@dc042 meta]# java -cp /usr/share/dcache/classes/je-*.jar com.sleepycat.je.util.DbCacheSize -records 14543762 -key 38 -data 219<br />
<br />
=== Environment Cache Overhead ===<br />
<br />
3,157,213 minimum bytes<br />
<br />
To account for JE daemon operation and record locks,<br />
a significantly larger amount is needed in practice.<br />
<br />
=== Database Cache Size ===<br />
<br />
Number of Bytes Description<br />
--------------- -----------<br />
1,195,759,832 Internal nodes only<br />
5,124,232,392 Internal nodes and leaf nodes<br />
<br />
* For further information see the DbCacheSize javadoc.<br />
<br />
* For this pool it tells us that we need a bit above 1195 MB to keep all the internal btree nodes cached. Oracle’s recommendation is that if the database is updated often, the cache is at least big enough to contain the internal nodes. Obviously you want to make it a bit bigger to leave room for it to grow. One could take this and configure the cache using the je.maxMemory setting. If the size is lower than the 20% of the max heap size you already use, then you don’t need to do anything (I do not suggest lowering it further).<br />
<br />
* Since this pool has 6000m of JAVA heap max space, it means that 20% = 1200MB, which is exactly cache for Internal btree nodes. So we will not touch this.<br />
<br />
* There is one caveat though: If you increase the cache size, less free space is left on the heap. If the pool is pushed to the limit, this may actually slow down the pool as garbage collection overhead increases. You need to ensure that enough space is left in addition to the cache (possibly by increasing the max heap size). An alternative to adjusting je.maxMemory is of course to adjust the max heap size. If you make it large enough so that 20% is enough to cache the internal btree nodes, then all is well. This may however mean you assign significantly more memory to the pool than it really needs. Yeah, there are lots of things to consider.<br />
<br />
* A little bonus: There is also this command:<br />
<br />
[root@dc042 meta]# java -cp /usr/share/dcache/classes/je-*.jar com.sleepycat.je.util.DbSpace -h .<br />
File Size (KB) % Used<br />
-------- --------- ------<br />
00001857 9765 41<br />
00001f2a 9765 43<br />
00001f2f 9765 39<br />
00001f72 9765 44<br />
0000216d 9765 40<br />
000021cb 9765 40<br />
0000226b 9765 45<br />
000022e8 9763 38<br />
000022fa 9765 39<br />
00002390 9765 38<br />
0000241d 9765 49<br />
0000252c 9765 51<br />
0000254c 9765 47<br />
00002567 9765 39<br />
000025b7 9765 42<br />
000025db 9765 42<br />
000025e5 9765 56<br />
0000265f 9765 39<br />
00002678 9765 61<br />
0000268d 9762 48<br />
00002692 9765 74<br />
000026be 9765 48<br />
000026cf 9765 40<br />
00002700 9765 41<br />
00002706 9765 38<br />
00002718 9765 56<br />
00002747 9765 44<br />
0000274d 9765 48<br />
000027db 9765 62<br />
00002855 9765 39<br />
0000285c 9765 39<br />
00002882 9765 41<br />
0000289d 9765 54<br />
000028ba 9765 52<br />
000028c1 9765 53<br />
000028d0 9765 41<br />
000028d7 9765 41<br />
000028e7 9765 39<br />
000028ed 9765 39<br />
00002926 9765 39<br />
0000292b 9765 43<br />
00002945 9765 41<br />
00002952 9765 41<br />
00002958 9765 62<br />
00002959 9764 38<br />
00002962 9765 53<br />
00002963 9765 47<br />
0000297d 9765 51<br />
0000297e 9765 41<br />
000029b3 9765 45<br />
000029c7 9765 56<br />
000029ce 9765 42<br />
000029d3 9765 43<br />
000029d9 9765 54<br />
000029fb 9765 42<br />
00002a00 9765 54<br />
00002a01 9765 39<br />
00002a06 9765 39<br />
00002a20 9765 45<br />
00002a26 9765 61<br />
00002a2b 9765 69<br />
00002a30 9765 65<br />
00002a44 9765 46<br />
00002a4f 9765 50<br />
00002a53 9765 64<br />
00002a54 9765 47<br />
00002a58 9765 66<br />
00002a59 9765 43<br />
00002a5e 9765 56<br />
00002a63 9765 44<br />
00002a69 9765 47<br />
00002a8f 9765 39<br />
00002aac 9765 54<br />
00002aba 9765 42<br />
00002abb 9765 44<br />
00002ae0 9763 38<br />
00002af3 9765 45<br />
00002b01 9765 41<br />
00002b08 9765 65<br />
00002b0e 9765 41<br />
00002b24 9765 45<br />
00002b28 9765 61<br />
00002b29 9765 41<br />
00002b3c 9765 40<br />
00002b3d 9765 40<br />
00002b41 9765 47<br />
00002b4c 9765 46<br />
00002b52 9765 57<br />
00002b64 9765 38<br />
00002b87 9765 38<br />
00002b8e 9765 47<br />
00002b8f 9760 38<br />
00002bad 9765 42<br />
00002bbf 9765 44<br />
00002bc6 9765 38<br />
00002be9 9765 47<br />
00002c00 9765 42<br />
00002c06 9765 69<br />
00002c0c 9765 55<br />
00002c13 9765 38<br />
00002c14 9765 38<br />
00002c2a 9765 41<br />
00002c2b 9765 41<br />
00002c30 9765 54<br />
00002c4c 9765 39<br />
00002c53 9765 45<br />
00002c5f 9765 44<br />
00002c7c 9765 38<br />
00002c8f 9765 49<br />
00002c95 9765 38<br />
00002ca4 9764 50<br />
00002ca9 9765 42<br />
00002cad 9763 41<br />
00002cc4 9765 39<br />
00002ccf 9765 43<br />
00002cd3 9765 45<br />
00002cf3 9765 48<br />
00002d19 9765 38<br />
00002d20 9765 38<br />
00002d3a 9765 40<br />
00002d3b 9765 41<br />
00002d41 9765 43<br />
00002d51 9765 39<br />
00002d59 9765 40<br />
00002d79 9765 62<br />
00002d8f 9765 39<br />
00002deb 9765 40<br />
00002e21 9765 39<br />
00002e40 9765 41<br />
00002e41 9765 45<br />
00002e45 9765 40<br />
00002e5b 9765 58<br />
00002e5f 9765 49<br />
00002e60 9765 67<br />
00002e7d 9765 48<br />
00002e95 9765 38<br />
00002eb1 9762 57<br />
00002eb6 9765 53<br />
00002ee1 9765 38<br />
00002ef4 9763 43<br />
00002ef8 9765 41<br />
00002f1a 9765 53<br />
00002f25 9765 65<br />
00002f5a 9765 52<br />
00002f5f 9765 63<br />
00002f66 9765 41<br />
00002f71 9765 42<br />
00002f79 9765 61<br />
00002f87 9765 49<br />
00002f9d 9765 41<br />
00002f9e 9765 42<br />
00002fa3 9765 38<br />
00002fa4 9765 46<br />
00002fb1 9762 55<br />
00002fc1 9765 41<br />
00002fde 9765 38<br />
0000300a 9765 43<br />
0000300e 9762 47<br />
00003026 9765 55<br />
00003027 9765 57<br />
0000303f 9765 63<br />
0000304f 9765 48<br />
00003056 9765 44<br />
0000306c 9765 48<br />
0000306d 9765 42<br />
00003071 9765 45<br />
0000308d 9765 39<br />
000030a1 9765 54<br />
000030a5 9765 51<br />
000030ab 9765 60<br />
000030c2 9765 46<br />
000030d5 9765 43<br />
000030dd 9765 38<br />
000030e3 9765 57<br />
000030e4 9765 55<br />
000030fd 9765 48<br />
0000310c 9764 42<br />
00003110 9765 46<br />
00003119 9765 47<br />
0000315c 9765 48<br />
0000315d 9765 54<br />
00003185 9765 56<br />
00003189 9765 52<br />
0000318a 9765 58<br />
0000319a 9765 46<br />
0000319f 9764 39<br />
000031b5 9764 53<br />
000031b8 9765 40<br />
000031b9 9765 74<br />
000031e2 9765 65<br />
000031fa 9765 58<br />
000031fb 9765 52<br />
00003201 9765 50<br />
00003202 9764 44<br />
00003208 9765 42<br />
00003210 9763 40<br />
00003218 9765 40<br />
00003224 9765 51<br />
00003229 9765 48<br />
00003235 9765 40<br />
0000324e 9765 46<br />
00003266 9765 41<br />
0000326c 9764 47<br />
0000327b 9765 55<br />
00003290 9765 48<br />
00003294 9765 56<br />
000032a7 9765 69<br />
000032ad 9765 62<br />
000032de 9765 47<br />
000032e1 9765 46<br />
000032e2 9765 71<br />
000032e9 9765 45<br />
00003315 9765 42<br />
00003316 9764 54<br />
00003320 9765 43<br />
0000332b 9765 52<br />
0000332c 9765 76<br />
0000336d 9765 69<br />
00003376 9765 72<br />
00003377 9765 46<br />
0000339f 9765 43<br />
000033a0 9765 75<br />
000033a6 9765 46<br />
000033b6 9765 40<br />
000033c0 9764 72<br />
000033d2 9765 38<br />
000033d3 9764 45<br />
000033d9 9765 42<br />
000033da 9765 44<br />
000033ee 9765 41<br />
000033fb 9764 38<br />
00003402 9765 42<br />
00003403 9765 53<br />
0000340c 9765 76<br />
00003428 9765 67<br />
00003431 9765 51<br />
00003436 9765 88<br />
0000343b 9765 85<br />
0000344c 9765 48<br />
00003457 9765 46<br />
0000345b 9765 46<br />
00003460 9765 50<br />
00003465 9765 45<br />
00003466 9762 40<br />
0000347c 9765 54<br />
0000347d 9765 61<br />
00003497 9765 63<br />
000034b3 9765 64<br />
000034bd 9765 54<br />
000034c9 9764 71<br />
000034d6 9765 65<br />
000034d9 9765 86<br />
000034da 9765 46<br />
000034df 9765 51<br />
000034e0 9764 60<br />
00003504 9762 80<br />
00003512 9761 76<br />
0000351b 9765 65<br />
00003525 9765 59<br />
00003532 9765 52<br />
00003540 9764 43<br />
00003553 9765 80<br />
00003558 9765 54<br />
00003580 9762 49<br />
00003586 9765 41<br />
00003587 9763 50<br />
0000359c 9765 40<br />
000035a0 9765 46<br />
000035a1 9765 37<br />
000035a6 9765 96<br />
000035ab 9765 52<br />
000035ac 9765 39<br />
000035c4 9765 61<br />
000035d0 9765 89<br />
000035dc 9765 74<br />
000035dd 9765 43<br />
000035f1 9765 43<br />
000035fd 9765 37<br />
00003604 9765 47<br />
00003609 9764 40<br />
0000360c 9765 59<br />
0000360d 9765 37<br />
00003618 9763 42<br />
00003621 9765 42<br />
00003626 9765 42<br />
00003627 9765 41<br />
0000362a 9765 38<br />
0000362b 9765 73<br />
00003630 9765 68<br />
00003631 9764 42<br />
0000363e 9763 39<br />
00003643 9765 43<br />
00003646 9765 38<br />
0000364c 9762 40<br />
00003659 9765 47<br />
0000365c 9764 44<br />
0000365f 9765 45<br />
00003664 9765 69<br />
00003668 9764 37<br />
00003671 9765 43<br />
00003676 9765 39<br />
00003688 9765 37<br />
00003691 9765 48<br />
00003692 9764 37<br />
00003695 9762 38<br />
000036a3 9764 50<br />
000036aa 9765 47<br />
000036ab 9765 49<br />
000036c2 9765 64<br />
000036c3 9764 57<br />
000036c5 9765 40<br />
000036da 9764 40<br />
000036e1 9765 43<br />
000036ed 9762 39<br />
000036fc 9765 41<br />
000036fd 9765 59<br />
00003701 9764 85<br />
00003708 9765 43<br />
0000370a 9763 49<br />
00003728 9762 41<br />
0000372a 9763 55<br />
0000372d 9765 51<br />
00003732 9763 48<br />
00003734 9765 60<br />
00003736 9765 42<br />
00003740 9765 48<br />
00003741 9765 39<br />
00003742 9765 38<br />
00003749 9764 40<br />
00003756 9763 39<br />
00003758 9762 46<br />
0000375c 9765 67<br />
0000375d 9765 37<br />
00003761 9765 63<br />
00003779 9765 55<br />
00003785 9765 43<br />
00003798 9763 60<br />
0000379b 9765 41<br />
0000379e 9765 37<br />
000037a5 9764 39<br />
000037a8 9764 48<br />
000037ab 9765 38<br />
000037ae 9765 54<br />
000037b3 9764 42<br />
000037b6 9761 36<br />
000037be 9763 46<br />
000037c1 9765 69<br />
000037c3 9765 36<br />
000037c4 9764 37<br />
000037ce 9762 50<br />
000037d1 9761 45<br />
000037d3 9765 45<br />
000037dd 9765 45<br />
000037df 9765 50<br />
000037e0 9763 39<br />
000037e2 9765 13<br />
000037e3 9763 18<br />
000037e7 9765 49<br />
000037e8 9763 41<br />
000037e9 9765 42<br />
000037ea 9763 40<br />
000037eb 9765 39<br />
000037ec 9761 39<br />
000037ed 9763 41<br />
000037ef 9763 36<br />
000037f0 9764 36<br />
000037f1 9764 39<br />
000037f3 9765 41<br />
000037f4 9761 53<br />
000037f5 9765 36<br />
000037f6 9765 54<br />
000037f7 9763 53<br />
000037f8 9765 63<br />
000037f9 9765 60<br />
000037fa 9765 70<br />
000037fb 9765 76<br />
000037fc 9765 72<br />
000037fd 9765 73<br />
000037fe 9765 83<br />
000037ff 9765 87<br />
00003800 9765 89<br />
00003801 9762 86<br />
00003802 9763 88<br />
00003803 9764 90<br />
00003804 8110 76<br />
TOTALS 3757891 49<br />
<br />
* It will tell you for each of the database files (the log segments) how big it is and what the utilization is. The Berkeley DB will try to keep the total utilization above 50%, but for a small pool like this one it cannot do it. I figure that you may find this interesting to know after reading about utilization above.<br />
<br />
* The log segment size and the utilization goal can be adjusted too, but I cannot give any sound advice on whether that’s a good idea and how to determine good values.<br />
<br />
* All of the above is only relevant if you use the Berkeley DB backend for pools.<br />
<br />
== Special thanks ==<br />
* This documentation has been deployed thanks to Gerd Behrmann, you can find more in at: https://gist.github.com/gbehrmann/df1abbe45c03356eff2e<br />
* Read more about this at: <br />
:* https://backstage.forgerock.com/knowledge/kb/book/b74223337/a49979000<br />
:* http://docs.oracle.com/cd/E17277_02/html/java/com/sleepycat/je/EnvironmentConfig.html</div>Mcaubethttps://pwiki.pic.es/index.php?title=DCACHE_CHECK_POOL_BTREE_MEMORY&diff=230DCACHE CHECK POOL BTREE MEMORY2017-10-06T13:29:43Z<p>Mcaubet: /* Special thanks */</p>
<hr />
<div>= For the MoD =<br />
* Open a ticket to incidents@support.pic.es with the alarm. <br />
* This alarm only affects the duration of the pool start up, nothing has to be done until the next working day where this can be fixed by dCache Administrators.<br />
* Same information as [[DCACHE CHECK POOL METADATA]] as both alarms are strictly related.<br />
<br />
= For the Admin =<br />
<br />
* Berkely DB should be tunned according to the number of meta files that a pool contains. If this not correctly adapted, pool start up will may need several hours to be fully available. Gerd explained how to tune up a pool according to this number. Below I show a modified explanation of what he said, adapting the explanation to the ATLAS pools.<br />
<br />
* The default je.log.fileCacheSize is 100. This could be increased by creating a file je.properties and adding the 'je.log.fileCacheSize=100'.<br />
<br />
* Also, other relevant settings are:<br />
<br />
je.maxMemory=<br />
je.maxMemoryPercent=20<br />
<br />
* These are equivalent. The former sets the number of bytes used for the btree cache, while the latter defines it as a percentage of the max heap size. E.g. if dcache.java.memory.heap is set to 2048m, then 410 MB is used for the btree cache.<br />
<br />
* The question is then how to determine a good size for the cache. There are some hidden utilities you can use to do this.<br />
<br />
* Below I show an example for dc042_1:<br />
<br />
[root@dc042 meta]# java -cp /usr/share/dcache/classes/je-6.3.8.jar com.sleepycat.je.util.DbPrintLog -h . -S<br />
<DbPrintLog><br />
Log statistics:<br />
type total provisional total min max avg entries<br />
count count bytes bytes bytes bytes as % of log<br />
MapLN 1,324 0 12,617,573 76 11,985 9,529 0.3<br />
NameLN 4 0 141 31 38 35 0<br />
FileSummaryLN 231,140 0 55,644,337 24 49,701 240 1.4<br />
IN 84,083 1,080 296,942,059 45 5,709 3,531 7.7<br />
BIN 114,865 114,865 377,569,148 44 6,675 3,287 9.8<br />
DbTree 318 0 1,034,997 2,680 4,136 3,254 0<br />
Commit 3,376,012 0 114,784,408 34 34 34 3<br />
CkptStart 158 0 4,899 30 33 31 0<br />
CkptEnd 226 0 22,560 67 186 99 0<br />
Trace 2,506 0 525,597 48 402 209 0<br />
FileHeader 385 0 14,630 38 38 38 0<br />
DEL_LN_TX 748,055 0 51,409,527 57 69 68 1.3<br />
INS_LN_TX 619,747 0 192,298,301 96 1,443 310 5<br />
UPD_LN_TX 2,008,226 0 398,854,718 32 1,919 198 10.4<br />
UPD_LN 6,636,173 0 1,997,042,458 18 1,819 300 51.9<br />
NewBINDelta 720,540 720,540 348,480,254 55 1,782 483 9.1<br />
key bytes 10,012,201 387,254,450 1 61 38 (10.1)<br />
data bytes 9,264,146 2,036,632,128 1 1,848 219 (52.9)<br />
<br />
Total bytes in portion of log read: 3,847,245,607<br />
Total number of entries: 14,543,762<br />
<br />
Per checkpoint interval info:<br />
lnTxn ln mapLNTxn mapLN end to end end to start start to end maxLNReplay ckptEnd<br />
273 63,082 0 10 30,455,407 3,857,456,063 0 63,355 0x1f2f/0x6f2ef<br />
5,168 130,892 0 24 39,782,369 3,827,000,656 0 136,060 0x226b/0x3a0d0<br />
27,098 132,905 0 15 50,120,541 16,509,395 33,611,146 160,003 0x252c/0x577ad<br />
128 32,773 0 3 9,904,160 3,737,097,746 0 134,154 0x254c/0x4014d<br />
82 31,341 0 12 10,576,583 3,727,193,586 0 31,423 0x2567/0xccd94<br />
0 72 0 4 711,899 51,780 660,119 72 0x2567/0x17aa6f<br />
36,445 61,509 0 0 28,499,838 3,715,905,104 0 97,954 0x25e5/0xc66d<br />
2 108 0 6 1,564,919 56,871 1,508,048 110 0x25e5/0x18a764<br />
10,732 50,299 0 5 18,877,564 3,685,840,347 0 61,055 0x2678/0x786e0<br />
33 190 0 6 1,627,301 94,279 1,533,022 223 0x2678/0x205b85<br />
8,277 84,136 0 3 28,217,133 17,808,699 10,408,434 92,434 0x26be/0x52732<br />
5,401 25,394 0 7 9,824,151 3,637,118,349 0 61,029 0x26cf/0x27849<br />
49,026 92,572 0 3 41,592,807 3,627,294,198 0 141,598 0x2747/0x1ac630<br />
2 46 0 6 357,170 45,107 312,063 48 0x2747/0x203962<br />
20,496 39,154 0 3 18,591,447 3,585,344,221 0 59,654 0x27db/0xabb39<br />
0 154 0 6 1,187,463 83,676 1,103,787 154 0x27db/0x1cd9c0<br />
17,107 49,468 0 3 18,905,722 3,565,565,311 0 66,583 0x285c/0xc273a<br />
58 0 0 6 392,501 62,920 329,581 58 0x285c/0x12246f<br />
113,890 201,969 0 17 79,488,418 3,546,267,088 0 315,874 0x28ed/0xa5611<br />
22,147 98,579 0 8 40,657,515 36,319,235 4,338,280 120,726 0x2952/0x145e7c<br />
0 122 0 6 1,296,043 70,361 1,225,682 28,367 0x2952/0x282527<br />
19,570 65,944 0 5 27,570,660 25,251,492 2,319,168 85,520 0x2962/0x3138b<br />
37,714 28,889 0 3 21,505,514 19,337,884 2,167,630 66,613 0x297d/0x1a0c75<br />
3 332 0 6 2,437,155 124,630 2,312,525 479 0x297d/0x3f3c98<br />
41,458 62,406 0 3 26,385,897 3,373,311,783 0 103,904 0x29c7/0x81701<br />
0 46 0 6 1,404,642 46,045 1,358,597 46 0x29c7/0x1d85e3<br />
41,443 87,747 0 3 39,909,271 14,624,816 25,284,455 129,198 0x29fb/0x1c237a<br />
0 123 0 5 1,185,318 68,652 1,116,666 79,470 0x29fb/0x2e39a0<br />
9,199 12,698 0 4 9,927,451 3,304,426,655 0 21,899 0x2a00/0x2d1e3b<br />
8 0 0 5 118,374 35,364 83,010 8 0x2a00/0x2eeca1<br />
122,697 96,787 0 3 66,966,017 3,294,380,830 0 219,488 0x2a44/0xa122<br />
53,436 12,002 0 3 21,830,901 3,227,414,813 0 65,438 0x2a53/0x1c9117<br />
20 167 0 6 1,308,679 85,964 1,222,715 187 0x2a53/0x30891e<br />
18,552 38,805 0 6 17,501,977 3,204,275,233 0 57,381 0x2a58/0xa6b37<br />
9 105 0 4 678,297 87,871 590,426 114 0x2a58/0x14c4d0<br />
25,434 36,231 0 10 19,064,711 3,186,094,959 0 61,675 0x2a5e/0x67f57<br />
15 159 0 6 1,467,746 83,228 1,384,518 174 0x2a5e/0x1ce4b9<br />
12,236 14,099 0 3 8,375,785 3,165,562,502 0 26,358 0x2a63/0x41c22<br />
6 184 0 6 1,058,988 89,948 969,040 190 0x2a63/0x1444ce<br />
71,844 24,018 0 3 29,188,061 27,937,328 1,250,733 95,870 0x2aac/0x7e12b<br />
9,407 25,608 0 7 11,016,096 3,126,939,668 0 35,029 0x2aba/0x17624b<br />
20 110 0 6 748,510 82,108 666,402 130 0x2aba/0x22ce29<br />
64,997 49,072 0 15 27,822,548 27,438,951 383,597 114,142 0x2af3/0x1947d<br />
10,534 23,197 0 6 12,103,837 3,087,352,514 0 33,737 0x2b01/0x21ae9a<br />
122 0 0 32 879,716 489,740 389,976 122 0x2b01/0x2f1afe<br />
60,029 128,431 0 24 58,556,679 56,338,979 2,217,700 188,485 0x2b3c/0x191505<br />
15 52 0 6 530,374 53,795 476,579 231 0x2b3c/0x212ccb<br />
48,799 11,045 0 4 18,178,427 17,837,860 340,567 59,855 0x2b41/0x56146<br />
45,998 29,564 0 3 20,154,236 2,997,103,481 0 75,573 0x2b52/0x7bbc2<br />
39,570 60,757 0 15 30,273,541 2,976,949,245 0 100,327 0x2b8e/0xbe847<br />
31,631 33,780 0 13 22,348,448 16,815,306 5,533,142 65,411 0x2bad/0x2fbde7<br />
16,831 40,178 0 5 20,571,223 2,924,327,256 0 57,047 0x2bc6/0x38753e<br />
4,292 18,826 0 8 8,363,076 2,903,756,033 0 23,118 0x2be9/0x1f7b02<br />
0 254 0 6 2,043,974 119,900 1,924,074 254 0x2be9/0x3eab48<br />
30 19,902 0 4 7,727,051 2,893,348,983 0 19,954 0x2c00/0x1bfc93<br />
27,984 75,433 0 32 29,896,927 2,885,621,932 0 103,417 0x2c13/0x1a69f2<br />
40,477 36,001 0 10 19,737,838 17,591,303 2,146,535 76,478 0x2c2a/0x1669e0<br />
2 2 0 4 194,668 36,808 157,860 10 0x2c2a/0x19624c<br />
29,246 68,367 0 3 31,325,183 2,835,792,499 0 97,615 0x2c4c/0x2d9acb<br />
0 26 0 5 1,118,302 27,604 1,090,698 26 0x2c4c/0x3eab29<br />
4 19,763 0 3 7,562,665 2,803,349,014 0 19,775 0x2c53/0x197a52<br />
26 194 0 6 1,436,963 103,803 1,333,160 220 0x2c53/0x2f6775<br />
6,255 14,988 0 8 9,972,102 2,794,349,386 0 21,292 0x2c5f/0x2efa7b<br />
39,212 18,305 0 5 18,591,243 2,784,377,284 0 57,517 0x2c8f/0x197b86<br />
0 41 0 6 1,495,215 121,720 1,373,495 41 0x2c8f/0x304c35<br />
347 23,340 0 3 7,036,643 2,764,290,826 0 23,697 0x2c95/0x31498<br />
23,408 9,116 0 3 10,098,528 2,757,254,183 0 32,524 0x2ca4/0x49578<br />
45,631 27,183 0 3 19,972,142 2,747,155,655 0 72,814 0x2cad/0x428a6<br />
0 0 0 1 963,802 69 963,733 0 0x2cad/0x12dd80<br />
276 19,066 0 4 10,705,056 5,926,693 4,778,363 19,342 0x2cc4/0x1d9fa0<br />
0 25,292 0 2 10,568,825 7,122,257 3,446,568 25,292 0x2ccf/0x264d99<br />
722 25,106 0 3 9,376,076 2,704,945,830 0 26,568 0x2cd3/0x1cc865<br />
15 96 0 6 601,987 65,907 536,080 111 0x2cd3/0x25f7e8<br />
50,662 51,942 0 4 29,283,798 2,694,967,767 0 102,611 0x2d20/0x1b0a3e<br />
18,486 16,140 0 4 12,291,893 2,665,683,969 0 34,626 0x2d3a/0x3e02f3<br />
67,481 104,443 0 5 48,233,597 15,261,303 32,972,294 171,924 0x2d79/0x230ef0<br />
19,276 34,062 0 3 18,103,592 2,605,158,479 0 171,476 0x2deb/0x61f18<br />
0 225 0 6 607,824 37,538 570,286 225 0x2deb/0xf6568<br />
14,038 14,130 0 3 13,699,081 2,586,447,063 0 28,385 0x2e21/0x47d6f1<br />
4,147 13,212 0 3 5,363,836 2,572,747,982 0 17,359 0x2e40/0x118ed<br />
0 0 0 4 133,826 37,921 95,905 0 0x2e40/0x323af<br />
31,744 33,396 0 3 22,247,625 2,567,250,320 0 65,140 0x2e45/0x256f78<br />
9 0 0 6 290,601 55,855 234,746 9 0x2e45/0x29dea1<br />
16 25,020 0 4 10,475,420 2,544,712,094 0 25,039 0x2e5b/0x311fbd<br />
1,682 19,008 0 3 8,507,134 2,534,236,674 0 20,690 0x2e5f/0x1a583b<br />
12 0 0 6 354,107 41,047 313,060 12 0x2e5f/0x1fbf76<br />
6,849 54,863 0 3 20,176,980 2,525,375,433 0 61,719 0x2e7d/0x2272ca<br />
30,012 67,588 0 5 28,465,075 27,606,214 858,861 97,600 0x2eb6/0xb06fd<br />
14,048 16,912 0 2 9,587,676 2,476,733,378 0 66,881 0x2ee1/0x4bc59<br />
15,183 12,704 0 6 11,270,153 6,834,792 4,435,361 27,887 0x2ef4/0x181de2<br />
23,012 39,873 0 7 18,610,316 2,455,875,549 0 62,948 0x2f1a/0x2e96e<br />
25,472 49,151 0 6 22,681,935 2,437,265,233 0 74,623 0x2f5a/0x2bd5bd<br />
1,031 60,319 0 6 20,425,359 2,414,583,298 0 61,350 0x2f66/0x32534c<br />
48,185 83,225 0 4 42,656,335 28,813,330 13,843,005 131,410 0x2f9d/0x5adb9b<br />
16,859 32,139 0 3 18,804,637 12,310,056 6,494,581 74,600 0x2fa3/0x489e38<br />
29,050 79,391 0 8 47,784,742 37,879,192 9,905,550 108,511 0x300a/0x26d0de<br />
32 38,306 0 6 19,991,849 14,244,764 5,747,085 117,588 0x3026/0x26b107<br />
0 37 0 6 345,851 39,476 306,375 25,114 0x3026/0x2bf802<br />
17,725 34,199 0 3 19,155,357 2,264,574,525 0 51,955 0x303f/0x1f149f<br />
1,379 23,909 0 6 10,787,679 2,245,419,168 0 25,288 0x304f/0x2b197e<br />
3 12 0 5 297,677 34,135 263,542 15 0x304f/0x2fa44b<br />
17,968 47,908 0 2 17,014,500 2,234,333,812 0 65,891 0x306c/0x2162f<br />
45,151 19,691 0 3 20,592,703 18,998,392 1,594,311 64,842 0x3071/0xb216e<br />
18,246 13,219 0 3 14,900,440 2,196,726,609 0 31,514 0x308d/0x55e7c6<br />
26 12,008 0 5 7,554,365 4,234,132 3,320,233 12,034 0x30a1/0x309683<br />
35,556 69,762 0 20 31,161,221 2,174,271,804 0 105,338 0x30c2/0x424e88<br />
33,203 114,692 0 5 49,168,609 43,804,782 5,363,827 147,895 0x30fd/0x359ee9<br />
182 19,563 0 7 10,363,229 2,093,941,974 0 87,398 0x310c/0x3b29c6<br />
14 19,817 0 4 9,355,037 4,743,642 4,611,395 19,831 0x3110/0x315263<br />
46 37,708 0 8 20,678,814 10,884,622 9,794,192 41,674 0x315c/0x3bae01<br />
88 279 0 5 761,100 44,429 716,671 3,390 0x315c/0x474b0d<br />
40,709 61,572 0 4 26,837,961 2,052,783,794 0 102,607 0x3189/0x170b56<br />
42 0 0 6 814,957 77,642 737,315 42 0x3189/0x237ac3<br />
69,126 107,504 0 5 52,094,116 47,372,898 4,721,218 176,645 0x31b8/0x436ee7<br />
46 0 0 5 393,197 51,476 341,721 82,129 0x31b8/0x496ed4<br />
48,348 117,823 0 3 48,879,763 41,965,088 6,914,675 166,196 0x3201/0x3856e7<br />
16,349 39,063 0 7 17,875,388 14,606,658 3,268,730 62,982 0x3208/0x17eba3<br />
105 0 0 6 3,394,912 60,919 3,333,993 2,571 0x3208/0x4bb903<br />
217 17,310 0 7 5,776,110 1,902,493,500 0 17,617 0x3210/0xb4571<br />
36,657 27,268 0 7 22,160,771 4,617,088 17,543,683 63,925 0x3224/0x2c3df4<br />
24,153 42,022 0 1 22,019,603 1,874,556,619 0 114,113 0x3235/0x4b0f07<br />
16 18,174 0 7 7,706,321 1,852,537,016 0 18,190 0x324e/0x280f58<br />
120 24,379 0 7 10,458,169 1,844,830,695 0 24,499 0x3266/0x2f0d11<br />
14,236 34,916 0 3 17,998,326 12,723,415 5,274,911 49,152 0x327b/0x108207<br />
79 0 0 5 673,666 37,699 635,967 2,808 0x327b/0x1ac989<br />
6,624 22,958 0 3 11,999,224 1,815,700,534 0 29,650 0x3290/0x394b01<br />
19,965 40,810 0 8 17,269,398 1,803,701,310 0 60,775 0x32a7/0xfa097<br />
7,517 58,175 0 5 21,772,528 1,786,431,912 0 65,692 0x32de/0x2aac87<br />
9,415 18,600 0 6 10,786,946 1,764,659,384 0 28,015 0x32e1/0x36ae89<br />
51 0 0 6 622,591 52,177 570,414 51 0x32e1/0x402e88<br />
30,139 66,899 0 4 30,497,693 1,753,249,847 0 97,065 0x3315/0x47c6a5<br />
6 41,284 0 1 16,342,999 12,394,744 3,948,255 41,290 0x3320/0xff97c<br />
21,091 17,059 0 6 11,525,646 1,706,409,155 0 38,984 0x332b/0x27410a<br />
0 94 0 4 892,946 34,897 858,049 94 0x332b/0x34e11c<br />
3 52,035 0 3 18,160,444 1,693,990,563 0 52,132 0x336d/0x18cf58<br />
58 27,035 0 6 8,947,956 1,675,830,119 0 27,093 0x3376/0x8c1cc<br />
3 66 0 4 501,044 49,007 452,037 69 0x3376/0x106700<br />
27,134 44,130 0 4 22,994,532 1,666,381,119 0 71,330 0x339f/0x3e1864<br />
0 108 0 4 439,894 35,044 404,850 108 0x339f/0x44ceba<br />
17 51,506 0 3 20,284,195 1,642,946,693 0 51,631 0x33a6/0x4924dd<br />
28 16,826 0 2 9,954,053 1,622,662,498 0 16,854 0x33b6/0x487162<br />
2,079 43,231 0 2 20,850,856 13,682,453 7,168,403 45,310 0x33d2/0x556d0a<br />
2 30,027 0 3 14,418,091 8,924,719 5,493,372 30,743 0x33d9/0x40b5<br />
0 1,157 0 6 5,510,215 155,394 5,354,821 3,103 0x33d9/0x5454fc<br />
31,517 31,808 0 3 18,110,480 1,571,929,283 0 64,085 0x33ee/0x37800c<br />
30,598 27,193 0 4 18,121,300 12,201,650 5,919,650 57,791 0x3402/0x1ad560<br />
38,905 34,440 0 4 18,443,706 15,100,893 3,342,813 78,249 0x340c/0x3161a<br />
51,887 95,209 0 4 40,262,214 29,791,799 10,470,415 152,283 0x343b/0x71660<br />
1 28,686 0 7 10,081,465 1,476,991,583 0 142,625 0x344c/0x85499<br />
28,401 13,707 0 3 10,169,877 1,466,910,118 0 42,108 0x3457/0xaec2e<br />
5 16,531 0 6 14,562,193 4,402,715 10,159,478 16,536 0x345b/0x50893f<br />
0 15,920 0 7 9,514,322 1,442,178,048 0 19,604 0x3460/0x492011<br />
4 16,264 0 6 10,131,895 1,432,663,726 0 16,268 0x3465/0x4b2348<br />
30,231 28,986 0 4 18,938,103 14,093,138 4,844,965 59,217 0x347c/0x3aef3f<br />
14 0 0 5 512,006 34,432 477,574 120 0x347c/0x42bf45<br />
20,678 38,294 0 3 17,101,301 1,403,081,722 0 58,975 0x3497/0x16843a<br />
9,869 25,099 0 5 12,548,707 1,385,980,421 0 34,968 0x34b3/0x3d681d<br />
36,465 60,605 0 5 26,361,794 24,558,695 1,803,099 97,070 0x34d6/0x5e45f<br />
36 23,345 0 3 9,621,974 6,699,792 2,922,182 23,911 0x34d9/0x1fb5<br />
25,436 49,579 0 7 20,880,498 18,608,524 2,271,974 75,850 0x34df/0xd8f27<br />
32 1,485 0 6 4,197,555 413,166 3,784,389 1,535 0x34df/0x4d9bda<br />
31,535 59,968 0 5 25,248,912 24,728,797 520,115 91,849 0x3512/0x51cea<br />
1,601 24,378 0 6 10,268,411 8,200,696 2,067,715 26,160 0x351b/0x93565<br />
14,147 23,162 0 8 11,300,331 1,276,852,570 0 37,400 0x3525/0x1d0cd0<br />
14,204 20,662 0 3 13,299,117 1,265,552,239 0 34,866 0x3532/0x4f63fd<br />
17,514 24,728 0 3 16,797,576 12,513,187 4,284,389 42,242 0x3553/0x1e8685<br />
7,303 46,343 0 7 23,094,370 14,762,461 8,331,909 54,344 0x3580/0x4dbde7<br />
45,620 41,812 0 7 27,514,220 21,511,990 6,002,230 90,879 0x359c/0x27cfd3<br />
16,861 12,552 0 3 9,619,073 1,184,846,956 0 104,424 0x35a0/0x21ffd4<br />
0 271 0 6 3,731,928 93,043 3,638,885 271 0x35a0/0x5af1ac<br />
30,860 48,187 0 7 26,338,978 24,109,627 2,229,351 79,097 0x35ab/0x2314ce<br />
39,031 24,255 0 4 20,202,284 1,145,156,977 0 63,301 0x35c4/0x262afa<br />
2,576 46,002 0 6 19,184,048 5,580,647 13,603,401 48,578 0x35dc/0x19b7aa<br />
2 0 0 6 174,028 33,089 140,939 31,536 0x35dc/0x1c5f76<br />
111,528 77,161 0 7 62,341,581 50,549,489 11,792,092 188,690 0x360c/0x401a43<br />
28,697 25,910 0 6 18,246,350 13,855,848 4,390,502 56,636 0x3618/0x255811<br />
4,335 12,127 0 6 9,004,926 5,130,460 3,874,466 18,985 0x3621/0x16290f<br />
18,413 8,607 0 5 8,878,516 7,701,363 1,177,153 27,027 0x3626/0x50c43<br />
0 2,910 0 6 6,228,723 848,601 5,380,122 2,910 0x3626/0x641736<br />
28,784 22,689 0 3 21,119,867 14,626,185 6,493,682 52,178 0x362a/0x752db1<br />
41,676 62,528 0 4 34,112,988 29,303,765 4,809,223 105,238 0x363e/0x1b598d<br />
17 735 0 5 12,521,216 31,105 12,490,111 1,300 0x3643/0x41d20d<br />
7,802 34,494 0 7 33,127,169 7,514,191 25,612,978 43,003 0x3659/0x71898e<br />
0 8,734 0 5 3,377,408 900,015,281 0 25,130 0x365c/0xc7c0e<br />
114 9,858 0 7 15,782,161 650,578 15,131,583 9,972 0x365f/0x64b69f<br />
1 11,516 0 7 5,378,418 880,855,712 0 19,548 0x3664/0x1e3191<br />
7 22,005 0 7 8,922,863 6,691,187 2,231,676 22,012 0x3668/0xdc200<br />
13 13,409 0 6 15,952,421 360,887 15,591,534 14,391 0x3671/0x6895a5<br />
1,749 8,568 0 3 3,660,457 850,602,010 0 22,698 0x3676/0x7d9ce<br />
5,390 10,146 0 6 14,882,555 1,855,609 13,026,946 15,536 0x3688/0x525a49<br />
10,230 9,513 0 5 4,816,061 832,058,998 0 28,465 0x3691/0x34086<br />
14,698 31,907 0 7 41,080,361 36,925,183 4,155,178 46,605 0x36aa/0x13bcaf<br />
2 1,755 0 6 4,666,159 345,619 4,320,540 47,630 0x36aa/0x5aefde<br />
25,935 30,385 0 5 38,398,767 781,496,417 0 57,119 0x36c5/0x42810d<br />
9,478 14,696 0 7 21,102,013 743,097,650 0 24,174 0x36e1/0x5351ca<br />
65 12,746 0 6 5,610,587 3,858,013 1,752,574 12,811 0x36ed/0x1057a5<br />
5,806 9,341 0 6 10,353,968 2,690,013 7,663,955 15,343 0x36fc/0x15be55<br />
5 0 0 6 4,894,393 33,744 4,860,649 5,931 0x36fc/0x606d0e<br />
28,754 58,230 0 18 28,401,618 701,136,689 0 86,988 0x3708/0x480960<br />
0 15,897 0 6 10,890,931 3,098,266 7,792,665 15,897 0x370a/0x55a193<br />
126 10,927 0 6 26,068,972 2,214,107 23,854,865 16,100 0x372d/0x19a5ff<br />
9,837 16,155 0 6 11,096,409 6,346,816 4,749,593 29,696 0x3732/0x2a60d8<br />
128 15,250 0 6 8,591,606 3,747,801 4,843,805 16,046 0x3734/0x14e34e<br />
123 28,336 0 6 16,047,724 5,161,792 10,885,932 31,737 0x3736/0x712b3a<br />
24 8,534 0 5 3,423,605 1,552,888 1,870,717 19,982 0x3740/0xcd22f<br />
533 0 0 6 20,325,930 275,776 20,050,154 3,884 0x3742/0x11cb59<br />
0 0 0 4 72,242 36,230 36,012 0 0x3742/0x12e58b<br />
26,449 16,768 0 3 20,777,536 576,217,652 0 43,217 0x3756/0x1ec2cb<br />
11,493 23,349 0 3 21,836,979 13,435,005 8,401,974 34,842 0x375c/0x3aca7e<br />
309 32,239 0 6 17,558,766 9,588,731 7,970,035 56,790 0x3761/0x158a6c<br />
12 26,562 0 7 12,407,639 6,425,279 5,982,360 27,257 0x3779/0x3a4743<br />
6,845 41,908 0 7 33,072,537 22,775,381 10,297,156 51,339 0x379b/0x69295c<br />
6,778 29,990 0 6 25,664,453 17,110,934 8,553,519 41,294 0x37a8/0x2701a1<br />
5 16,509 0 9 19,648,397 8,979,547 10,668,850 44,387 0x37ae/0x21a42e<br />
116,569 137,423 0 12 129,250,694 72,386,145 56,864,549 267,033 0x37e2/0x163534<br />
13 17,546 0 6 28,042,727 34,685 28,008,042 187,464 0x37e7/0x90ee1b<br />
33,326 30,872 0 6 43,125,925 9,805,827 33,320,098 81,756 0x37ec/0x280a40<br />
3 5,332 0 5 19,873,538 37,653 19,835,885 30,103 0x37ef/0x261c42<br />
17 2,306 0 6 19,166,423 244,987 18,921,436 7,657 0x37f1/0x196419<br />
10,753 20,277 0 6 40,648,779 11,079,498 29,569,281 32,618 0x37f6/0x234a64<br />
10 22,876 0 6 18,391,375 164,634 18,226,741 49,034 0x37f8/0xabeb3<br />
30 34,377 0 7 32,006,385 1,775,081 30,231,304 56,545 0x37fb/0x295c24<br />
4 23,090 0 7 38,624,657 2,288,459 36,336,198 52,101 0x37ff/0x145fb5<br />
6 8,689 0 6 27,108,848 949,959 26,158,889 24,449 0x3801/0x80d8a5<br />
6 5,117 0 7 18,644,436 55,620 18,588,816 11,221 0x3803/0x6c2979<br />
837 0 0 6 6,661,670 288,422 6,373,248 5,956 0x3804/0x39391f<br />
5,863 6,917 0 0 3,705,888 3,705,888 0 12,780 0x3804/0x71c53f<br />
</DbPrintLog><br />
<br />
* This prints some statistics about the database. In particular the two rows called key bytes and data bytes are relevant. For the following step you need the values from the avg column - i.e. 38 and 219 in this case. You also need the value of “Total number of entries” (14543762 in this case). Now we take those values and put them into this command:<br />
<br />
[root@dc042 meta]# java -cp /usr/share/dcache/classes/je-*.jar com.sleepycat.je.util.DbCacheSize -records 14543762 -key 38 -data 219<br />
<br />
=== Environment Cache Overhead ===<br />
<br />
3,157,213 minimum bytes<br />
<br />
To account for JE daemon operation and record locks,<br />
a significantly larger amount is needed in practice.<br />
<br />
=== Database Cache Size ===<br />
<br />
Number of Bytes Description<br />
--------------- -----------<br />
1,195,759,832 Internal nodes only<br />
5,124,232,392 Internal nodes and leaf nodes<br />
<br />
* For further information see the DbCacheSize javadoc.<br />
<br />
* For this pool it tells us that we need a bit above 1195 MB to keep all the internal btree nodes cached. Oracle’s recommendation is that if the database is updated often, the cache is at least big enough to contain the internal nodes. Obviously you want to make it a bit bigger to leave room for it to grow. One could take this and configure the cache using the je.maxMemory setting. If the size is lower than the 20% of the max heap size you already use, then you don’t need to do anything (I do not suggest lowering it further).<br />
<br />
* Since this pool has 6000m of JAVA heap max space, it means that 20% = 1200MB, which is exactly cache for Internal btree nodes. So we will not touch this.<br />
<br />
* There is one caveat though: If you increase the cache size, less free space is left on the heap. If the pool is pushed to the limit, this may actually slow down the pool as garbage collection overhead increases. You need to ensure that enough space is left in addition to the cache (possibly by increasing the max heap size). An alternative to adjusting je.maxMemory is of course to adjust the max heap size. If you make it large enough so that 20% is enough to cache the internal btree nodes, then all is well. This may however mean you assign significantly more memory to the pool than it really needs. Yeah, there are lots of things to consider.<br />
<br />
* A little bonus: There is also this command:<br />
<br />
[root@dc042 meta]# java -cp /usr/share/dcache/classes/je-*.jar com.sleepycat.je.util.DbSpace -h .<br />
File Size (KB) % Used<br />
-------- --------- ------<br />
00001857 9765 41<br />
00001f2a 9765 43<br />
00001f2f 9765 39<br />
00001f72 9765 44<br />
0000216d 9765 40<br />
000021cb 9765 40<br />
0000226b 9765 45<br />
000022e8 9763 38<br />
000022fa 9765 39<br />
00002390 9765 38<br />
0000241d 9765 49<br />
0000252c 9765 51<br />
0000254c 9765 47<br />
00002567 9765 39<br />
000025b7 9765 42<br />
000025db 9765 42<br />
000025e5 9765 56<br />
0000265f 9765 39<br />
00002678 9765 61<br />
0000268d 9762 48<br />
00002692 9765 74<br />
000026be 9765 48<br />
000026cf 9765 40<br />
00002700 9765 41<br />
00002706 9765 38<br />
00002718 9765 56<br />
00002747 9765 44<br />
0000274d 9765 48<br />
000027db 9765 62<br />
00002855 9765 39<br />
0000285c 9765 39<br />
00002882 9765 41<br />
0000289d 9765 54<br />
000028ba 9765 52<br />
000028c1 9765 53<br />
000028d0 9765 41<br />
000028d7 9765 41<br />
000028e7 9765 39<br />
000028ed 9765 39<br />
00002926 9765 39<br />
0000292b 9765 43<br />
00002945 9765 41<br />
00002952 9765 41<br />
00002958 9765 62<br />
00002959 9764 38<br />
00002962 9765 53<br />
00002963 9765 47<br />
0000297d 9765 51<br />
0000297e 9765 41<br />
000029b3 9765 45<br />
000029c7 9765 56<br />
000029ce 9765 42<br />
000029d3 9765 43<br />
000029d9 9765 54<br />
000029fb 9765 42<br />
00002a00 9765 54<br />
00002a01 9765 39<br />
00002a06 9765 39<br />
00002a20 9765 45<br />
00002a26 9765 61<br />
00002a2b 9765 69<br />
00002a30 9765 65<br />
00002a44 9765 46<br />
00002a4f 9765 50<br />
00002a53 9765 64<br />
00002a54 9765 47<br />
00002a58 9765 66<br />
00002a59 9765 43<br />
00002a5e 9765 56<br />
00002a63 9765 44<br />
00002a69 9765 47<br />
00002a8f 9765 39<br />
00002aac 9765 54<br />
00002aba 9765 42<br />
00002abb 9765 44<br />
00002ae0 9763 38<br />
00002af3 9765 45<br />
00002b01 9765 41<br />
00002b08 9765 65<br />
00002b0e 9765 41<br />
00002b24 9765 45<br />
00002b28 9765 61<br />
00002b29 9765 41<br />
00002b3c 9765 40<br />
00002b3d 9765 40<br />
00002b41 9765 47<br />
00002b4c 9765 46<br />
00002b52 9765 57<br />
00002b64 9765 38<br />
00002b87 9765 38<br />
00002b8e 9765 47<br />
00002b8f 9760 38<br />
00002bad 9765 42<br />
00002bbf 9765 44<br />
00002bc6 9765 38<br />
00002be9 9765 47<br />
00002c00 9765 42<br />
00002c06 9765 69<br />
00002c0c 9765 55<br />
00002c13 9765 38<br />
00002c14 9765 38<br />
00002c2a 9765 41<br />
00002c2b 9765 41<br />
00002c30 9765 54<br />
00002c4c 9765 39<br />
00002c53 9765 45<br />
00002c5f 9765 44<br />
00002c7c 9765 38<br />
00002c8f 9765 49<br />
00002c95 9765 38<br />
00002ca4 9764 50<br />
00002ca9 9765 42<br />
00002cad 9763 41<br />
00002cc4 9765 39<br />
00002ccf 9765 43<br />
00002cd3 9765 45<br />
00002cf3 9765 48<br />
00002d19 9765 38<br />
00002d20 9765 38<br />
00002d3a 9765 40<br />
00002d3b 9765 41<br />
00002d41 9765 43<br />
00002d51 9765 39<br />
00002d59 9765 40<br />
00002d79 9765 62<br />
00002d8f 9765 39<br />
00002deb 9765 40<br />
00002e21 9765 39<br />
00002e40 9765 41<br />
00002e41 9765 45<br />
00002e45 9765 40<br />
00002e5b 9765 58<br />
00002e5f 9765 49<br />
00002e60 9765 67<br />
00002e7d 9765 48<br />
00002e95 9765 38<br />
00002eb1 9762 57<br />
00002eb6 9765 53<br />
00002ee1 9765 38<br />
00002ef4 9763 43<br />
00002ef8 9765 41<br />
00002f1a 9765 53<br />
00002f25 9765 65<br />
00002f5a 9765 52<br />
00002f5f 9765 63<br />
00002f66 9765 41<br />
00002f71 9765 42<br />
00002f79 9765 61<br />
00002f87 9765 49<br />
00002f9d 9765 41<br />
00002f9e 9765 42<br />
00002fa3 9765 38<br />
00002fa4 9765 46<br />
00002fb1 9762 55<br />
00002fc1 9765 41<br />
00002fde 9765 38<br />
0000300a 9765 43<br />
0000300e 9762 47<br />
00003026 9765 55<br />
00003027 9765 57<br />
0000303f 9765 63<br />
0000304f 9765 48<br />
00003056 9765 44<br />
0000306c 9765 48<br />
0000306d 9765 42<br />
00003071 9765 45<br />
0000308d 9765 39<br />
000030a1 9765 54<br />
000030a5 9765 51<br />
000030ab 9765 60<br />
000030c2 9765 46<br />
000030d5 9765 43<br />
000030dd 9765 38<br />
000030e3 9765 57<br />
000030e4 9765 55<br />
000030fd 9765 48<br />
0000310c 9764 42<br />
00003110 9765 46<br />
00003119 9765 47<br />
0000315c 9765 48<br />
0000315d 9765 54<br />
00003185 9765 56<br />
00003189 9765 52<br />
0000318a 9765 58<br />
0000319a 9765 46<br />
0000319f 9764 39<br />
000031b5 9764 53<br />
000031b8 9765 40<br />
000031b9 9765 74<br />
000031e2 9765 65<br />
000031fa 9765 58<br />
000031fb 9765 52<br />
00003201 9765 50<br />
00003202 9764 44<br />
00003208 9765 42<br />
00003210 9763 40<br />
00003218 9765 40<br />
00003224 9765 51<br />
00003229 9765 48<br />
00003235 9765 40<br />
0000324e 9765 46<br />
00003266 9765 41<br />
0000326c 9764 47<br />
0000327b 9765 55<br />
00003290 9765 48<br />
00003294 9765 56<br />
000032a7 9765 69<br />
000032ad 9765 62<br />
000032de 9765 47<br />
000032e1 9765 46<br />
000032e2 9765 71<br />
000032e9 9765 45<br />
00003315 9765 42<br />
00003316 9764 54<br />
00003320 9765 43<br />
0000332b 9765 52<br />
0000332c 9765 76<br />
0000336d 9765 69<br />
00003376 9765 72<br />
00003377 9765 46<br />
0000339f 9765 43<br />
000033a0 9765 75<br />
000033a6 9765 46<br />
000033b6 9765 40<br />
000033c0 9764 72<br />
000033d2 9765 38<br />
000033d3 9764 45<br />
000033d9 9765 42<br />
000033da 9765 44<br />
000033ee 9765 41<br />
000033fb 9764 38<br />
00003402 9765 42<br />
00003403 9765 53<br />
0000340c 9765 76<br />
00003428 9765 67<br />
00003431 9765 51<br />
00003436 9765 88<br />
0000343b 9765 85<br />
0000344c 9765 48<br />
00003457 9765 46<br />
0000345b 9765 46<br />
00003460 9765 50<br />
00003465 9765 45<br />
00003466 9762 40<br />
0000347c 9765 54<br />
0000347d 9765 61<br />
00003497 9765 63<br />
000034b3 9765 64<br />
000034bd 9765 54<br />
000034c9 9764 71<br />
000034d6 9765 65<br />
000034d9 9765 86<br />
000034da 9765 46<br />
000034df 9765 51<br />
000034e0 9764 60<br />
00003504 9762 80<br />
00003512 9761 76<br />
0000351b 9765 65<br />
00003525 9765 59<br />
00003532 9765 52<br />
00003540 9764 43<br />
00003553 9765 80<br />
00003558 9765 54<br />
00003580 9762 49<br />
00003586 9765 41<br />
00003587 9763 50<br />
0000359c 9765 40<br />
000035a0 9765 46<br />
000035a1 9765 37<br />
000035a6 9765 96<br />
000035ab 9765 52<br />
000035ac 9765 39<br />
000035c4 9765 61<br />
000035d0 9765 89<br />
000035dc 9765 74<br />
000035dd 9765 43<br />
000035f1 9765 43<br />
000035fd 9765 37<br />
00003604 9765 47<br />
00003609 9764 40<br />
0000360c 9765 59<br />
0000360d 9765 37<br />
00003618 9763 42<br />
00003621 9765 42<br />
00003626 9765 42<br />
00003627 9765 41<br />
0000362a 9765 38<br />
0000362b 9765 73<br />
00003630 9765 68<br />
00003631 9764 42<br />
0000363e 9763 39<br />
00003643 9765 43<br />
00003646 9765 38<br />
0000364c 9762 40<br />
00003659 9765 47<br />
0000365c 9764 44<br />
0000365f 9765 45<br />
00003664 9765 69<br />
00003668 9764 37<br />
00003671 9765 43<br />
00003676 9765 39<br />
00003688 9765 37<br />
00003691 9765 48<br />
00003692 9764 37<br />
00003695 9762 38<br />
000036a3 9764 50<br />
000036aa 9765 47<br />
000036ab 9765 49<br />
000036c2 9765 64<br />
000036c3 9764 57<br />
000036c5 9765 40<br />
000036da 9764 40<br />
000036e1 9765 43<br />
000036ed 9762 39<br />
000036fc 9765 41<br />
000036fd 9765 59<br />
00003701 9764 85<br />
00003708 9765 43<br />
0000370a 9763 49<br />
00003728 9762 41<br />
0000372a 9763 55<br />
0000372d 9765 51<br />
00003732 9763 48<br />
00003734 9765 60<br />
00003736 9765 42<br />
00003740 9765 48<br />
00003741 9765 39<br />
00003742 9765 38<br />
00003749 9764 40<br />
00003756 9763 39<br />
00003758 9762 46<br />
0000375c 9765 67<br />
0000375d 9765 37<br />
00003761 9765 63<br />
00003779 9765 55<br />
00003785 9765 43<br />
00003798 9763 60<br />
0000379b 9765 41<br />
0000379e 9765 37<br />
000037a5 9764 39<br />
000037a8 9764 48<br />
000037ab 9765 38<br />
000037ae 9765 54<br />
000037b3 9764 42<br />
000037b6 9761 36<br />
000037be 9763 46<br />
000037c1 9765 69<br />
000037c3 9765 36<br />
000037c4 9764 37<br />
000037ce 9762 50<br />
000037d1 9761 45<br />
000037d3 9765 45<br />
000037dd 9765 45<br />
000037df 9765 50<br />
000037e0 9763 39<br />
000037e2 9765 13<br />
000037e3 9763 18<br />
000037e7 9765 49<br />
000037e8 9763 41<br />
000037e9 9765 42<br />
000037ea 9763 40<br />
000037eb 9765 39<br />
000037ec 9761 39<br />
000037ed 9763 41<br />
000037ef 9763 36<br />
000037f0 9764 36<br />
000037f1 9764 39<br />
000037f3 9765 41<br />
000037f4 9761 53<br />
000037f5 9765 36<br />
000037f6 9765 54<br />
000037f7 9763 53<br />
000037f8 9765 63<br />
000037f9 9765 60<br />
000037fa 9765 70<br />
000037fb 9765 76<br />
000037fc 9765 72<br />
000037fd 9765 73<br />
000037fe 9765 83<br />
000037ff 9765 87<br />
00003800 9765 89<br />
00003801 9762 86<br />
00003802 9763 88<br />
00003803 9764 90<br />
00003804 8110 76<br />
TOTALS 3757891 49<br />
<br />
* It will tell you for each of the database files (the log segments) how big it is and what the utilization is. The Berkeley DB will try to keep the total utilization above 50%, but for a small pool like this one it cannot do it. I figure that you may find this interesting to know after reading about utilization above.<br />
<br />
* The log segment size and the utilization goal can be adjusted too, but I cannot give any sound advice on whether that’s a good idea and how to determine good values.<br />
<br />
* All of the above is only relevant if you use the Berkeley DB backend for pools.<br />
<br />
== Special thanks ==<br />
* This documentation has been deployed thanks to Gerd Behrmann, you can find more in at: https://gist.github.com/gbehrmann/df1abbe45c03356eff2e<br />
* Read more about this at: <br />
:* https://backstage.forgerock.com/knowledge/kb/book/b74223337/a49979000<br />
:* http://docs.oracle.com/cd/E17277_02/html/java/com/sleepycat/je/EnvironmentConfig.html</div>Mcaubethttps://pwiki.pic.es/index.php?title=DCACHE_CHECK_POOL_METADATA&diff=229DCACHE CHECK POOL METADATA2017-10-06T08:38:13Z<p>Mcaubet: /* Special thanks */</p>
<hr />
<div>= For the MoD =<br />
* Open a ticket to incidents@support.pic.es with the alarm. <br />
* This alarm only affects the duration of the pool start up, nothing has to be done until the next working day where this can be fixed by dCache Administrators.<br />
* Same information as [[DCACHE CHECK POOL BTREE MEMORY]] as both alarms are strictly related.<br />
<br />
= For the Admin =<br />
<br />
* Berkely DB should be tunned according to the number of meta files that a pool contains. If this not correctly adapted, pool start up will may need several hours to be fully available. Gerd explained how to tune up a pool according to this number. Below I show a modified explanation of what he said, adapting the explanation to the ATLAS pools.<br />
<br />
* The default je.log.fileCacheSize is 100. This could be increased by creating a file je.properties and adding the 'je.log.fileCacheSize=100'.<br />
<br />
* Also, other relevant settings are:<br />
<br />
je.maxMemory=<br />
je.maxMemoryPercent=20<br />
<br />
* These are equivalent. The former sets the number of bytes used for the btree cache, while the latter defines it as a percentage of the max heap size. E.g. if dcache.java.memory.heap is set to 2048m, then 410 MB is used for the btree cache.<br />
<br />
* The question is then how to determine a good size for the cache. There are some hidden utilities you can use to do this.<br />
<br />
* Below I show an example for dc042_1:<br />
<br />
[root@dc042 meta]# java -cp /usr/share/dcache/classes/je-*.jar com.sleepycat.je.util.DbPrintLog -h . -S<br />
<DbPrintLog><br />
Log statistics:<br />
type total provisional total min max avg entries<br />
count count bytes bytes bytes bytes as % of log<br />
MapLN 1,324 0 12,617,573 76 11,985 9,529 0.3<br />
NameLN 4 0 141 31 38 35 0<br />
FileSummaryLN 231,140 0 55,644,337 24 49,701 240 1.4<br />
IN 84,083 1,080 296,942,059 45 5,709 3,531 7.7<br />
BIN 114,865 114,865 377,569,148 44 6,675 3,287 9.8<br />
DbTree 318 0 1,034,997 2,680 4,136 3,254 0<br />
Commit 3,376,012 0 114,784,408 34 34 34 3<br />
CkptStart 158 0 4,899 30 33 31 0<br />
CkptEnd 226 0 22,560 67 186 99 0<br />
Trace 2,506 0 525,597 48 402 209 0<br />
FileHeader 385 0 14,630 38 38 38 0<br />
DEL_LN_TX 748,055 0 51,409,527 57 69 68 1.3<br />
INS_LN_TX 619,747 0 192,298,301 96 1,443 310 5<br />
UPD_LN_TX 2,008,226 0 398,854,718 32 1,919 198 10.4<br />
UPD_LN 6,636,173 0 1,997,042,458 18 1,819 300 51.9<br />
NewBINDelta 720,540 720,540 348,480,254 55 1,782 483 9.1<br />
key bytes 10,012,201 387,254,450 1 61 38 (10.1)<br />
data bytes 9,264,146 2,036,632,128 1 1,848 219 (52.9)<br />
<br />
Total bytes in portion of log read: 3,847,245,607<br />
Total number of entries: 14,543,762<br />
<br />
Per checkpoint interval info:<br />
lnTxn ln mapLNTxn mapLN end to end end to start start to end maxLNReplay ckptEnd<br />
273 63,082 0 10 30,455,407 3,857,456,063 0 63,355 0x1f2f/0x6f2ef<br />
5,168 130,892 0 24 39,782,369 3,827,000,656 0 136,060 0x226b/0x3a0d0<br />
27,098 132,905 0 15 50,120,541 16,509,395 33,611,146 160,003 0x252c/0x577ad<br />
128 32,773 0 3 9,904,160 3,737,097,746 0 134,154 0x254c/0x4014d<br />
82 31,341 0 12 10,576,583 3,727,193,586 0 31,423 0x2567/0xccd94<br />
0 72 0 4 711,899 51,780 660,119 72 0x2567/0x17aa6f<br />
36,445 61,509 0 0 28,499,838 3,715,905,104 0 97,954 0x25e5/0xc66d<br />
2 108 0 6 1,564,919 56,871 1,508,048 110 0x25e5/0x18a764<br />
10,732 50,299 0 5 18,877,564 3,685,840,347 0 61,055 0x2678/0x786e0<br />
33 190 0 6 1,627,301 94,279 1,533,022 223 0x2678/0x205b85<br />
8,277 84,136 0 3 28,217,133 17,808,699 10,408,434 92,434 0x26be/0x52732<br />
5,401 25,394 0 7 9,824,151 3,637,118,349 0 61,029 0x26cf/0x27849<br />
49,026 92,572 0 3 41,592,807 3,627,294,198 0 141,598 0x2747/0x1ac630<br />
2 46 0 6 357,170 45,107 312,063 48 0x2747/0x203962<br />
20,496 39,154 0 3 18,591,447 3,585,344,221 0 59,654 0x27db/0xabb39<br />
0 154 0 6 1,187,463 83,676 1,103,787 154 0x27db/0x1cd9c0<br />
17,107 49,468 0 3 18,905,722 3,565,565,311 0 66,583 0x285c/0xc273a<br />
58 0 0 6 392,501 62,920 329,581 58 0x285c/0x12246f<br />
113,890 201,969 0 17 79,488,418 3,546,267,088 0 315,874 0x28ed/0xa5611<br />
22,147 98,579 0 8 40,657,515 36,319,235 4,338,280 120,726 0x2952/0x145e7c<br />
0 122 0 6 1,296,043 70,361 1,225,682 28,367 0x2952/0x282527<br />
19,570 65,944 0 5 27,570,660 25,251,492 2,319,168 85,520 0x2962/0x3138b<br />
37,714 28,889 0 3 21,505,514 19,337,884 2,167,630 66,613 0x297d/0x1a0c75<br />
3 332 0 6 2,437,155 124,630 2,312,525 479 0x297d/0x3f3c98<br />
41,458 62,406 0 3 26,385,897 3,373,311,783 0 103,904 0x29c7/0x81701<br />
0 46 0 6 1,404,642 46,045 1,358,597 46 0x29c7/0x1d85e3<br />
41,443 87,747 0 3 39,909,271 14,624,816 25,284,455 129,198 0x29fb/0x1c237a<br />
0 123 0 5 1,185,318 68,652 1,116,666 79,470 0x29fb/0x2e39a0<br />
9,199 12,698 0 4 9,927,451 3,304,426,655 0 21,899 0x2a00/0x2d1e3b<br />
8 0 0 5 118,374 35,364 83,010 8 0x2a00/0x2eeca1<br />
122,697 96,787 0 3 66,966,017 3,294,380,830 0 219,488 0x2a44/0xa122<br />
53,436 12,002 0 3 21,830,901 3,227,414,813 0 65,438 0x2a53/0x1c9117<br />
20 167 0 6 1,308,679 85,964 1,222,715 187 0x2a53/0x30891e<br />
18,552 38,805 0 6 17,501,977 3,204,275,233 0 57,381 0x2a58/0xa6b37<br />
9 105 0 4 678,297 87,871 590,426 114 0x2a58/0x14c4d0<br />
25,434 36,231 0 10 19,064,711 3,186,094,959 0 61,675 0x2a5e/0x67f57<br />
15 159 0 6 1,467,746 83,228 1,384,518 174 0x2a5e/0x1ce4b9<br />
12,236 14,099 0 3 8,375,785 3,165,562,502 0 26,358 0x2a63/0x41c22<br />
6 184 0 6 1,058,988 89,948 969,040 190 0x2a63/0x1444ce<br />
71,844 24,018 0 3 29,188,061 27,937,328 1,250,733 95,870 0x2aac/0x7e12b<br />
9,407 25,608 0 7 11,016,096 3,126,939,668 0 35,029 0x2aba/0x17624b<br />
20 110 0 6 748,510 82,108 666,402 130 0x2aba/0x22ce29<br />
64,997 49,072 0 15 27,822,548 27,438,951 383,597 114,142 0x2af3/0x1947d<br />
10,534 23,197 0 6 12,103,837 3,087,352,514 0 33,737 0x2b01/0x21ae9a<br />
122 0 0 32 879,716 489,740 389,976 122 0x2b01/0x2f1afe<br />
60,029 128,431 0 24 58,556,679 56,338,979 2,217,700 188,485 0x2b3c/0x191505<br />
15 52 0 6 530,374 53,795 476,579 231 0x2b3c/0x212ccb<br />
48,799 11,045 0 4 18,178,427 17,837,860 340,567 59,855 0x2b41/0x56146<br />
45,998 29,564 0 3 20,154,236 2,997,103,481 0 75,573 0x2b52/0x7bbc2<br />
39,570 60,757 0 15 30,273,541 2,976,949,245 0 100,327 0x2b8e/0xbe847<br />
31,631 33,780 0 13 22,348,448 16,815,306 5,533,142 65,411 0x2bad/0x2fbde7<br />
16,831 40,178 0 5 20,571,223 2,924,327,256 0 57,047 0x2bc6/0x38753e<br />
4,292 18,826 0 8 8,363,076 2,903,756,033 0 23,118 0x2be9/0x1f7b02<br />
0 254 0 6 2,043,974 119,900 1,924,074 254 0x2be9/0x3eab48<br />
30 19,902 0 4 7,727,051 2,893,348,983 0 19,954 0x2c00/0x1bfc93<br />
27,984 75,433 0 32 29,896,927 2,885,621,932 0 103,417 0x2c13/0x1a69f2<br />
40,477 36,001 0 10 19,737,838 17,591,303 2,146,535 76,478 0x2c2a/0x1669e0<br />
2 2 0 4 194,668 36,808 157,860 10 0x2c2a/0x19624c<br />
29,246 68,367 0 3 31,325,183 2,835,792,499 0 97,615 0x2c4c/0x2d9acb<br />
0 26 0 5 1,118,302 27,604 1,090,698 26 0x2c4c/0x3eab29<br />
4 19,763 0 3 7,562,665 2,803,349,014 0 19,775 0x2c53/0x197a52<br />
26 194 0 6 1,436,963 103,803 1,333,160 220 0x2c53/0x2f6775<br />
6,255 14,988 0 8 9,972,102 2,794,349,386 0 21,292 0x2c5f/0x2efa7b<br />
39,212 18,305 0 5 18,591,243 2,784,377,284 0 57,517 0x2c8f/0x197b86<br />
0 41 0 6 1,495,215 121,720 1,373,495 41 0x2c8f/0x304c35<br />
347 23,340 0 3 7,036,643 2,764,290,826 0 23,697 0x2c95/0x31498<br />
23,408 9,116 0 3 10,098,528 2,757,254,183 0 32,524 0x2ca4/0x49578<br />
45,631 27,183 0 3 19,972,142 2,747,155,655 0 72,814 0x2cad/0x428a6<br />
0 0 0 1 963,802 69 963,733 0 0x2cad/0x12dd80<br />
276 19,066 0 4 10,705,056 5,926,693 4,778,363 19,342 0x2cc4/0x1d9fa0<br />
0 25,292 0 2 10,568,825 7,122,257 3,446,568 25,292 0x2ccf/0x264d99<br />
722 25,106 0 3 9,376,076 2,704,945,830 0 26,568 0x2cd3/0x1cc865<br />
15 96 0 6 601,987 65,907 536,080 111 0x2cd3/0x25f7e8<br />
50,662 51,942 0 4 29,283,798 2,694,967,767 0 102,611 0x2d20/0x1b0a3e<br />
18,486 16,140 0 4 12,291,893 2,665,683,969 0 34,626 0x2d3a/0x3e02f3<br />
67,481 104,443 0 5 48,233,597 15,261,303 32,972,294 171,924 0x2d79/0x230ef0<br />
19,276 34,062 0 3 18,103,592 2,605,158,479 0 171,476 0x2deb/0x61f18<br />
0 225 0 6 607,824 37,538 570,286 225 0x2deb/0xf6568<br />
14,038 14,130 0 3 13,699,081 2,586,447,063 0 28,385 0x2e21/0x47d6f1<br />
4,147 13,212 0 3 5,363,836 2,572,747,982 0 17,359 0x2e40/0x118ed<br />
0 0 0 4 133,826 37,921 95,905 0 0x2e40/0x323af<br />
31,744 33,396 0 3 22,247,625 2,567,250,320 0 65,140 0x2e45/0x256f78<br />
9 0 0 6 290,601 55,855 234,746 9 0x2e45/0x29dea1<br />
16 25,020 0 4 10,475,420 2,544,712,094 0 25,039 0x2e5b/0x311fbd<br />
1,682 19,008 0 3 8,507,134 2,534,236,674 0 20,690 0x2e5f/0x1a583b<br />
12 0 0 6 354,107 41,047 313,060 12 0x2e5f/0x1fbf76<br />
6,849 54,863 0 3 20,176,980 2,525,375,433 0 61,719 0x2e7d/0x2272ca<br />
30,012 67,588 0 5 28,465,075 27,606,214 858,861 97,600 0x2eb6/0xb06fd<br />
14,048 16,912 0 2 9,587,676 2,476,733,378 0 66,881 0x2ee1/0x4bc59<br />
15,183 12,704 0 6 11,270,153 6,834,792 4,435,361 27,887 0x2ef4/0x181de2<br />
23,012 39,873 0 7 18,610,316 2,455,875,549 0 62,948 0x2f1a/0x2e96e<br />
25,472 49,151 0 6 22,681,935 2,437,265,233 0 74,623 0x2f5a/0x2bd5bd<br />
1,031 60,319 0 6 20,425,359 2,414,583,298 0 61,350 0x2f66/0x32534c<br />
48,185 83,225 0 4 42,656,335 28,813,330 13,843,005 131,410 0x2f9d/0x5adb9b<br />
16,859 32,139 0 3 18,804,637 12,310,056 6,494,581 74,600 0x2fa3/0x489e38<br />
29,050 79,391 0 8 47,784,742 37,879,192 9,905,550 108,511 0x300a/0x26d0de<br />
32 38,306 0 6 19,991,849 14,244,764 5,747,085 117,588 0x3026/0x26b107<br />
0 37 0 6 345,851 39,476 306,375 25,114 0x3026/0x2bf802<br />
17,725 34,199 0 3 19,155,357 2,264,574,525 0 51,955 0x303f/0x1f149f<br />
1,379 23,909 0 6 10,787,679 2,245,419,168 0 25,288 0x304f/0x2b197e<br />
3 12 0 5 297,677 34,135 263,542 15 0x304f/0x2fa44b<br />
17,968 47,908 0 2 17,014,500 2,234,333,812 0 65,891 0x306c/0x2162f<br />
45,151 19,691 0 3 20,592,703 18,998,392 1,594,311 64,842 0x3071/0xb216e<br />
18,246 13,219 0 3 14,900,440 2,196,726,609 0 31,514 0x308d/0x55e7c6<br />
26 12,008 0 5 7,554,365 4,234,132 3,320,233 12,034 0x30a1/0x309683<br />
35,556 69,762 0 20 31,161,221 2,174,271,804 0 105,338 0x30c2/0x424e88<br />
33,203 114,692 0 5 49,168,609 43,804,782 5,363,827 147,895 0x30fd/0x359ee9<br />
182 19,563 0 7 10,363,229 2,093,941,974 0 87,398 0x310c/0x3b29c6<br />
14 19,817 0 4 9,355,037 4,743,642 4,611,395 19,831 0x3110/0x315263<br />
46 37,708 0 8 20,678,814 10,884,622 9,794,192 41,674 0x315c/0x3bae01<br />
88 279 0 5 761,100 44,429 716,671 3,390 0x315c/0x474b0d<br />
40,709 61,572 0 4 26,837,961 2,052,783,794 0 102,607 0x3189/0x170b56<br />
42 0 0 6 814,957 77,642 737,315 42 0x3189/0x237ac3<br />
69,126 107,504 0 5 52,094,116 47,372,898 4,721,218 176,645 0x31b8/0x436ee7<br />
46 0 0 5 393,197 51,476 341,721 82,129 0x31b8/0x496ed4<br />
48,348 117,823 0 3 48,879,763 41,965,088 6,914,675 166,196 0x3201/0x3856e7<br />
16,349 39,063 0 7 17,875,388 14,606,658 3,268,730 62,982 0x3208/0x17eba3<br />
105 0 0 6 3,394,912 60,919 3,333,993 2,571 0x3208/0x4bb903<br />
217 17,310 0 7 5,776,110 1,902,493,500 0 17,617 0x3210/0xb4571<br />
36,657 27,268 0 7 22,160,771 4,617,088 17,543,683 63,925 0x3224/0x2c3df4<br />
24,153 42,022 0 1 22,019,603 1,874,556,619 0 114,113 0x3235/0x4b0f07<br />
16 18,174 0 7 7,706,321 1,852,537,016 0 18,190 0x324e/0x280f58<br />
120 24,379 0 7 10,458,169 1,844,830,695 0 24,499 0x3266/0x2f0d11<br />
14,236 34,916 0 3 17,998,326 12,723,415 5,274,911 49,152 0x327b/0x108207<br />
79 0 0 5 673,666 37,699 635,967 2,808 0x327b/0x1ac989<br />
6,624 22,958 0 3 11,999,224 1,815,700,534 0 29,650 0x3290/0x394b01<br />
19,965 40,810 0 8 17,269,398 1,803,701,310 0 60,775 0x32a7/0xfa097<br />
7,517 58,175 0 5 21,772,528 1,786,431,912 0 65,692 0x32de/0x2aac87<br />
9,415 18,600 0 6 10,786,946 1,764,659,384 0 28,015 0x32e1/0x36ae89<br />
51 0 0 6 622,591 52,177 570,414 51 0x32e1/0x402e88<br />
30,139 66,899 0 4 30,497,693 1,753,249,847 0 97,065 0x3315/0x47c6a5<br />
6 41,284 0 1 16,342,999 12,394,744 3,948,255 41,290 0x3320/0xff97c<br />
21,091 17,059 0 6 11,525,646 1,706,409,155 0 38,984 0x332b/0x27410a<br />
0 94 0 4 892,946 34,897 858,049 94 0x332b/0x34e11c<br />
3 52,035 0 3 18,160,444 1,693,990,563 0 52,132 0x336d/0x18cf58<br />
58 27,035 0 6 8,947,956 1,675,830,119 0 27,093 0x3376/0x8c1cc<br />
3 66 0 4 501,044 49,007 452,037 69 0x3376/0x106700<br />
27,134 44,130 0 4 22,994,532 1,666,381,119 0 71,330 0x339f/0x3e1864<br />
0 108 0 4 439,894 35,044 404,850 108 0x339f/0x44ceba<br />
17 51,506 0 3 20,284,195 1,642,946,693 0 51,631 0x33a6/0x4924dd<br />
28 16,826 0 2 9,954,053 1,622,662,498 0 16,854 0x33b6/0x487162<br />
2,079 43,231 0 2 20,850,856 13,682,453 7,168,403 45,310 0x33d2/0x556d0a<br />
2 30,027 0 3 14,418,091 8,924,719 5,493,372 30,743 0x33d9/0x40b5<br />
0 1,157 0 6 5,510,215 155,394 5,354,821 3,103 0x33d9/0x5454fc<br />
31,517 31,808 0 3 18,110,480 1,571,929,283 0 64,085 0x33ee/0x37800c<br />
30,598 27,193 0 4 18,121,300 12,201,650 5,919,650 57,791 0x3402/0x1ad560<br />
38,905 34,440 0 4 18,443,706 15,100,893 3,342,813 78,249 0x340c/0x3161a<br />
51,887 95,209 0 4 40,262,214 29,791,799 10,470,415 152,283 0x343b/0x71660<br />
1 28,686 0 7 10,081,465 1,476,991,583 0 142,625 0x344c/0x85499<br />
28,401 13,707 0 3 10,169,877 1,466,910,118 0 42,108 0x3457/0xaec2e<br />
5 16,531 0 6 14,562,193 4,402,715 10,159,478 16,536 0x345b/0x50893f<br />
0 15,920 0 7 9,514,322 1,442,178,048 0 19,604 0x3460/0x492011<br />
4 16,264 0 6 10,131,895 1,432,663,726 0 16,268 0x3465/0x4b2348<br />
30,231 28,986 0 4 18,938,103 14,093,138 4,844,965 59,217 0x347c/0x3aef3f<br />
14 0 0 5 512,006 34,432 477,574 120 0x347c/0x42bf45<br />
20,678 38,294 0 3 17,101,301 1,403,081,722 0 58,975 0x3497/0x16843a<br />
9,869 25,099 0 5 12,548,707 1,385,980,421 0 34,968 0x34b3/0x3d681d<br />
36,465 60,605 0 5 26,361,794 24,558,695 1,803,099 97,070 0x34d6/0x5e45f<br />
36 23,345 0 3 9,621,974 6,699,792 2,922,182 23,911 0x34d9/0x1fb5<br />
25,436 49,579 0 7 20,880,498 18,608,524 2,271,974 75,850 0x34df/0xd8f27<br />
32 1,485 0 6 4,197,555 413,166 3,784,389 1,535 0x34df/0x4d9bda<br />
31,535 59,968 0 5 25,248,912 24,728,797 520,115 91,849 0x3512/0x51cea<br />
1,601 24,378 0 6 10,268,411 8,200,696 2,067,715 26,160 0x351b/0x93565<br />
14,147 23,162 0 8 11,300,331 1,276,852,570 0 37,400 0x3525/0x1d0cd0<br />
14,204 20,662 0 3 13,299,117 1,265,552,239 0 34,866 0x3532/0x4f63fd<br />
17,514 24,728 0 3 16,797,576 12,513,187 4,284,389 42,242 0x3553/0x1e8685<br />
7,303 46,343 0 7 23,094,370 14,762,461 8,331,909 54,344 0x3580/0x4dbde7<br />
45,620 41,812 0 7 27,514,220 21,511,990 6,002,230 90,879 0x359c/0x27cfd3<br />
16,861 12,552 0 3 9,619,073 1,184,846,956 0 104,424 0x35a0/0x21ffd4<br />
0 271 0 6 3,731,928 93,043 3,638,885 271 0x35a0/0x5af1ac<br />
30,860 48,187 0 7 26,338,978 24,109,627 2,229,351 79,097 0x35ab/0x2314ce<br />
39,031 24,255 0 4 20,202,284 1,145,156,977 0 63,301 0x35c4/0x262afa<br />
2,576 46,002 0 6 19,184,048 5,580,647 13,603,401 48,578 0x35dc/0x19b7aa<br />
2 0 0 6 174,028 33,089 140,939 31,536 0x35dc/0x1c5f76<br />
111,528 77,161 0 7 62,341,581 50,549,489 11,792,092 188,690 0x360c/0x401a43<br />
28,697 25,910 0 6 18,246,350 13,855,848 4,390,502 56,636 0x3618/0x255811<br />
4,335 12,127 0 6 9,004,926 5,130,460 3,874,466 18,985 0x3621/0x16290f<br />
18,413 8,607 0 5 8,878,516 7,701,363 1,177,153 27,027 0x3626/0x50c43<br />
0 2,910 0 6 6,228,723 848,601 5,380,122 2,910 0x3626/0x641736<br />
28,784 22,689 0 3 21,119,867 14,626,185 6,493,682 52,178 0x362a/0x752db1<br />
41,676 62,528 0 4 34,112,988 29,303,765 4,809,223 105,238 0x363e/0x1b598d<br />
17 735 0 5 12,521,216 31,105 12,490,111 1,300 0x3643/0x41d20d<br />
7,802 34,494 0 7 33,127,169 7,514,191 25,612,978 43,003 0x3659/0x71898e<br />
0 8,734 0 5 3,377,408 900,015,281 0 25,130 0x365c/0xc7c0e<br />
114 9,858 0 7 15,782,161 650,578 15,131,583 9,972 0x365f/0x64b69f<br />
1 11,516 0 7 5,378,418 880,855,712 0 19,548 0x3664/0x1e3191<br />
7 22,005 0 7 8,922,863 6,691,187 2,231,676 22,012 0x3668/0xdc200<br />
13 13,409 0 6 15,952,421 360,887 15,591,534 14,391 0x3671/0x6895a5<br />
1,749 8,568 0 3 3,660,457 850,602,010 0 22,698 0x3676/0x7d9ce<br />
5,390 10,146 0 6 14,882,555 1,855,609 13,026,946 15,536 0x3688/0x525a49<br />
10,230 9,513 0 5 4,816,061 832,058,998 0 28,465 0x3691/0x34086<br />
14,698 31,907 0 7 41,080,361 36,925,183 4,155,178 46,605 0x36aa/0x13bcaf<br />
2 1,755 0 6 4,666,159 345,619 4,320,540 47,630 0x36aa/0x5aefde<br />
25,935 30,385 0 5 38,398,767 781,496,417 0 57,119 0x36c5/0x42810d<br />
9,478 14,696 0 7 21,102,013 743,097,650 0 24,174 0x36e1/0x5351ca<br />
65 12,746 0 6 5,610,587 3,858,013 1,752,574 12,811 0x36ed/0x1057a5<br />
5,806 9,341 0 6 10,353,968 2,690,013 7,663,955 15,343 0x36fc/0x15be55<br />
5 0 0 6 4,894,393 33,744 4,860,649 5,931 0x36fc/0x606d0e<br />
28,754 58,230 0 18 28,401,618 701,136,689 0 86,988 0x3708/0x480960<br />
0 15,897 0 6 10,890,931 3,098,266 7,792,665 15,897 0x370a/0x55a193<br />
126 10,927 0 6 26,068,972 2,214,107 23,854,865 16,100 0x372d/0x19a5ff<br />
9,837 16,155 0 6 11,096,409 6,346,816 4,749,593 29,696 0x3732/0x2a60d8<br />
128 15,250 0 6 8,591,606 3,747,801 4,843,805 16,046 0x3734/0x14e34e<br />
123 28,336 0 6 16,047,724 5,161,792 10,885,932 31,737 0x3736/0x712b3a<br />
24 8,534 0 5 3,423,605 1,552,888 1,870,717 19,982 0x3740/0xcd22f<br />
533 0 0 6 20,325,930 275,776 20,050,154 3,884 0x3742/0x11cb59<br />
0 0 0 4 72,242 36,230 36,012 0 0x3742/0x12e58b<br />
26,449 16,768 0 3 20,777,536 576,217,652 0 43,217 0x3756/0x1ec2cb<br />
11,493 23,349 0 3 21,836,979 13,435,005 8,401,974 34,842 0x375c/0x3aca7e<br />
309 32,239 0 6 17,558,766 9,588,731 7,970,035 56,790 0x3761/0x158a6c<br />
12 26,562 0 7 12,407,639 6,425,279 5,982,360 27,257 0x3779/0x3a4743<br />
6,845 41,908 0 7 33,072,537 22,775,381 10,297,156 51,339 0x379b/0x69295c<br />
6,778 29,990 0 6 25,664,453 17,110,934 8,553,519 41,294 0x37a8/0x2701a1<br />
5 16,509 0 9 19,648,397 8,979,547 10,668,850 44,387 0x37ae/0x21a42e<br />
116,569 137,423 0 12 129,250,694 72,386,145 56,864,549 267,033 0x37e2/0x163534<br />
13 17,546 0 6 28,042,727 34,685 28,008,042 187,464 0x37e7/0x90ee1b<br />
33,326 30,872 0 6 43,125,925 9,805,827 33,320,098 81,756 0x37ec/0x280a40<br />
3 5,332 0 5 19,873,538 37,653 19,835,885 30,103 0x37ef/0x261c42<br />
17 2,306 0 6 19,166,423 244,987 18,921,436 7,657 0x37f1/0x196419<br />
10,753 20,277 0 6 40,648,779 11,079,498 29,569,281 32,618 0x37f6/0x234a64<br />
10 22,876 0 6 18,391,375 164,634 18,226,741 49,034 0x37f8/0xabeb3<br />
30 34,377 0 7 32,006,385 1,775,081 30,231,304 56,545 0x37fb/0x295c24<br />
4 23,090 0 7 38,624,657 2,288,459 36,336,198 52,101 0x37ff/0x145fb5<br />
6 8,689 0 6 27,108,848 949,959 26,158,889 24,449 0x3801/0x80d8a5<br />
6 5,117 0 7 18,644,436 55,620 18,588,816 11,221 0x3803/0x6c2979<br />
837 0 0 6 6,661,670 288,422 6,373,248 5,956 0x3804/0x39391f<br />
5,863 6,917 0 0 3,705,888 3,705,888 0 12,780 0x3804/0x71c53f<br />
</DbPrintLog><br />
<br />
* This prints some statistics about the database. In particular the two rows called key bytes and data bytes are relevant. For the following step you need the values from the avg column - i.e. 38 and 219 in this case. You also need the value of “Total number of entries” (14543762 in this case). Now we take those values and put them into this command:<br />
<br />
[root@dc042 meta]# java -cp /usr/share/dcache/classes/je-*.jar com.sleepycat.je.util.DbCacheSize -records 14543762 -key 38 -data 219<br />
<br />
=== Environment Cache Overhead ===<br />
<br />
3,157,213 minimum bytes<br />
<br />
To account for JE daemon operation and record locks,<br />
a significantly larger amount is needed in practice.<br />
<br />
=== Database Cache Size ===<br />
<br />
Number of Bytes Description<br />
--------------- -----------<br />
1,195,759,832 Internal nodes only<br />
5,124,232,392 Internal nodes and leaf nodes<br />
<br />
* For further information see the DbCacheSize javadoc.<br />
<br />
* For this pool it tells us that we need a bit above 1195 MB to keep all the internal btree nodes cached. Oracle’s recommendation is that if the database is updated often, the cache is at least big enough to contain the internal nodes. Obviously you want to make it a bit bigger to leave room for it to grow. One could take this and configure the cache using the je.maxMemory setting. If the size is lower than the 20% of the max heap size you already use, then you don’t need to do anything (I do not suggest lowering it further).<br />
<br />
* Since this pool has 6000m of JAVA heap max space, it means that 20% = 1200MB, which is exactly cache for Internal btree nodes. So we will not touch this.<br />
<br />
* There is one caveat though: If you increase the cache size, less free space is left on the heap. If the pool is pushed to the limit, this may actually slow down the pool as garbage collection overhead increases. You need to ensure that enough space is left in addition to the cache (possibly by increasing the max heap size). An alternative to adjusting je.maxMemory is of course to adjust the max heap size. If you make it large enough so that 20% is enough to cache the internal btree nodes, then all is well. This may however mean you assign significantly more memory to the pool than it really needs. Yeah, there are lots of things to consider.<br />
<br />
* A little bonus: There is also this command:<br />
<br />
[root@dc042 meta]# java -cp /usr/share/dcache/classes/je-*.jar com.sleepycat.je.util.DbSpace -h .<br />
File Size (KB) % Used<br />
-------- --------- ------<br />
00001857 9765 41<br />
00001f2a 9765 43<br />
00001f2f 9765 39<br />
00001f72 9765 44<br />
0000216d 9765 40<br />
000021cb 9765 40<br />
0000226b 9765 45<br />
000022e8 9763 38<br />
000022fa 9765 39<br />
00002390 9765 38<br />
0000241d 9765 49<br />
0000252c 9765 51<br />
0000254c 9765 47<br />
00002567 9765 39<br />
000025b7 9765 42<br />
000025db 9765 42<br />
000025e5 9765 56<br />
0000265f 9765 39<br />
00002678 9765 61<br />
0000268d 9762 48<br />
00002692 9765 74<br />
000026be 9765 48<br />
000026cf 9765 40<br />
00002700 9765 41<br />
00002706 9765 38<br />
00002718 9765 56<br />
00002747 9765 44<br />
0000274d 9765 48<br />
000027db 9765 62<br />
00002855 9765 39<br />
0000285c 9765 39<br />
00002882 9765 41<br />
0000289d 9765 54<br />
000028ba 9765 52<br />
000028c1 9765 53<br />
000028d0 9765 41<br />
000028d7 9765 41<br />
000028e7 9765 39<br />
000028ed 9765 39<br />
00002926 9765 39<br />
0000292b 9765 43<br />
00002945 9765 41<br />
00002952 9765 41<br />
00002958 9765 62<br />
00002959 9764 38<br />
00002962 9765 53<br />
00002963 9765 47<br />
0000297d 9765 51<br />
0000297e 9765 41<br />
000029b3 9765 45<br />
000029c7 9765 56<br />
000029ce 9765 42<br />
000029d3 9765 43<br />
000029d9 9765 54<br />
000029fb 9765 42<br />
00002a00 9765 54<br />
00002a01 9765 39<br />
00002a06 9765 39<br />
00002a20 9765 45<br />
00002a26 9765 61<br />
00002a2b 9765 69<br />
00002a30 9765 65<br />
00002a44 9765 46<br />
00002a4f 9765 50<br />
00002a53 9765 64<br />
00002a54 9765 47<br />
00002a58 9765 66<br />
00002a59 9765 43<br />
00002a5e 9765 56<br />
00002a63 9765 44<br />
00002a69 9765 47<br />
00002a8f 9765 39<br />
00002aac 9765 54<br />
00002aba 9765 42<br />
00002abb 9765 44<br />
00002ae0 9763 38<br />
00002af3 9765 45<br />
00002b01 9765 41<br />
00002b08 9765 65<br />
00002b0e 9765 41<br />
00002b24 9765 45<br />
00002b28 9765 61<br />
00002b29 9765 41<br />
00002b3c 9765 40<br />
00002b3d 9765 40<br />
00002b41 9765 47<br />
00002b4c 9765 46<br />
00002b52 9765 57<br />
00002b64 9765 38<br />
00002b87 9765 38<br />
00002b8e 9765 47<br />
00002b8f 9760 38<br />
00002bad 9765 42<br />
00002bbf 9765 44<br />
00002bc6 9765 38<br />
00002be9 9765 47<br />
00002c00 9765 42<br />
00002c06 9765 69<br />
00002c0c 9765 55<br />
00002c13 9765 38<br />
00002c14 9765 38<br />
00002c2a 9765 41<br />
00002c2b 9765 41<br />
00002c30 9765 54<br />
00002c4c 9765 39<br />
00002c53 9765 45<br />
00002c5f 9765 44<br />
00002c7c 9765 38<br />
00002c8f 9765 49<br />
00002c95 9765 38<br />
00002ca4 9764 50<br />
00002ca9 9765 42<br />
00002cad 9763 41<br />
00002cc4 9765 39<br />
00002ccf 9765 43<br />
00002cd3 9765 45<br />
00002cf3 9765 48<br />
00002d19 9765 38<br />
00002d20 9765 38<br />
00002d3a 9765 40<br />
00002d3b 9765 41<br />
00002d41 9765 43<br />
00002d51 9765 39<br />
00002d59 9765 40<br />
00002d79 9765 62<br />
00002d8f 9765 39<br />
00002deb 9765 40<br />
00002e21 9765 39<br />
00002e40 9765 41<br />
00002e41 9765 45<br />
00002e45 9765 40<br />
00002e5b 9765 58<br />
00002e5f 9765 49<br />
00002e60 9765 67<br />
00002e7d 9765 48<br />
00002e95 9765 38<br />
00002eb1 9762 57<br />
00002eb6 9765 53<br />
00002ee1 9765 38<br />
00002ef4 9763 43<br />
00002ef8 9765 41<br />
00002f1a 9765 53<br />
00002f25 9765 65<br />
00002f5a 9765 52<br />
00002f5f 9765 63<br />
00002f66 9765 41<br />
00002f71 9765 42<br />
00002f79 9765 61<br />
00002f87 9765 49<br />
00002f9d 9765 41<br />
00002f9e 9765 42<br />
00002fa3 9765 38<br />
00002fa4 9765 46<br />
00002fb1 9762 55<br />
00002fc1 9765 41<br />
00002fde 9765 38<br />
0000300a 9765 43<br />
0000300e 9762 47<br />
00003026 9765 55<br />
00003027 9765 57<br />
0000303f 9765 63<br />
0000304f 9765 48<br />
00003056 9765 44<br />
0000306c 9765 48<br />
0000306d 9765 42<br />
00003071 9765 45<br />
0000308d 9765 39<br />
000030a1 9765 54<br />
000030a5 9765 51<br />
000030ab 9765 60<br />
000030c2 9765 46<br />
000030d5 9765 43<br />
000030dd 9765 38<br />
000030e3 9765 57<br />
000030e4 9765 55<br />
000030fd 9765 48<br />
0000310c 9764 42<br />
00003110 9765 46<br />
00003119 9765 47<br />
0000315c 9765 48<br />
0000315d 9765 54<br />
00003185 9765 56<br />
00003189 9765 52<br />
0000318a 9765 58<br />
0000319a 9765 46<br />
0000319f 9764 39<br />
000031b5 9764 53<br />
000031b8 9765 40<br />
000031b9 9765 74<br />
000031e2 9765 65<br />
000031fa 9765 58<br />
000031fb 9765 52<br />
00003201 9765 50<br />
00003202 9764 44<br />
00003208 9765 42<br />
00003210 9763 40<br />
00003218 9765 40<br />
00003224 9765 51<br />
00003229 9765 48<br />
00003235 9765 40<br />
0000324e 9765 46<br />
00003266 9765 41<br />
0000326c 9764 47<br />
0000327b 9765 55<br />
00003290 9765 48<br />
00003294 9765 56<br />
000032a7 9765 69<br />
000032ad 9765 62<br />
000032de 9765 47<br />
000032e1 9765 46<br />
000032e2 9765 71<br />
000032e9 9765 45<br />
00003315 9765 42<br />
00003316 9764 54<br />
00003320 9765 43<br />
0000332b 9765 52<br />
0000332c 9765 76<br />
0000336d 9765 69<br />
00003376 9765 72<br />
00003377 9765 46<br />
0000339f 9765 43<br />
000033a0 9765 75<br />
000033a6 9765 46<br />
000033b6 9765 40<br />
000033c0 9764 72<br />
000033d2 9765 38<br />
000033d3 9764 45<br />
000033d9 9765 42<br />
000033da 9765 44<br />
000033ee 9765 41<br />
000033fb 9764 38<br />
00003402 9765 42<br />
00003403 9765 53<br />
0000340c 9765 76<br />
00003428 9765 67<br />
00003431 9765 51<br />
00003436 9765 88<br />
0000343b 9765 85<br />
0000344c 9765 48<br />
00003457 9765 46<br />
0000345b 9765 46<br />
00003460 9765 50<br />
00003465 9765 45<br />
00003466 9762 40<br />
0000347c 9765 54<br />
0000347d 9765 61<br />
00003497 9765 63<br />
000034b3 9765 64<br />
000034bd 9765 54<br />
000034c9 9764 71<br />
000034d6 9765 65<br />
000034d9 9765 86<br />
000034da 9765 46<br />
000034df 9765 51<br />
000034e0 9764 60<br />
00003504 9762 80<br />
00003512 9761 76<br />
0000351b 9765 65<br />
00003525 9765 59<br />
00003532 9765 52<br />
00003540 9764 43<br />
00003553 9765 80<br />
00003558 9765 54<br />
00003580 9762 49<br />
00003586 9765 41<br />
00003587 9763 50<br />
0000359c 9765 40<br />
000035a0 9765 46<br />
000035a1 9765 37<br />
000035a6 9765 96<br />
000035ab 9765 52<br />
000035ac 9765 39<br />
000035c4 9765 61<br />
000035d0 9765 89<br />
000035dc 9765 74<br />
000035dd 9765 43<br />
000035f1 9765 43<br />
000035fd 9765 37<br />
00003604 9765 47<br />
00003609 9764 40<br />
0000360c 9765 59<br />
0000360d 9765 37<br />
00003618 9763 42<br />
00003621 9765 42<br />
00003626 9765 42<br />
00003627 9765 41<br />
0000362a 9765 38<br />
0000362b 9765 73<br />
00003630 9765 68<br />
00003631 9764 42<br />
0000363e 9763 39<br />
00003643 9765 43<br />
00003646 9765 38<br />
0000364c 9762 40<br />
00003659 9765 47<br />
0000365c 9764 44<br />
0000365f 9765 45<br />
00003664 9765 69<br />
00003668 9764 37<br />
00003671 9765 43<br />
00003676 9765 39<br />
00003688 9765 37<br />
00003691 9765 48<br />
00003692 9764 37<br />
00003695 9762 38<br />
000036a3 9764 50<br />
000036aa 9765 47<br />
000036ab 9765 49<br />
000036c2 9765 64<br />
000036c3 9764 57<br />
000036c5 9765 40<br />
000036da 9764 40<br />
000036e1 9765 43<br />
000036ed 9762 39<br />
000036fc 9765 41<br />
000036fd 9765 59<br />
00003701 9764 85<br />
00003708 9765 43<br />
0000370a 9763 49<br />
00003728 9762 41<br />
0000372a 9763 55<br />
0000372d 9765 51<br />
00003732 9763 48<br />
00003734 9765 60<br />
00003736 9765 42<br />
00003740 9765 48<br />
00003741 9765 39<br />
00003742 9765 38<br />
00003749 9764 40<br />
00003756 9763 39<br />
00003758 9762 46<br />
0000375c 9765 67<br />
0000375d 9765 37<br />
00003761 9765 63<br />
00003779 9765 55<br />
00003785 9765 43<br />
00003798 9763 60<br />
0000379b 9765 41<br />
0000379e 9765 37<br />
000037a5 9764 39<br />
000037a8 9764 48<br />
000037ab 9765 38<br />
000037ae 9765 54<br />
000037b3 9764 42<br />
000037b6 9761 36<br />
000037be 9763 46<br />
000037c1 9765 69<br />
000037c3 9765 36<br />
000037c4 9764 37<br />
000037ce 9762 50<br />
000037d1 9761 45<br />
000037d3 9765 45<br />
000037dd 9765 45<br />
000037df 9765 50<br />
000037e0 9763 39<br />
000037e2 9765 13<br />
000037e3 9763 18<br />
000037e7 9765 49<br />
000037e8 9763 41<br />
000037e9 9765 42<br />
000037ea 9763 40<br />
000037eb 9765 39<br />
000037ec 9761 39<br />
000037ed 9763 41<br />
000037ef 9763 36<br />
000037f0 9764 36<br />
000037f1 9764 39<br />
000037f3 9765 41<br />
000037f4 9761 53<br />
000037f5 9765 36<br />
000037f6 9765 54<br />
000037f7 9763 53<br />
000037f8 9765 63<br />
000037f9 9765 60<br />
000037fa 9765 70<br />
000037fb 9765 76<br />
000037fc 9765 72<br />
000037fd 9765 73<br />
000037fe 9765 83<br />
000037ff 9765 87<br />
00003800 9765 89<br />
00003801 9762 86<br />
00003802 9763 88<br />
00003803 9764 90<br />
00003804 8110 76<br />
TOTALS 3757891 49<br />
<br />
* It will tell you for each of the database files (the log segments) how big it is and what the utilization is. The Berkeley DB will try to keep the total utilization above 50%, but for a small pool like this one it cannot do it. I figure that you may find this interesting to know after reading about utilization above.<br />
<br />
* The log segment size and the utilization goal can be adjusted too, but I cannot give any sound advice on whether that’s a good idea and how to determine good values.<br />
<br />
* All of the above is only relevant if you use the Berkeley DB backend for pools.<br />
<br />
== Special thanks ==<br />
* This documentation has been deployed thanks to Gerd Behrmann, you can find more in at: https://gist.github.com/gbehrmann/df1abbe45c03356eff2e<br />
* Read more about this at: https://backstage.forgerock.com/knowledge/kb/book/b74223337/a49979000</div>Mcaubethttps://pwiki.pic.es/index.php?title=DCACHE_CHECK_POOL_BTREE_MEMORY&diff=228DCACHE CHECK POOL BTREE MEMORY2017-10-06T08:38:06Z<p>Mcaubet: /* Special thanks */</p>
<hr />
<div>= For the MoD =<br />
* Open a ticket to incidents@support.pic.es with the alarm. <br />
* This alarm only affects the duration of the pool start up, nothing has to be done until the next working day where this can be fixed by dCache Administrators.<br />
* Same information as [[DCACHE CHECK POOL METADATA]] as both alarms are strictly related.<br />
<br />
= For the Admin =<br />
<br />
* Berkely DB should be tunned according to the number of meta files that a pool contains. If this not correctly adapted, pool start up will may need several hours to be fully available. Gerd explained how to tune up a pool according to this number. Below I show a modified explanation of what he said, adapting the explanation to the ATLAS pools.<br />
<br />
* The default je.log.fileCacheSize is 100. This could be increased by creating a file je.properties and adding the 'je.log.fileCacheSize=100'.<br />
<br />
* Also, other relevant settings are:<br />
<br />
je.maxMemory=<br />
je.maxMemoryPercent=20<br />
<br />
* These are equivalent. The former sets the number of bytes used for the btree cache, while the latter defines it as a percentage of the max heap size. E.g. if dcache.java.memory.heap is set to 2048m, then 410 MB is used for the btree cache.<br />
<br />
* The question is then how to determine a good size for the cache. There are some hidden utilities you can use to do this.<br />
<br />
* Below I show an example for dc042_1:<br />
<br />
[root@dc042 meta]# java -cp /usr/share/dcache/classes/je-6.3.8.jar com.sleepycat.je.util.DbPrintLog -h . -S<br />
<DbPrintLog><br />
Log statistics:<br />
type total provisional total min max avg entries<br />
count count bytes bytes bytes bytes as % of log<br />
MapLN 1,324 0 12,617,573 76 11,985 9,529 0.3<br />
NameLN 4 0 141 31 38 35 0<br />
FileSummaryLN 231,140 0 55,644,337 24 49,701 240 1.4<br />
IN 84,083 1,080 296,942,059 45 5,709 3,531 7.7<br />
BIN 114,865 114,865 377,569,148 44 6,675 3,287 9.8<br />
DbTree 318 0 1,034,997 2,680 4,136 3,254 0<br />
Commit 3,376,012 0 114,784,408 34 34 34 3<br />
CkptStart 158 0 4,899 30 33 31 0<br />
CkptEnd 226 0 22,560 67 186 99 0<br />
Trace 2,506 0 525,597 48 402 209 0<br />
FileHeader 385 0 14,630 38 38 38 0<br />
DEL_LN_TX 748,055 0 51,409,527 57 69 68 1.3<br />
INS_LN_TX 619,747 0 192,298,301 96 1,443 310 5<br />
UPD_LN_TX 2,008,226 0 398,854,718 32 1,919 198 10.4<br />
UPD_LN 6,636,173 0 1,997,042,458 18 1,819 300 51.9<br />
NewBINDelta 720,540 720,540 348,480,254 55 1,782 483 9.1<br />
key bytes 10,012,201 387,254,450 1 61 38 (10.1)<br />
data bytes 9,264,146 2,036,632,128 1 1,848 219 (52.9)<br />
<br />
Total bytes in portion of log read: 3,847,245,607<br />
Total number of entries: 14,543,762<br />
<br />
Per checkpoint interval info:<br />
lnTxn ln mapLNTxn mapLN end to end end to start start to end maxLNReplay ckptEnd<br />
273 63,082 0 10 30,455,407 3,857,456,063 0 63,355 0x1f2f/0x6f2ef<br />
5,168 130,892 0 24 39,782,369 3,827,000,656 0 136,060 0x226b/0x3a0d0<br />
27,098 132,905 0 15 50,120,541 16,509,395 33,611,146 160,003 0x252c/0x577ad<br />
128 32,773 0 3 9,904,160 3,737,097,746 0 134,154 0x254c/0x4014d<br />
82 31,341 0 12 10,576,583 3,727,193,586 0 31,423 0x2567/0xccd94<br />
0 72 0 4 711,899 51,780 660,119 72 0x2567/0x17aa6f<br />
36,445 61,509 0 0 28,499,838 3,715,905,104 0 97,954 0x25e5/0xc66d<br />
2 108 0 6 1,564,919 56,871 1,508,048 110 0x25e5/0x18a764<br />
10,732 50,299 0 5 18,877,564 3,685,840,347 0 61,055 0x2678/0x786e0<br />
33 190 0 6 1,627,301 94,279 1,533,022 223 0x2678/0x205b85<br />
8,277 84,136 0 3 28,217,133 17,808,699 10,408,434 92,434 0x26be/0x52732<br />
5,401 25,394 0 7 9,824,151 3,637,118,349 0 61,029 0x26cf/0x27849<br />
49,026 92,572 0 3 41,592,807 3,627,294,198 0 141,598 0x2747/0x1ac630<br />
2 46 0 6 357,170 45,107 312,063 48 0x2747/0x203962<br />
20,496 39,154 0 3 18,591,447 3,585,344,221 0 59,654 0x27db/0xabb39<br />
0 154 0 6 1,187,463 83,676 1,103,787 154 0x27db/0x1cd9c0<br />
17,107 49,468 0 3 18,905,722 3,565,565,311 0 66,583 0x285c/0xc273a<br />
58 0 0 6 392,501 62,920 329,581 58 0x285c/0x12246f<br />
113,890 201,969 0 17 79,488,418 3,546,267,088 0 315,874 0x28ed/0xa5611<br />
22,147 98,579 0 8 40,657,515 36,319,235 4,338,280 120,726 0x2952/0x145e7c<br />
0 122 0 6 1,296,043 70,361 1,225,682 28,367 0x2952/0x282527<br />
19,570 65,944 0 5 27,570,660 25,251,492 2,319,168 85,520 0x2962/0x3138b<br />
37,714 28,889 0 3 21,505,514 19,337,884 2,167,630 66,613 0x297d/0x1a0c75<br />
3 332 0 6 2,437,155 124,630 2,312,525 479 0x297d/0x3f3c98<br />
41,458 62,406 0 3 26,385,897 3,373,311,783 0 103,904 0x29c7/0x81701<br />
0 46 0 6 1,404,642 46,045 1,358,597 46 0x29c7/0x1d85e3<br />
41,443 87,747 0 3 39,909,271 14,624,816 25,284,455 129,198 0x29fb/0x1c237a<br />
0 123 0 5 1,185,318 68,652 1,116,666 79,470 0x29fb/0x2e39a0<br />
9,199 12,698 0 4 9,927,451 3,304,426,655 0 21,899 0x2a00/0x2d1e3b<br />
8 0 0 5 118,374 35,364 83,010 8 0x2a00/0x2eeca1<br />
122,697 96,787 0 3 66,966,017 3,294,380,830 0 219,488 0x2a44/0xa122<br />
53,436 12,002 0 3 21,830,901 3,227,414,813 0 65,438 0x2a53/0x1c9117<br />
20 167 0 6 1,308,679 85,964 1,222,715 187 0x2a53/0x30891e<br />
18,552 38,805 0 6 17,501,977 3,204,275,233 0 57,381 0x2a58/0xa6b37<br />
9 105 0 4 678,297 87,871 590,426 114 0x2a58/0x14c4d0<br />
25,434 36,231 0 10 19,064,711 3,186,094,959 0 61,675 0x2a5e/0x67f57<br />
15 159 0 6 1,467,746 83,228 1,384,518 174 0x2a5e/0x1ce4b9<br />
12,236 14,099 0 3 8,375,785 3,165,562,502 0 26,358 0x2a63/0x41c22<br />
6 184 0 6 1,058,988 89,948 969,040 190 0x2a63/0x1444ce<br />
71,844 24,018 0 3 29,188,061 27,937,328 1,250,733 95,870 0x2aac/0x7e12b<br />
9,407 25,608 0 7 11,016,096 3,126,939,668 0 35,029 0x2aba/0x17624b<br />
20 110 0 6 748,510 82,108 666,402 130 0x2aba/0x22ce29<br />
64,997 49,072 0 15 27,822,548 27,438,951 383,597 114,142 0x2af3/0x1947d<br />
10,534 23,197 0 6 12,103,837 3,087,352,514 0 33,737 0x2b01/0x21ae9a<br />
122 0 0 32 879,716 489,740 389,976 122 0x2b01/0x2f1afe<br />
60,029 128,431 0 24 58,556,679 56,338,979 2,217,700 188,485 0x2b3c/0x191505<br />
15 52 0 6 530,374 53,795 476,579 231 0x2b3c/0x212ccb<br />
48,799 11,045 0 4 18,178,427 17,837,860 340,567 59,855 0x2b41/0x56146<br />
45,998 29,564 0 3 20,154,236 2,997,103,481 0 75,573 0x2b52/0x7bbc2<br />
39,570 60,757 0 15 30,273,541 2,976,949,245 0 100,327 0x2b8e/0xbe847<br />
31,631 33,780 0 13 22,348,448 16,815,306 5,533,142 65,411 0x2bad/0x2fbde7<br />
16,831 40,178 0 5 20,571,223 2,924,327,256 0 57,047 0x2bc6/0x38753e<br />
4,292 18,826 0 8 8,363,076 2,903,756,033 0 23,118 0x2be9/0x1f7b02<br />
0 254 0 6 2,043,974 119,900 1,924,074 254 0x2be9/0x3eab48<br />
30 19,902 0 4 7,727,051 2,893,348,983 0 19,954 0x2c00/0x1bfc93<br />
27,984 75,433 0 32 29,896,927 2,885,621,932 0 103,417 0x2c13/0x1a69f2<br />
40,477 36,001 0 10 19,737,838 17,591,303 2,146,535 76,478 0x2c2a/0x1669e0<br />
2 2 0 4 194,668 36,808 157,860 10 0x2c2a/0x19624c<br />
29,246 68,367 0 3 31,325,183 2,835,792,499 0 97,615 0x2c4c/0x2d9acb<br />
0 26 0 5 1,118,302 27,604 1,090,698 26 0x2c4c/0x3eab29<br />
4 19,763 0 3 7,562,665 2,803,349,014 0 19,775 0x2c53/0x197a52<br />
26 194 0 6 1,436,963 103,803 1,333,160 220 0x2c53/0x2f6775<br />
6,255 14,988 0 8 9,972,102 2,794,349,386 0 21,292 0x2c5f/0x2efa7b<br />
39,212 18,305 0 5 18,591,243 2,784,377,284 0 57,517 0x2c8f/0x197b86<br />
0 41 0 6 1,495,215 121,720 1,373,495 41 0x2c8f/0x304c35<br />
347 23,340 0 3 7,036,643 2,764,290,826 0 23,697 0x2c95/0x31498<br />
23,408 9,116 0 3 10,098,528 2,757,254,183 0 32,524 0x2ca4/0x49578<br />
45,631 27,183 0 3 19,972,142 2,747,155,655 0 72,814 0x2cad/0x428a6<br />
0 0 0 1 963,802 69 963,733 0 0x2cad/0x12dd80<br />
276 19,066 0 4 10,705,056 5,926,693 4,778,363 19,342 0x2cc4/0x1d9fa0<br />
0 25,292 0 2 10,568,825 7,122,257 3,446,568 25,292 0x2ccf/0x264d99<br />
722 25,106 0 3 9,376,076 2,704,945,830 0 26,568 0x2cd3/0x1cc865<br />
15 96 0 6 601,987 65,907 536,080 111 0x2cd3/0x25f7e8<br />
50,662 51,942 0 4 29,283,798 2,694,967,767 0 102,611 0x2d20/0x1b0a3e<br />
18,486 16,140 0 4 12,291,893 2,665,683,969 0 34,626 0x2d3a/0x3e02f3<br />
67,481 104,443 0 5 48,233,597 15,261,303 32,972,294 171,924 0x2d79/0x230ef0<br />
19,276 34,062 0 3 18,103,592 2,605,158,479 0 171,476 0x2deb/0x61f18<br />
0 225 0 6 607,824 37,538 570,286 225 0x2deb/0xf6568<br />
14,038 14,130 0 3 13,699,081 2,586,447,063 0 28,385 0x2e21/0x47d6f1<br />
4,147 13,212 0 3 5,363,836 2,572,747,982 0 17,359 0x2e40/0x118ed<br />
0 0 0 4 133,826 37,921 95,905 0 0x2e40/0x323af<br />
31,744 33,396 0 3 22,247,625 2,567,250,320 0 65,140 0x2e45/0x256f78<br />
9 0 0 6 290,601 55,855 234,746 9 0x2e45/0x29dea1<br />
16 25,020 0 4 10,475,420 2,544,712,094 0 25,039 0x2e5b/0x311fbd<br />
1,682 19,008 0 3 8,507,134 2,534,236,674 0 20,690 0x2e5f/0x1a583b<br />
12 0 0 6 354,107 41,047 313,060 12 0x2e5f/0x1fbf76<br />
6,849 54,863 0 3 20,176,980 2,525,375,433 0 61,719 0x2e7d/0x2272ca<br />
30,012 67,588 0 5 28,465,075 27,606,214 858,861 97,600 0x2eb6/0xb06fd<br />
14,048 16,912 0 2 9,587,676 2,476,733,378 0 66,881 0x2ee1/0x4bc59<br />
15,183 12,704 0 6 11,270,153 6,834,792 4,435,361 27,887 0x2ef4/0x181de2<br />
23,012 39,873 0 7 18,610,316 2,455,875,549 0 62,948 0x2f1a/0x2e96e<br />
25,472 49,151 0 6 22,681,935 2,437,265,233 0 74,623 0x2f5a/0x2bd5bd<br />
1,031 60,319 0 6 20,425,359 2,414,583,298 0 61,350 0x2f66/0x32534c<br />
48,185 83,225 0 4 42,656,335 28,813,330 13,843,005 131,410 0x2f9d/0x5adb9b<br />
16,859 32,139 0 3 18,804,637 12,310,056 6,494,581 74,600 0x2fa3/0x489e38<br />
29,050 79,391 0 8 47,784,742 37,879,192 9,905,550 108,511 0x300a/0x26d0de<br />
32 38,306 0 6 19,991,849 14,244,764 5,747,085 117,588 0x3026/0x26b107<br />
0 37 0 6 345,851 39,476 306,375 25,114 0x3026/0x2bf802<br />
17,725 34,199 0 3 19,155,357 2,264,574,525 0 51,955 0x303f/0x1f149f<br />
1,379 23,909 0 6 10,787,679 2,245,419,168 0 25,288 0x304f/0x2b197e<br />
3 12 0 5 297,677 34,135 263,542 15 0x304f/0x2fa44b<br />
17,968 47,908 0 2 17,014,500 2,234,333,812 0 65,891 0x306c/0x2162f<br />
45,151 19,691 0 3 20,592,703 18,998,392 1,594,311 64,842 0x3071/0xb216e<br />
18,246 13,219 0 3 14,900,440 2,196,726,609 0 31,514 0x308d/0x55e7c6<br />
26 12,008 0 5 7,554,365 4,234,132 3,320,233 12,034 0x30a1/0x309683<br />
35,556 69,762 0 20 31,161,221 2,174,271,804 0 105,338 0x30c2/0x424e88<br />
33,203 114,692 0 5 49,168,609 43,804,782 5,363,827 147,895 0x30fd/0x359ee9<br />
182 19,563 0 7 10,363,229 2,093,941,974 0 87,398 0x310c/0x3b29c6<br />
14 19,817 0 4 9,355,037 4,743,642 4,611,395 19,831 0x3110/0x315263<br />
46 37,708 0 8 20,678,814 10,884,622 9,794,192 41,674 0x315c/0x3bae01<br />
88 279 0 5 761,100 44,429 716,671 3,390 0x315c/0x474b0d<br />
40,709 61,572 0 4 26,837,961 2,052,783,794 0 102,607 0x3189/0x170b56<br />
42 0 0 6 814,957 77,642 737,315 42 0x3189/0x237ac3<br />
69,126 107,504 0 5 52,094,116 47,372,898 4,721,218 176,645 0x31b8/0x436ee7<br />
46 0 0 5 393,197 51,476 341,721 82,129 0x31b8/0x496ed4<br />
48,348 117,823 0 3 48,879,763 41,965,088 6,914,675 166,196 0x3201/0x3856e7<br />
16,349 39,063 0 7 17,875,388 14,606,658 3,268,730 62,982 0x3208/0x17eba3<br />
105 0 0 6 3,394,912 60,919 3,333,993 2,571 0x3208/0x4bb903<br />
217 17,310 0 7 5,776,110 1,902,493,500 0 17,617 0x3210/0xb4571<br />
36,657 27,268 0 7 22,160,771 4,617,088 17,543,683 63,925 0x3224/0x2c3df4<br />
24,153 42,022 0 1 22,019,603 1,874,556,619 0 114,113 0x3235/0x4b0f07<br />
16 18,174 0 7 7,706,321 1,852,537,016 0 18,190 0x324e/0x280f58<br />
120 24,379 0 7 10,458,169 1,844,830,695 0 24,499 0x3266/0x2f0d11<br />
14,236 34,916 0 3 17,998,326 12,723,415 5,274,911 49,152 0x327b/0x108207<br />
79 0 0 5 673,666 37,699 635,967 2,808 0x327b/0x1ac989<br />
6,624 22,958 0 3 11,999,224 1,815,700,534 0 29,650 0x3290/0x394b01<br />
19,965 40,810 0 8 17,269,398 1,803,701,310 0 60,775 0x32a7/0xfa097<br />
7,517 58,175 0 5 21,772,528 1,786,431,912 0 65,692 0x32de/0x2aac87<br />
9,415 18,600 0 6 10,786,946 1,764,659,384 0 28,015 0x32e1/0x36ae89<br />
51 0 0 6 622,591 52,177 570,414 51 0x32e1/0x402e88<br />
30,139 66,899 0 4 30,497,693 1,753,249,847 0 97,065 0x3315/0x47c6a5<br />
6 41,284 0 1 16,342,999 12,394,744 3,948,255 41,290 0x3320/0xff97c<br />
21,091 17,059 0 6 11,525,646 1,706,409,155 0 38,984 0x332b/0x27410a<br />
0 94 0 4 892,946 34,897 858,049 94 0x332b/0x34e11c<br />
3 52,035 0 3 18,160,444 1,693,990,563 0 52,132 0x336d/0x18cf58<br />
58 27,035 0 6 8,947,956 1,675,830,119 0 27,093 0x3376/0x8c1cc<br />
3 66 0 4 501,044 49,007 452,037 69 0x3376/0x106700<br />
27,134 44,130 0 4 22,994,532 1,666,381,119 0 71,330 0x339f/0x3e1864<br />
0 108 0 4 439,894 35,044 404,850 108 0x339f/0x44ceba<br />
17 51,506 0 3 20,284,195 1,642,946,693 0 51,631 0x33a6/0x4924dd<br />
28 16,826 0 2 9,954,053 1,622,662,498 0 16,854 0x33b6/0x487162<br />
2,079 43,231 0 2 20,850,856 13,682,453 7,168,403 45,310 0x33d2/0x556d0a<br />
2 30,027 0 3 14,418,091 8,924,719 5,493,372 30,743 0x33d9/0x40b5<br />
0 1,157 0 6 5,510,215 155,394 5,354,821 3,103 0x33d9/0x5454fc<br />
31,517 31,808 0 3 18,110,480 1,571,929,283 0 64,085 0x33ee/0x37800c<br />
30,598 27,193 0 4 18,121,300 12,201,650 5,919,650 57,791 0x3402/0x1ad560<br />
38,905 34,440 0 4 18,443,706 15,100,893 3,342,813 78,249 0x340c/0x3161a<br />
51,887 95,209 0 4 40,262,214 29,791,799 10,470,415 152,283 0x343b/0x71660<br />
1 28,686 0 7 10,081,465 1,476,991,583 0 142,625 0x344c/0x85499<br />
28,401 13,707 0 3 10,169,877 1,466,910,118 0 42,108 0x3457/0xaec2e<br />
5 16,531 0 6 14,562,193 4,402,715 10,159,478 16,536 0x345b/0x50893f<br />
0 15,920 0 7 9,514,322 1,442,178,048 0 19,604 0x3460/0x492011<br />
4 16,264 0 6 10,131,895 1,432,663,726 0 16,268 0x3465/0x4b2348<br />
30,231 28,986 0 4 18,938,103 14,093,138 4,844,965 59,217 0x347c/0x3aef3f<br />
14 0 0 5 512,006 34,432 477,574 120 0x347c/0x42bf45<br />
20,678 38,294 0 3 17,101,301 1,403,081,722 0 58,975 0x3497/0x16843a<br />
9,869 25,099 0 5 12,548,707 1,385,980,421 0 34,968 0x34b3/0x3d681d<br />
36,465 60,605 0 5 26,361,794 24,558,695 1,803,099 97,070 0x34d6/0x5e45f<br />
36 23,345 0 3 9,621,974 6,699,792 2,922,182 23,911 0x34d9/0x1fb5<br />
25,436 49,579 0 7 20,880,498 18,608,524 2,271,974 75,850 0x34df/0xd8f27<br />
32 1,485 0 6 4,197,555 413,166 3,784,389 1,535 0x34df/0x4d9bda<br />
31,535 59,968 0 5 25,248,912 24,728,797 520,115 91,849 0x3512/0x51cea<br />
1,601 24,378 0 6 10,268,411 8,200,696 2,067,715 26,160 0x351b/0x93565<br />
14,147 23,162 0 8 11,300,331 1,276,852,570 0 37,400 0x3525/0x1d0cd0<br />
14,204 20,662 0 3 13,299,117 1,265,552,239 0 34,866 0x3532/0x4f63fd<br />
17,514 24,728 0 3 16,797,576 12,513,187 4,284,389 42,242 0x3553/0x1e8685<br />
7,303 46,343 0 7 23,094,370 14,762,461 8,331,909 54,344 0x3580/0x4dbde7<br />
45,620 41,812 0 7 27,514,220 21,511,990 6,002,230 90,879 0x359c/0x27cfd3<br />
16,861 12,552 0 3 9,619,073 1,184,846,956 0 104,424 0x35a0/0x21ffd4<br />
0 271 0 6 3,731,928 93,043 3,638,885 271 0x35a0/0x5af1ac<br />
30,860 48,187 0 7 26,338,978 24,109,627 2,229,351 79,097 0x35ab/0x2314ce<br />
39,031 24,255 0 4 20,202,284 1,145,156,977 0 63,301 0x35c4/0x262afa<br />
2,576 46,002 0 6 19,184,048 5,580,647 13,603,401 48,578 0x35dc/0x19b7aa<br />
2 0 0 6 174,028 33,089 140,939 31,536 0x35dc/0x1c5f76<br />
111,528 77,161 0 7 62,341,581 50,549,489 11,792,092 188,690 0x360c/0x401a43<br />
28,697 25,910 0 6 18,246,350 13,855,848 4,390,502 56,636 0x3618/0x255811<br />
4,335 12,127 0 6 9,004,926 5,130,460 3,874,466 18,985 0x3621/0x16290f<br />
18,413 8,607 0 5 8,878,516 7,701,363 1,177,153 27,027 0x3626/0x50c43<br />
0 2,910 0 6 6,228,723 848,601 5,380,122 2,910 0x3626/0x641736<br />
28,784 22,689 0 3 21,119,867 14,626,185 6,493,682 52,178 0x362a/0x752db1<br />
41,676 62,528 0 4 34,112,988 29,303,765 4,809,223 105,238 0x363e/0x1b598d<br />
17 735 0 5 12,521,216 31,105 12,490,111 1,300 0x3643/0x41d20d<br />
7,802 34,494 0 7 33,127,169 7,514,191 25,612,978 43,003 0x3659/0x71898e<br />
0 8,734 0 5 3,377,408 900,015,281 0 25,130 0x365c/0xc7c0e<br />
114 9,858 0 7 15,782,161 650,578 15,131,583 9,972 0x365f/0x64b69f<br />
1 11,516 0 7 5,378,418 880,855,712 0 19,548 0x3664/0x1e3191<br />
7 22,005 0 7 8,922,863 6,691,187 2,231,676 22,012 0x3668/0xdc200<br />
13 13,409 0 6 15,952,421 360,887 15,591,534 14,391 0x3671/0x6895a5<br />
1,749 8,568 0 3 3,660,457 850,602,010 0 22,698 0x3676/0x7d9ce<br />
5,390 10,146 0 6 14,882,555 1,855,609 13,026,946 15,536 0x3688/0x525a49<br />
10,230 9,513 0 5 4,816,061 832,058,998 0 28,465 0x3691/0x34086<br />
14,698 31,907 0 7 41,080,361 36,925,183 4,155,178 46,605 0x36aa/0x13bcaf<br />
2 1,755 0 6 4,666,159 345,619 4,320,540 47,630 0x36aa/0x5aefde<br />
25,935 30,385 0 5 38,398,767 781,496,417 0 57,119 0x36c5/0x42810d<br />
9,478 14,696 0 7 21,102,013 743,097,650 0 24,174 0x36e1/0x5351ca<br />
65 12,746 0 6 5,610,587 3,858,013 1,752,574 12,811 0x36ed/0x1057a5<br />
5,806 9,341 0 6 10,353,968 2,690,013 7,663,955 15,343 0x36fc/0x15be55<br />
5 0 0 6 4,894,393 33,744 4,860,649 5,931 0x36fc/0x606d0e<br />
28,754 58,230 0 18 28,401,618 701,136,689 0 86,988 0x3708/0x480960<br />
0 15,897 0 6 10,890,931 3,098,266 7,792,665 15,897 0x370a/0x55a193<br />
126 10,927 0 6 26,068,972 2,214,107 23,854,865 16,100 0x372d/0x19a5ff<br />
9,837 16,155 0 6 11,096,409 6,346,816 4,749,593 29,696 0x3732/0x2a60d8<br />
128 15,250 0 6 8,591,606 3,747,801 4,843,805 16,046 0x3734/0x14e34e<br />
123 28,336 0 6 16,047,724 5,161,792 10,885,932 31,737 0x3736/0x712b3a<br />
24 8,534 0 5 3,423,605 1,552,888 1,870,717 19,982 0x3740/0xcd22f<br />
533 0 0 6 20,325,930 275,776 20,050,154 3,884 0x3742/0x11cb59<br />
0 0 0 4 72,242 36,230 36,012 0 0x3742/0x12e58b<br />
26,449 16,768 0 3 20,777,536 576,217,652 0 43,217 0x3756/0x1ec2cb<br />
11,493 23,349 0 3 21,836,979 13,435,005 8,401,974 34,842 0x375c/0x3aca7e<br />
309 32,239 0 6 17,558,766 9,588,731 7,970,035 56,790 0x3761/0x158a6c<br />
12 26,562 0 7 12,407,639 6,425,279 5,982,360 27,257 0x3779/0x3a4743<br />
6,845 41,908 0 7 33,072,537 22,775,381 10,297,156 51,339 0x379b/0x69295c<br />
6,778 29,990 0 6 25,664,453 17,110,934 8,553,519 41,294 0x37a8/0x2701a1<br />
5 16,509 0 9 19,648,397 8,979,547 10,668,850 44,387 0x37ae/0x21a42e<br />
116,569 137,423 0 12 129,250,694 72,386,145 56,864,549 267,033 0x37e2/0x163534<br />
13 17,546 0 6 28,042,727 34,685 28,008,042 187,464 0x37e7/0x90ee1b<br />
33,326 30,872 0 6 43,125,925 9,805,827 33,320,098 81,756 0x37ec/0x280a40<br />
3 5,332 0 5 19,873,538 37,653 19,835,885 30,103 0x37ef/0x261c42<br />
17 2,306 0 6 19,166,423 244,987 18,921,436 7,657 0x37f1/0x196419<br />
10,753 20,277 0 6 40,648,779 11,079,498 29,569,281 32,618 0x37f6/0x234a64<br />
10 22,876 0 6 18,391,375 164,634 18,226,741 49,034 0x37f8/0xabeb3<br />
30 34,377 0 7 32,006,385 1,775,081 30,231,304 56,545 0x37fb/0x295c24<br />
4 23,090 0 7 38,624,657 2,288,459 36,336,198 52,101 0x37ff/0x145fb5<br />
6 8,689 0 6 27,108,848 949,959 26,158,889 24,449 0x3801/0x80d8a5<br />
6 5,117 0 7 18,644,436 55,620 18,588,816 11,221 0x3803/0x6c2979<br />
837 0 0 6 6,661,670 288,422 6,373,248 5,956 0x3804/0x39391f<br />
5,863 6,917 0 0 3,705,888 3,705,888 0 12,780 0x3804/0x71c53f<br />
</DbPrintLog><br />
<br />
* This prints some statistics about the database. In particular the two rows called key bytes and data bytes are relevant. For the following step you need the values from the avg column - i.e. 38 and 219 in this case. You also need the value of “Total number of entries” (14543762 in this case). Now we take those values and put them into this command:<br />
<br />
[root@dc042 meta]# java -cp /usr/share/dcache/classes/je-*.jar com.sleepycat.je.util.DbCacheSize -records 14543762 -key 38 -data 219<br />
<br />
=== Environment Cache Overhead ===<br />
<br />
3,157,213 minimum bytes<br />
<br />
To account for JE daemon operation and record locks,<br />
a significantly larger amount is needed in practice.<br />
<br />
=== Database Cache Size ===<br />
<br />
Number of Bytes Description<br />
--------------- -----------<br />
1,195,759,832 Internal nodes only<br />
5,124,232,392 Internal nodes and leaf nodes<br />
<br />
* For further information see the DbCacheSize javadoc.<br />
<br />
* For this pool it tells us that we need a bit above 1195 MB to keep all the internal btree nodes cached. Oracle’s recommendation is that if the database is updated often, the cache is at least big enough to contain the internal nodes. Obviously you want to make it a bit bigger to leave room for it to grow. One could take this and configure the cache using the je.maxMemory setting. If the size is lower than the 20% of the max heap size you already use, then you don’t need to do anything (I do not suggest lowering it further).<br />
<br />
* Since this pool has 6000m of JAVA heap max space, it means that 20% = 1200MB, which is exactly cache for Internal btree nodes. So we will not touch this.<br />
<br />
* There is one caveat though: If you increase the cache size, less free space is left on the heap. If the pool is pushed to the limit, this may actually slow down the pool as garbage collection overhead increases. You need to ensure that enough space is left in addition to the cache (possibly by increasing the max heap size). An alternative to adjusting je.maxMemory is of course to adjust the max heap size. If you make it large enough so that 20% is enough to cache the internal btree nodes, then all is well. This may however mean you assign significantly more memory to the pool than it really needs. Yeah, there are lots of things to consider.<br />
<br />
* A little bonus: There is also this command:<br />
<br />
[root@dc042 meta]# java -cp /usr/share/dcache/classes/je-*.jar com.sleepycat.je.util.DbSpace -h .<br />
File Size (KB) % Used<br />
-------- --------- ------<br />
00001857 9765 41<br />
00001f2a 9765 43<br />
00001f2f 9765 39<br />
00001f72 9765 44<br />
0000216d 9765 40<br />
000021cb 9765 40<br />
0000226b 9765 45<br />
000022e8 9763 38<br />
000022fa 9765 39<br />
00002390 9765 38<br />
0000241d 9765 49<br />
0000252c 9765 51<br />
0000254c 9765 47<br />
00002567 9765 39<br />
000025b7 9765 42<br />
000025db 9765 42<br />
000025e5 9765 56<br />
0000265f 9765 39<br />
00002678 9765 61<br />
0000268d 9762 48<br />
00002692 9765 74<br />
000026be 9765 48<br />
000026cf 9765 40<br />
00002700 9765 41<br />
00002706 9765 38<br />
00002718 9765 56<br />
00002747 9765 44<br />
0000274d 9765 48<br />
000027db 9765 62<br />
00002855 9765 39<br />
0000285c 9765 39<br />
00002882 9765 41<br />
0000289d 9765 54<br />
000028ba 9765 52<br />
000028c1 9765 53<br />
000028d0 9765 41<br />
000028d7 9765 41<br />
000028e7 9765 39<br />
000028ed 9765 39<br />
00002926 9765 39<br />
0000292b 9765 43<br />
00002945 9765 41<br />
00002952 9765 41<br />
00002958 9765 62<br />
00002959 9764 38<br />
00002962 9765 53<br />
00002963 9765 47<br />
0000297d 9765 51<br />
0000297e 9765 41<br />
000029b3 9765 45<br />
000029c7 9765 56<br />
000029ce 9765 42<br />
000029d3 9765 43<br />
000029d9 9765 54<br />
000029fb 9765 42<br />
00002a00 9765 54<br />
00002a01 9765 39<br />
00002a06 9765 39<br />
00002a20 9765 45<br />
00002a26 9765 61<br />
00002a2b 9765 69<br />
00002a30 9765 65<br />
00002a44 9765 46<br />
00002a4f 9765 50<br />
00002a53 9765 64<br />
00002a54 9765 47<br />
00002a58 9765 66<br />
00002a59 9765 43<br />
00002a5e 9765 56<br />
00002a63 9765 44<br />
00002a69 9765 47<br />
00002a8f 9765 39<br />
00002aac 9765 54<br />
00002aba 9765 42<br />
00002abb 9765 44<br />
00002ae0 9763 38<br />
00002af3 9765 45<br />
00002b01 9765 41<br />
00002b08 9765 65<br />
00002b0e 9765 41<br />
00002b24 9765 45<br />
00002b28 9765 61<br />
00002b29 9765 41<br />
00002b3c 9765 40<br />
00002b3d 9765 40<br />
00002b41 9765 47<br />
00002b4c 9765 46<br />
00002b52 9765 57<br />
00002b64 9765 38<br />
00002b87 9765 38<br />
00002b8e 9765 47<br />
00002b8f 9760 38<br />
00002bad 9765 42<br />
00002bbf 9765 44<br />
00002bc6 9765 38<br />
00002be9 9765 47<br />
00002c00 9765 42<br />
00002c06 9765 69<br />
00002c0c 9765 55<br />
00002c13 9765 38<br />
00002c14 9765 38<br />
00002c2a 9765 41<br />
00002c2b 9765 41<br />
00002c30 9765 54<br />
00002c4c 9765 39<br />
00002c53 9765 45<br />
00002c5f 9765 44<br />
00002c7c 9765 38<br />
00002c8f 9765 49<br />
00002c95 9765 38<br />
00002ca4 9764 50<br />
00002ca9 9765 42<br />
00002cad 9763 41<br />
00002cc4 9765 39<br />
00002ccf 9765 43<br />
00002cd3 9765 45<br />
00002cf3 9765 48<br />
00002d19 9765 38<br />
00002d20 9765 38<br />
00002d3a 9765 40<br />
00002d3b 9765 41<br />
00002d41 9765 43<br />
00002d51 9765 39<br />
00002d59 9765 40<br />
00002d79 9765 62<br />
00002d8f 9765 39<br />
00002deb 9765 40<br />
00002e21 9765 39<br />
00002e40 9765 41<br />
00002e41 9765 45<br />
00002e45 9765 40<br />
00002e5b 9765 58<br />
00002e5f 9765 49<br />
00002e60 9765 67<br />
00002e7d 9765 48<br />
00002e95 9765 38<br />
00002eb1 9762 57<br />
00002eb6 9765 53<br />
00002ee1 9765 38<br />
00002ef4 9763 43<br />
00002ef8 9765 41<br />
00002f1a 9765 53<br />
00002f25 9765 65<br />
00002f5a 9765 52<br />
00002f5f 9765 63<br />
00002f66 9765 41<br />
00002f71 9765 42<br />
00002f79 9765 61<br />
00002f87 9765 49<br />
00002f9d 9765 41<br />
00002f9e 9765 42<br />
00002fa3 9765 38<br />
00002fa4 9765 46<br />
00002fb1 9762 55<br />
00002fc1 9765 41<br />
00002fde 9765 38<br />
0000300a 9765 43<br />
0000300e 9762 47<br />
00003026 9765 55<br />
00003027 9765 57<br />
0000303f 9765 63<br />
0000304f 9765 48<br />
00003056 9765 44<br />
0000306c 9765 48<br />
0000306d 9765 42<br />
00003071 9765 45<br />
0000308d 9765 39<br />
000030a1 9765 54<br />
000030a5 9765 51<br />
000030ab 9765 60<br />
000030c2 9765 46<br />
000030d5 9765 43<br />
000030dd 9765 38<br />
000030e3 9765 57<br />
000030e4 9765 55<br />
000030fd 9765 48<br />
0000310c 9764 42<br />
00003110 9765 46<br />
00003119 9765 47<br />
0000315c 9765 48<br />
0000315d 9765 54<br />
00003185 9765 56<br />
00003189 9765 52<br />
0000318a 9765 58<br />
0000319a 9765 46<br />
0000319f 9764 39<br />
000031b5 9764 53<br />
000031b8 9765 40<br />
000031b9 9765 74<br />
000031e2 9765 65<br />
000031fa 9765 58<br />
000031fb 9765 52<br />
00003201 9765 50<br />
00003202 9764 44<br />
00003208 9765 42<br />
00003210 9763 40<br />
00003218 9765 40<br />
00003224 9765 51<br />
00003229 9765 48<br />
00003235 9765 40<br />
0000324e 9765 46<br />
00003266 9765 41<br />
0000326c 9764 47<br />
0000327b 9765 55<br />
00003290 9765 48<br />
00003294 9765 56<br />
000032a7 9765 69<br />
000032ad 9765 62<br />
000032de 9765 47<br />
000032e1 9765 46<br />
000032e2 9765 71<br />
000032e9 9765 45<br />
00003315 9765 42<br />
00003316 9764 54<br />
00003320 9765 43<br />
0000332b 9765 52<br />
0000332c 9765 76<br />
0000336d 9765 69<br />
00003376 9765 72<br />
00003377 9765 46<br />
0000339f 9765 43<br />
000033a0 9765 75<br />
000033a6 9765 46<br />
000033b6 9765 40<br />
000033c0 9764 72<br />
000033d2 9765 38<br />
000033d3 9764 45<br />
000033d9 9765 42<br />
000033da 9765 44<br />
000033ee 9765 41<br />
000033fb 9764 38<br />
00003402 9765 42<br />
00003403 9765 53<br />
0000340c 9765 76<br />
00003428 9765 67<br />
00003431 9765 51<br />
00003436 9765 88<br />
0000343b 9765 85<br />
0000344c 9765 48<br />
00003457 9765 46<br />
0000345b 9765 46<br />
00003460 9765 50<br />
00003465 9765 45<br />
00003466 9762 40<br />
0000347c 9765 54<br />
0000347d 9765 61<br />
00003497 9765 63<br />
000034b3 9765 64<br />
000034bd 9765 54<br />
000034c9 9764 71<br />
000034d6 9765 65<br />
000034d9 9765 86<br />
000034da 9765 46<br />
000034df 9765 51<br />
000034e0 9764 60<br />
00003504 9762 80<br />
00003512 9761 76<br />
0000351b 9765 65<br />
00003525 9765 59<br />
00003532 9765 52<br />
00003540 9764 43<br />
00003553 9765 80<br />
00003558 9765 54<br />
00003580 9762 49<br />
00003586 9765 41<br />
00003587 9763 50<br />
0000359c 9765 40<br />
000035a0 9765 46<br />
000035a1 9765 37<br />
000035a6 9765 96<br />
000035ab 9765 52<br />
000035ac 9765 39<br />
000035c4 9765 61<br />
000035d0 9765 89<br />
000035dc 9765 74<br />
000035dd 9765 43<br />
000035f1 9765 43<br />
000035fd 9765 37<br />
00003604 9765 47<br />
00003609 9764 40<br />
0000360c 9765 59<br />
0000360d 9765 37<br />
00003618 9763 42<br />
00003621 9765 42<br />
00003626 9765 42<br />
00003627 9765 41<br />
0000362a 9765 38<br />
0000362b 9765 73<br />
00003630 9765 68<br />
00003631 9764 42<br />
0000363e 9763 39<br />
00003643 9765 43<br />
00003646 9765 38<br />
0000364c 9762 40<br />
00003659 9765 47<br />
0000365c 9764 44<br />
0000365f 9765 45<br />
00003664 9765 69<br />
00003668 9764 37<br />
00003671 9765 43<br />
00003676 9765 39<br />
00003688 9765 37<br />
00003691 9765 48<br />
00003692 9764 37<br />
00003695 9762 38<br />
000036a3 9764 50<br />
000036aa 9765 47<br />
000036ab 9765 49<br />
000036c2 9765 64<br />
000036c3 9764 57<br />
000036c5 9765 40<br />
000036da 9764 40<br />
000036e1 9765 43<br />
000036ed 9762 39<br />
000036fc 9765 41<br />
000036fd 9765 59<br />
00003701 9764 85<br />
00003708 9765 43<br />
0000370a 9763 49<br />
00003728 9762 41<br />
0000372a 9763 55<br />
0000372d 9765 51<br />
00003732 9763 48<br />
00003734 9765 60<br />
00003736 9765 42<br />
00003740 9765 48<br />
00003741 9765 39<br />
00003742 9765 38<br />
00003749 9764 40<br />
00003756 9763 39<br />
00003758 9762 46<br />
0000375c 9765 67<br />
0000375d 9765 37<br />
00003761 9765 63<br />
00003779 9765 55<br />
00003785 9765 43<br />
00003798 9763 60<br />
0000379b 9765 41<br />
0000379e 9765 37<br />
000037a5 9764 39<br />
000037a8 9764 48<br />
000037ab 9765 38<br />
000037ae 9765 54<br />
000037b3 9764 42<br />
000037b6 9761 36<br />
000037be 9763 46<br />
000037c1 9765 69<br />
000037c3 9765 36<br />
000037c4 9764 37<br />
000037ce 9762 50<br />
000037d1 9761 45<br />
000037d3 9765 45<br />
000037dd 9765 45<br />
000037df 9765 50<br />
000037e0 9763 39<br />
000037e2 9765 13<br />
000037e3 9763 18<br />
000037e7 9765 49<br />
000037e8 9763 41<br />
000037e9 9765 42<br />
000037ea 9763 40<br />
000037eb 9765 39<br />
000037ec 9761 39<br />
000037ed 9763 41<br />
000037ef 9763 36<br />
000037f0 9764 36<br />
000037f1 9764 39<br />
000037f3 9765 41<br />
000037f4 9761 53<br />
000037f5 9765 36<br />
000037f6 9765 54<br />
000037f7 9763 53<br />
000037f8 9765 63<br />
000037f9 9765 60<br />
000037fa 9765 70<br />
000037fb 9765 76<br />
000037fc 9765 72<br />
000037fd 9765 73<br />
000037fe 9765 83<br />
000037ff 9765 87<br />
00003800 9765 89<br />
00003801 9762 86<br />
00003802 9763 88<br />
00003803 9764 90<br />
00003804 8110 76<br />
TOTALS 3757891 49<br />
<br />
* It will tell you for each of the database files (the log segments) how big it is and what the utilization is. The Berkeley DB will try to keep the total utilization above 50%, but for a small pool like this one it cannot do it. I figure that you may find this interesting to know after reading about utilization above.<br />
<br />
* The log segment size and the utilization goal can be adjusted too, but I cannot give any sound advice on whether that’s a good idea and how to determine good values.<br />
<br />
* All of the above is only relevant if you use the Berkeley DB backend for pools.<br />
<br />
== Special thanks ==<br />
* This documentation has been deployed thanks to Gerd Behrmann, you can find more in at: https://gist.github.com/gbehrmann/df1abbe45c03356eff2e<br />
* Read more about this at: https://backstage.forgerock.com/knowledge/kb/book/b74223337/a49979000</div>Mcaubethttps://pwiki.pic.es/index.php?title=DCACHE_CHECK_POOL_METADATA&diff=227DCACHE CHECK POOL METADATA2017-10-06T08:31:09Z<p>Mcaubet: </p>
<hr />
<div>= For the MoD =<br />
* Open a ticket to incidents@support.pic.es with the alarm. <br />
* This alarm only affects the duration of the pool start up, nothing has to be done until the next working day where this can be fixed by dCache Administrators.<br />
* Same information as [[DCACHE CHECK POOL BTREE MEMORY]] as both alarms are strictly related.<br />
<br />
= For the Admin =<br />
<br />
* Berkely DB should be tunned according to the number of meta files that a pool contains. If this not correctly adapted, pool start up will may need several hours to be fully available. Gerd explained how to tune up a pool according to this number. Below I show a modified explanation of what he said, adapting the explanation to the ATLAS pools.<br />
<br />
* The default je.log.fileCacheSize is 100. This could be increased by creating a file je.properties and adding the 'je.log.fileCacheSize=100'.<br />
<br />
* Also, other relevant settings are:<br />
<br />
je.maxMemory=<br />
je.maxMemoryPercent=20<br />
<br />
* These are equivalent. The former sets the number of bytes used for the btree cache, while the latter defines it as a percentage of the max heap size. E.g. if dcache.java.memory.heap is set to 2048m, then 410 MB is used for the btree cache.<br />
<br />
* The question is then how to determine a good size for the cache. There are some hidden utilities you can use to do this.<br />
<br />
* Below I show an example for dc042_1:<br />
<br />
[root@dc042 meta]# java -cp /usr/share/dcache/classes/je-*.jar com.sleepycat.je.util.DbPrintLog -h . -S<br />
<DbPrintLog><br />
Log statistics:<br />
type total provisional total min max avg entries<br />
count count bytes bytes bytes bytes as % of log<br />
MapLN 1,324 0 12,617,573 76 11,985 9,529 0.3<br />
NameLN 4 0 141 31 38 35 0<br />
FileSummaryLN 231,140 0 55,644,337 24 49,701 240 1.4<br />
IN 84,083 1,080 296,942,059 45 5,709 3,531 7.7<br />
BIN 114,865 114,865 377,569,148 44 6,675 3,287 9.8<br />
DbTree 318 0 1,034,997 2,680 4,136 3,254 0<br />
Commit 3,376,012 0 114,784,408 34 34 34 3<br />
CkptStart 158 0 4,899 30 33 31 0<br />
CkptEnd 226 0 22,560 67 186 99 0<br />
Trace 2,506 0 525,597 48 402 209 0<br />
FileHeader 385 0 14,630 38 38 38 0<br />
DEL_LN_TX 748,055 0 51,409,527 57 69 68 1.3<br />
INS_LN_TX 619,747 0 192,298,301 96 1,443 310 5<br />
UPD_LN_TX 2,008,226 0 398,854,718 32 1,919 198 10.4<br />
UPD_LN 6,636,173 0 1,997,042,458 18 1,819 300 51.9<br />
NewBINDelta 720,540 720,540 348,480,254 55 1,782 483 9.1<br />
key bytes 10,012,201 387,254,450 1 61 38 (10.1)<br />
data bytes 9,264,146 2,036,632,128 1 1,848 219 (52.9)<br />
<br />
Total bytes in portion of log read: 3,847,245,607<br />
Total number of entries: 14,543,762<br />
<br />
Per checkpoint interval info:<br />
lnTxn ln mapLNTxn mapLN end to end end to start start to end maxLNReplay ckptEnd<br />
273 63,082 0 10 30,455,407 3,857,456,063 0 63,355 0x1f2f/0x6f2ef<br />
5,168 130,892 0 24 39,782,369 3,827,000,656 0 136,060 0x226b/0x3a0d0<br />
27,098 132,905 0 15 50,120,541 16,509,395 33,611,146 160,003 0x252c/0x577ad<br />
128 32,773 0 3 9,904,160 3,737,097,746 0 134,154 0x254c/0x4014d<br />
82 31,341 0 12 10,576,583 3,727,193,586 0 31,423 0x2567/0xccd94<br />
0 72 0 4 711,899 51,780 660,119 72 0x2567/0x17aa6f<br />
36,445 61,509 0 0 28,499,838 3,715,905,104 0 97,954 0x25e5/0xc66d<br />
2 108 0 6 1,564,919 56,871 1,508,048 110 0x25e5/0x18a764<br />
10,732 50,299 0 5 18,877,564 3,685,840,347 0 61,055 0x2678/0x786e0<br />
33 190 0 6 1,627,301 94,279 1,533,022 223 0x2678/0x205b85<br />
8,277 84,136 0 3 28,217,133 17,808,699 10,408,434 92,434 0x26be/0x52732<br />
5,401 25,394 0 7 9,824,151 3,637,118,349 0 61,029 0x26cf/0x27849<br />
49,026 92,572 0 3 41,592,807 3,627,294,198 0 141,598 0x2747/0x1ac630<br />
2 46 0 6 357,170 45,107 312,063 48 0x2747/0x203962<br />
20,496 39,154 0 3 18,591,447 3,585,344,221 0 59,654 0x27db/0xabb39<br />
0 154 0 6 1,187,463 83,676 1,103,787 154 0x27db/0x1cd9c0<br />
17,107 49,468 0 3 18,905,722 3,565,565,311 0 66,583 0x285c/0xc273a<br />
58 0 0 6 392,501 62,920 329,581 58 0x285c/0x12246f<br />
113,890 201,969 0 17 79,488,418 3,546,267,088 0 315,874 0x28ed/0xa5611<br />
22,147 98,579 0 8 40,657,515 36,319,235 4,338,280 120,726 0x2952/0x145e7c<br />
0 122 0 6 1,296,043 70,361 1,225,682 28,367 0x2952/0x282527<br />
19,570 65,944 0 5 27,570,660 25,251,492 2,319,168 85,520 0x2962/0x3138b<br />
37,714 28,889 0 3 21,505,514 19,337,884 2,167,630 66,613 0x297d/0x1a0c75<br />
3 332 0 6 2,437,155 124,630 2,312,525 479 0x297d/0x3f3c98<br />
41,458 62,406 0 3 26,385,897 3,373,311,783 0 103,904 0x29c7/0x81701<br />
0 46 0 6 1,404,642 46,045 1,358,597 46 0x29c7/0x1d85e3<br />
41,443 87,747 0 3 39,909,271 14,624,816 25,284,455 129,198 0x29fb/0x1c237a<br />
0 123 0 5 1,185,318 68,652 1,116,666 79,470 0x29fb/0x2e39a0<br />
9,199 12,698 0 4 9,927,451 3,304,426,655 0 21,899 0x2a00/0x2d1e3b<br />
8 0 0 5 118,374 35,364 83,010 8 0x2a00/0x2eeca1<br />
122,697 96,787 0 3 66,966,017 3,294,380,830 0 219,488 0x2a44/0xa122<br />
53,436 12,002 0 3 21,830,901 3,227,414,813 0 65,438 0x2a53/0x1c9117<br />
20 167 0 6 1,308,679 85,964 1,222,715 187 0x2a53/0x30891e<br />
18,552 38,805 0 6 17,501,977 3,204,275,233 0 57,381 0x2a58/0xa6b37<br />
9 105 0 4 678,297 87,871 590,426 114 0x2a58/0x14c4d0<br />
25,434 36,231 0 10 19,064,711 3,186,094,959 0 61,675 0x2a5e/0x67f57<br />
15 159 0 6 1,467,746 83,228 1,384,518 174 0x2a5e/0x1ce4b9<br />
12,236 14,099 0 3 8,375,785 3,165,562,502 0 26,358 0x2a63/0x41c22<br />
6 184 0 6 1,058,988 89,948 969,040 190 0x2a63/0x1444ce<br />
71,844 24,018 0 3 29,188,061 27,937,328 1,250,733 95,870 0x2aac/0x7e12b<br />
9,407 25,608 0 7 11,016,096 3,126,939,668 0 35,029 0x2aba/0x17624b<br />
20 110 0 6 748,510 82,108 666,402 130 0x2aba/0x22ce29<br />
64,997 49,072 0 15 27,822,548 27,438,951 383,597 114,142 0x2af3/0x1947d<br />
10,534 23,197 0 6 12,103,837 3,087,352,514 0 33,737 0x2b01/0x21ae9a<br />
122 0 0 32 879,716 489,740 389,976 122 0x2b01/0x2f1afe<br />
60,029 128,431 0 24 58,556,679 56,338,979 2,217,700 188,485 0x2b3c/0x191505<br />
15 52 0 6 530,374 53,795 476,579 231 0x2b3c/0x212ccb<br />
48,799 11,045 0 4 18,178,427 17,837,860 340,567 59,855 0x2b41/0x56146<br />
45,998 29,564 0 3 20,154,236 2,997,103,481 0 75,573 0x2b52/0x7bbc2<br />
39,570 60,757 0 15 30,273,541 2,976,949,245 0 100,327 0x2b8e/0xbe847<br />
31,631 33,780 0 13 22,348,448 16,815,306 5,533,142 65,411 0x2bad/0x2fbde7<br />
16,831 40,178 0 5 20,571,223 2,924,327,256 0 57,047 0x2bc6/0x38753e<br />
4,292 18,826 0 8 8,363,076 2,903,756,033 0 23,118 0x2be9/0x1f7b02<br />
0 254 0 6 2,043,974 119,900 1,924,074 254 0x2be9/0x3eab48<br />
30 19,902 0 4 7,727,051 2,893,348,983 0 19,954 0x2c00/0x1bfc93<br />
27,984 75,433 0 32 29,896,927 2,885,621,932 0 103,417 0x2c13/0x1a69f2<br />
40,477 36,001 0 10 19,737,838 17,591,303 2,146,535 76,478 0x2c2a/0x1669e0<br />
2 2 0 4 194,668 36,808 157,860 10 0x2c2a/0x19624c<br />
29,246 68,367 0 3 31,325,183 2,835,792,499 0 97,615 0x2c4c/0x2d9acb<br />
0 26 0 5 1,118,302 27,604 1,090,698 26 0x2c4c/0x3eab29<br />
4 19,763 0 3 7,562,665 2,803,349,014 0 19,775 0x2c53/0x197a52<br />
26 194 0 6 1,436,963 103,803 1,333,160 220 0x2c53/0x2f6775<br />
6,255 14,988 0 8 9,972,102 2,794,349,386 0 21,292 0x2c5f/0x2efa7b<br />
39,212 18,305 0 5 18,591,243 2,784,377,284 0 57,517 0x2c8f/0x197b86<br />
0 41 0 6 1,495,215 121,720 1,373,495 41 0x2c8f/0x304c35<br />
347 23,340 0 3 7,036,643 2,764,290,826 0 23,697 0x2c95/0x31498<br />
23,408 9,116 0 3 10,098,528 2,757,254,183 0 32,524 0x2ca4/0x49578<br />
45,631 27,183 0 3 19,972,142 2,747,155,655 0 72,814 0x2cad/0x428a6<br />
0 0 0 1 963,802 69 963,733 0 0x2cad/0x12dd80<br />
276 19,066 0 4 10,705,056 5,926,693 4,778,363 19,342 0x2cc4/0x1d9fa0<br />
0 25,292 0 2 10,568,825 7,122,257 3,446,568 25,292 0x2ccf/0x264d99<br />
722 25,106 0 3 9,376,076 2,704,945,830 0 26,568 0x2cd3/0x1cc865<br />
15 96 0 6 601,987 65,907 536,080 111 0x2cd3/0x25f7e8<br />
50,662 51,942 0 4 29,283,798 2,694,967,767 0 102,611 0x2d20/0x1b0a3e<br />
18,486 16,140 0 4 12,291,893 2,665,683,969 0 34,626 0x2d3a/0x3e02f3<br />
67,481 104,443 0 5 48,233,597 15,261,303 32,972,294 171,924 0x2d79/0x230ef0<br />
19,276 34,062 0 3 18,103,592 2,605,158,479 0 171,476 0x2deb/0x61f18<br />
0 225 0 6 607,824 37,538 570,286 225 0x2deb/0xf6568<br />
14,038 14,130 0 3 13,699,081 2,586,447,063 0 28,385 0x2e21/0x47d6f1<br />
4,147 13,212 0 3 5,363,836 2,572,747,982 0 17,359 0x2e40/0x118ed<br />
0 0 0 4 133,826 37,921 95,905 0 0x2e40/0x323af<br />
31,744 33,396 0 3 22,247,625 2,567,250,320 0 65,140 0x2e45/0x256f78<br />
9 0 0 6 290,601 55,855 234,746 9 0x2e45/0x29dea1<br />
16 25,020 0 4 10,475,420 2,544,712,094 0 25,039 0x2e5b/0x311fbd<br />
1,682 19,008 0 3 8,507,134 2,534,236,674 0 20,690 0x2e5f/0x1a583b<br />
12 0 0 6 354,107 41,047 313,060 12 0x2e5f/0x1fbf76<br />
6,849 54,863 0 3 20,176,980 2,525,375,433 0 61,719 0x2e7d/0x2272ca<br />
30,012 67,588 0 5 28,465,075 27,606,214 858,861 97,600 0x2eb6/0xb06fd<br />
14,048 16,912 0 2 9,587,676 2,476,733,378 0 66,881 0x2ee1/0x4bc59<br />
15,183 12,704 0 6 11,270,153 6,834,792 4,435,361 27,887 0x2ef4/0x181de2<br />
23,012 39,873 0 7 18,610,316 2,455,875,549 0 62,948 0x2f1a/0x2e96e<br />
25,472 49,151 0 6 22,681,935 2,437,265,233 0 74,623 0x2f5a/0x2bd5bd<br />
1,031 60,319 0 6 20,425,359 2,414,583,298 0 61,350 0x2f66/0x32534c<br />
48,185 83,225 0 4 42,656,335 28,813,330 13,843,005 131,410 0x2f9d/0x5adb9b<br />
16,859 32,139 0 3 18,804,637 12,310,056 6,494,581 74,600 0x2fa3/0x489e38<br />
29,050 79,391 0 8 47,784,742 37,879,192 9,905,550 108,511 0x300a/0x26d0de<br />
32 38,306 0 6 19,991,849 14,244,764 5,747,085 117,588 0x3026/0x26b107<br />
0 37 0 6 345,851 39,476 306,375 25,114 0x3026/0x2bf802<br />
17,725 34,199 0 3 19,155,357 2,264,574,525 0 51,955 0x303f/0x1f149f<br />
1,379 23,909 0 6 10,787,679 2,245,419,168 0 25,288 0x304f/0x2b197e<br />
3 12 0 5 297,677 34,135 263,542 15 0x304f/0x2fa44b<br />
17,968 47,908 0 2 17,014,500 2,234,333,812 0 65,891 0x306c/0x2162f<br />
45,151 19,691 0 3 20,592,703 18,998,392 1,594,311 64,842 0x3071/0xb216e<br />
18,246 13,219 0 3 14,900,440 2,196,726,609 0 31,514 0x308d/0x55e7c6<br />
26 12,008 0 5 7,554,365 4,234,132 3,320,233 12,034 0x30a1/0x309683<br />
35,556 69,762 0 20 31,161,221 2,174,271,804 0 105,338 0x30c2/0x424e88<br />
33,203 114,692 0 5 49,168,609 43,804,782 5,363,827 147,895 0x30fd/0x359ee9<br />
182 19,563 0 7 10,363,229 2,093,941,974 0 87,398 0x310c/0x3b29c6<br />
14 19,817 0 4 9,355,037 4,743,642 4,611,395 19,831 0x3110/0x315263<br />
46 37,708 0 8 20,678,814 10,884,622 9,794,192 41,674 0x315c/0x3bae01<br />
88 279 0 5 761,100 44,429 716,671 3,390 0x315c/0x474b0d<br />
40,709 61,572 0 4 26,837,961 2,052,783,794 0 102,607 0x3189/0x170b56<br />
42 0 0 6 814,957 77,642 737,315 42 0x3189/0x237ac3<br />
69,126 107,504 0 5 52,094,116 47,372,898 4,721,218 176,645 0x31b8/0x436ee7<br />
46 0 0 5 393,197 51,476 341,721 82,129 0x31b8/0x496ed4<br />
48,348 117,823 0 3 48,879,763 41,965,088 6,914,675 166,196 0x3201/0x3856e7<br />
16,349 39,063 0 7 17,875,388 14,606,658 3,268,730 62,982 0x3208/0x17eba3<br />
105 0 0 6 3,394,912 60,919 3,333,993 2,571 0x3208/0x4bb903<br />
217 17,310 0 7 5,776,110 1,902,493,500 0 17,617 0x3210/0xb4571<br />
36,657 27,268 0 7 22,160,771 4,617,088 17,543,683 63,925 0x3224/0x2c3df4<br />
24,153 42,022 0 1 22,019,603 1,874,556,619 0 114,113 0x3235/0x4b0f07<br />
16 18,174 0 7 7,706,321 1,852,537,016 0 18,190 0x324e/0x280f58<br />
120 24,379 0 7 10,458,169 1,844,830,695 0 24,499 0x3266/0x2f0d11<br />
14,236 34,916 0 3 17,998,326 12,723,415 5,274,911 49,152 0x327b/0x108207<br />
79 0 0 5 673,666 37,699 635,967 2,808 0x327b/0x1ac989<br />
6,624 22,958 0 3 11,999,224 1,815,700,534 0 29,650 0x3290/0x394b01<br />
19,965 40,810 0 8 17,269,398 1,803,701,310 0 60,775 0x32a7/0xfa097<br />
7,517 58,175 0 5 21,772,528 1,786,431,912 0 65,692 0x32de/0x2aac87<br />
9,415 18,600 0 6 10,786,946 1,764,659,384 0 28,015 0x32e1/0x36ae89<br />
51 0 0 6 622,591 52,177 570,414 51 0x32e1/0x402e88<br />
30,139 66,899 0 4 30,497,693 1,753,249,847 0 97,065 0x3315/0x47c6a5<br />
6 41,284 0 1 16,342,999 12,394,744 3,948,255 41,290 0x3320/0xff97c<br />
21,091 17,059 0 6 11,525,646 1,706,409,155 0 38,984 0x332b/0x27410a<br />
0 94 0 4 892,946 34,897 858,049 94 0x332b/0x34e11c<br />
3 52,035 0 3 18,160,444 1,693,990,563 0 52,132 0x336d/0x18cf58<br />
58 27,035 0 6 8,947,956 1,675,830,119 0 27,093 0x3376/0x8c1cc<br />
3 66 0 4 501,044 49,007 452,037 69 0x3376/0x106700<br />
27,134 44,130 0 4 22,994,532 1,666,381,119 0 71,330 0x339f/0x3e1864<br />
0 108 0 4 439,894 35,044 404,850 108 0x339f/0x44ceba<br />
17 51,506 0 3 20,284,195 1,642,946,693 0 51,631 0x33a6/0x4924dd<br />
28 16,826 0 2 9,954,053 1,622,662,498 0 16,854 0x33b6/0x487162<br />
2,079 43,231 0 2 20,850,856 13,682,453 7,168,403 45,310 0x33d2/0x556d0a<br />
2 30,027 0 3 14,418,091 8,924,719 5,493,372 30,743 0x33d9/0x40b5<br />
0 1,157 0 6 5,510,215 155,394 5,354,821 3,103 0x33d9/0x5454fc<br />
31,517 31,808 0 3 18,110,480 1,571,929,283 0 64,085 0x33ee/0x37800c<br />
30,598 27,193 0 4 18,121,300 12,201,650 5,919,650 57,791 0x3402/0x1ad560<br />
38,905 34,440 0 4 18,443,706 15,100,893 3,342,813 78,249 0x340c/0x3161a<br />
51,887 95,209 0 4 40,262,214 29,791,799 10,470,415 152,283 0x343b/0x71660<br />
1 28,686 0 7 10,081,465 1,476,991,583 0 142,625 0x344c/0x85499<br />
28,401 13,707 0 3 10,169,877 1,466,910,118 0 42,108 0x3457/0xaec2e<br />
5 16,531 0 6 14,562,193 4,402,715 10,159,478 16,536 0x345b/0x50893f<br />
0 15,920 0 7 9,514,322 1,442,178,048 0 19,604 0x3460/0x492011<br />
4 16,264 0 6 10,131,895 1,432,663,726 0 16,268 0x3465/0x4b2348<br />
30,231 28,986 0 4 18,938,103 14,093,138 4,844,965 59,217 0x347c/0x3aef3f<br />
14 0 0 5 512,006 34,432 477,574 120 0x347c/0x42bf45<br />
20,678 38,294 0 3 17,101,301 1,403,081,722 0 58,975 0x3497/0x16843a<br />
9,869 25,099 0 5 12,548,707 1,385,980,421 0 34,968 0x34b3/0x3d681d<br />
36,465 60,605 0 5 26,361,794 24,558,695 1,803,099 97,070 0x34d6/0x5e45f<br />
36 23,345 0 3 9,621,974 6,699,792 2,922,182 23,911 0x34d9/0x1fb5<br />
25,436 49,579 0 7 20,880,498 18,608,524 2,271,974 75,850 0x34df/0xd8f27<br />
32 1,485 0 6 4,197,555 413,166 3,784,389 1,535 0x34df/0x4d9bda<br />
31,535 59,968 0 5 25,248,912 24,728,797 520,115 91,849 0x3512/0x51cea<br />
1,601 24,378 0 6 10,268,411 8,200,696 2,067,715 26,160 0x351b/0x93565<br />
14,147 23,162 0 8 11,300,331 1,276,852,570 0 37,400 0x3525/0x1d0cd0<br />
14,204 20,662 0 3 13,299,117 1,265,552,239 0 34,866 0x3532/0x4f63fd<br />
17,514 24,728 0 3 16,797,576 12,513,187 4,284,389 42,242 0x3553/0x1e8685<br />
7,303 46,343 0 7 23,094,370 14,762,461 8,331,909 54,344 0x3580/0x4dbde7<br />
45,620 41,812 0 7 27,514,220 21,511,990 6,002,230 90,879 0x359c/0x27cfd3<br />
16,861 12,552 0 3 9,619,073 1,184,846,956 0 104,424 0x35a0/0x21ffd4<br />
0 271 0 6 3,731,928 93,043 3,638,885 271 0x35a0/0x5af1ac<br />
30,860 48,187 0 7 26,338,978 24,109,627 2,229,351 79,097 0x35ab/0x2314ce<br />
39,031 24,255 0 4 20,202,284 1,145,156,977 0 63,301 0x35c4/0x262afa<br />
2,576 46,002 0 6 19,184,048 5,580,647 13,603,401 48,578 0x35dc/0x19b7aa<br />
2 0 0 6 174,028 33,089 140,939 31,536 0x35dc/0x1c5f76<br />
111,528 77,161 0 7 62,341,581 50,549,489 11,792,092 188,690 0x360c/0x401a43<br />
28,697 25,910 0 6 18,246,350 13,855,848 4,390,502 56,636 0x3618/0x255811<br />
4,335 12,127 0 6 9,004,926 5,130,460 3,874,466 18,985 0x3621/0x16290f<br />
18,413 8,607 0 5 8,878,516 7,701,363 1,177,153 27,027 0x3626/0x50c43<br />
0 2,910 0 6 6,228,723 848,601 5,380,122 2,910 0x3626/0x641736<br />
28,784 22,689 0 3 21,119,867 14,626,185 6,493,682 52,178 0x362a/0x752db1<br />
41,676 62,528 0 4 34,112,988 29,303,765 4,809,223 105,238 0x363e/0x1b598d<br />
17 735 0 5 12,521,216 31,105 12,490,111 1,300 0x3643/0x41d20d<br />
7,802 34,494 0 7 33,127,169 7,514,191 25,612,978 43,003 0x3659/0x71898e<br />
0 8,734 0 5 3,377,408 900,015,281 0 25,130 0x365c/0xc7c0e<br />
114 9,858 0 7 15,782,161 650,578 15,131,583 9,972 0x365f/0x64b69f<br />
1 11,516 0 7 5,378,418 880,855,712 0 19,548 0x3664/0x1e3191<br />
7 22,005 0 7 8,922,863 6,691,187 2,231,676 22,012 0x3668/0xdc200<br />
13 13,409 0 6 15,952,421 360,887 15,591,534 14,391 0x3671/0x6895a5<br />
1,749 8,568 0 3 3,660,457 850,602,010 0 22,698 0x3676/0x7d9ce<br />
5,390 10,146 0 6 14,882,555 1,855,609 13,026,946 15,536 0x3688/0x525a49<br />
10,230 9,513 0 5 4,816,061 832,058,998 0 28,465 0x3691/0x34086<br />
14,698 31,907 0 7 41,080,361 36,925,183 4,155,178 46,605 0x36aa/0x13bcaf<br />
2 1,755 0 6 4,666,159 345,619 4,320,540 47,630 0x36aa/0x5aefde<br />
25,935 30,385 0 5 38,398,767 781,496,417 0 57,119 0x36c5/0x42810d<br />
9,478 14,696 0 7 21,102,013 743,097,650 0 24,174 0x36e1/0x5351ca<br />
65 12,746 0 6 5,610,587 3,858,013 1,752,574 12,811 0x36ed/0x1057a5<br />
5,806 9,341 0 6 10,353,968 2,690,013 7,663,955 15,343 0x36fc/0x15be55<br />
5 0 0 6 4,894,393 33,744 4,860,649 5,931 0x36fc/0x606d0e<br />
28,754 58,230 0 18 28,401,618 701,136,689 0 86,988 0x3708/0x480960<br />
0 15,897 0 6 10,890,931 3,098,266 7,792,665 15,897 0x370a/0x55a193<br />
126 10,927 0 6 26,068,972 2,214,107 23,854,865 16,100 0x372d/0x19a5ff<br />
9,837 16,155 0 6 11,096,409 6,346,816 4,749,593 29,696 0x3732/0x2a60d8<br />
128 15,250 0 6 8,591,606 3,747,801 4,843,805 16,046 0x3734/0x14e34e<br />
123 28,336 0 6 16,047,724 5,161,792 10,885,932 31,737 0x3736/0x712b3a<br />
24 8,534 0 5 3,423,605 1,552,888 1,870,717 19,982 0x3740/0xcd22f<br />
533 0 0 6 20,325,930 275,776 20,050,154 3,884 0x3742/0x11cb59<br />
0 0 0 4 72,242 36,230 36,012 0 0x3742/0x12e58b<br />
26,449 16,768 0 3 20,777,536 576,217,652 0 43,217 0x3756/0x1ec2cb<br />
11,493 23,349 0 3 21,836,979 13,435,005 8,401,974 34,842 0x375c/0x3aca7e<br />
309 32,239 0 6 17,558,766 9,588,731 7,970,035 56,790 0x3761/0x158a6c<br />
12 26,562 0 7 12,407,639 6,425,279 5,982,360 27,257 0x3779/0x3a4743<br />
6,845 41,908 0 7 33,072,537 22,775,381 10,297,156 51,339 0x379b/0x69295c<br />
6,778 29,990 0 6 25,664,453 17,110,934 8,553,519 41,294 0x37a8/0x2701a1<br />
5 16,509 0 9 19,648,397 8,979,547 10,668,850 44,387 0x37ae/0x21a42e<br />
116,569 137,423 0 12 129,250,694 72,386,145 56,864,549 267,033 0x37e2/0x163534<br />
13 17,546 0 6 28,042,727 34,685 28,008,042 187,464 0x37e7/0x90ee1b<br />
33,326 30,872 0 6 43,125,925 9,805,827 33,320,098 81,756 0x37ec/0x280a40<br />
3 5,332 0 5 19,873,538 37,653 19,835,885 30,103 0x37ef/0x261c42<br />
17 2,306 0 6 19,166,423 244,987 18,921,436 7,657 0x37f1/0x196419<br />
10,753 20,277 0 6 40,648,779 11,079,498 29,569,281 32,618 0x37f6/0x234a64<br />
10 22,876 0 6 18,391,375 164,634 18,226,741 49,034 0x37f8/0xabeb3<br />
30 34,377 0 7 32,006,385 1,775,081 30,231,304 56,545 0x37fb/0x295c24<br />
4 23,090 0 7 38,624,657 2,288,459 36,336,198 52,101 0x37ff/0x145fb5<br />
6 8,689 0 6 27,108,848 949,959 26,158,889 24,449 0x3801/0x80d8a5<br />
6 5,117 0 7 18,644,436 55,620 18,588,816 11,221 0x3803/0x6c2979<br />
837 0 0 6 6,661,670 288,422 6,373,248 5,956 0x3804/0x39391f<br />
5,863 6,917 0 0 3,705,888 3,705,888 0 12,780 0x3804/0x71c53f<br />
</DbPrintLog><br />
<br />
* This prints some statistics about the database. In particular the two rows called key bytes and data bytes are relevant. For the following step you need the values from the avg column - i.e. 38 and 219 in this case. You also need the value of “Total number of entries” (14543762 in this case). Now we take those values and put them into this command:<br />
<br />
[root@dc042 meta]# java -cp /usr/share/dcache/classes/je-*.jar com.sleepycat.je.util.DbCacheSize -records 14543762 -key 38 -data 219<br />
<br />
=== Environment Cache Overhead ===<br />
<br />
3,157,213 minimum bytes<br />
<br />
To account for JE daemon operation and record locks,<br />
a significantly larger amount is needed in practice.<br />
<br />
=== Database Cache Size ===<br />
<br />
Number of Bytes Description<br />
--------------- -----------<br />
1,195,759,832 Internal nodes only<br />
5,124,232,392 Internal nodes and leaf nodes<br />
<br />
* For further information see the DbCacheSize javadoc.<br />
<br />
* For this pool it tells us that we need a bit above 1195 MB to keep all the internal btree nodes cached. Oracle’s recommendation is that if the database is updated often, the cache is at least big enough to contain the internal nodes. Obviously you want to make it a bit bigger to leave room for it to grow. One could take this and configure the cache using the je.maxMemory setting. If the size is lower than the 20% of the max heap size you already use, then you don’t need to do anything (I do not suggest lowering it further).<br />
<br />
* Since this pool has 6000m of JAVA heap max space, it means that 20% = 1200MB, which is exactly cache for Internal btree nodes. So we will not touch this.<br />
<br />
* There is one caveat though: If you increase the cache size, less free space is left on the heap. If the pool is pushed to the limit, this may actually slow down the pool as garbage collection overhead increases. You need to ensure that enough space is left in addition to the cache (possibly by increasing the max heap size). An alternative to adjusting je.maxMemory is of course to adjust the max heap size. If you make it large enough so that 20% is enough to cache the internal btree nodes, then all is well. This may however mean you assign significantly more memory to the pool than it really needs. Yeah, there are lots of things to consider.<br />
<br />
* A little bonus: There is also this command:<br />
<br />
[root@dc042 meta]# java -cp /usr/share/dcache/classes/je-*.jar com.sleepycat.je.util.DbSpace -h .<br />
File Size (KB) % Used<br />
-------- --------- ------<br />
00001857 9765 41<br />
00001f2a 9765 43<br />
00001f2f 9765 39<br />
00001f72 9765 44<br />
0000216d 9765 40<br />
000021cb 9765 40<br />
0000226b 9765 45<br />
000022e8 9763 38<br />
000022fa 9765 39<br />
00002390 9765 38<br />
0000241d 9765 49<br />
0000252c 9765 51<br />
0000254c 9765 47<br />
00002567 9765 39<br />
000025b7 9765 42<br />
000025db 9765 42<br />
000025e5 9765 56<br />
0000265f 9765 39<br />
00002678 9765 61<br />
0000268d 9762 48<br />
00002692 9765 74<br />
000026be 9765 48<br />
000026cf 9765 40<br />
00002700 9765 41<br />
00002706 9765 38<br />
00002718 9765 56<br />
00002747 9765 44<br />
0000274d 9765 48<br />
000027db 9765 62<br />
00002855 9765 39<br />
0000285c 9765 39<br />
00002882 9765 41<br />
0000289d 9765 54<br />
000028ba 9765 52<br />
000028c1 9765 53<br />
000028d0 9765 41<br />
000028d7 9765 41<br />
000028e7 9765 39<br />
000028ed 9765 39<br />
00002926 9765 39<br />
0000292b 9765 43<br />
00002945 9765 41<br />
00002952 9765 41<br />
00002958 9765 62<br />
00002959 9764 38<br />
00002962 9765 53<br />
00002963 9765 47<br />
0000297d 9765 51<br />
0000297e 9765 41<br />
000029b3 9765 45<br />
000029c7 9765 56<br />
000029ce 9765 42<br />
000029d3 9765 43<br />
000029d9 9765 54<br />
000029fb 9765 42<br />
00002a00 9765 54<br />
00002a01 9765 39<br />
00002a06 9765 39<br />
00002a20 9765 45<br />
00002a26 9765 61<br />
00002a2b 9765 69<br />
00002a30 9765 65<br />
00002a44 9765 46<br />
00002a4f 9765 50<br />
00002a53 9765 64<br />
00002a54 9765 47<br />
00002a58 9765 66<br />
00002a59 9765 43<br />
00002a5e 9765 56<br />
00002a63 9765 44<br />
00002a69 9765 47<br />
00002a8f 9765 39<br />
00002aac 9765 54<br />
00002aba 9765 42<br />
00002abb 9765 44<br />
00002ae0 9763 38<br />
00002af3 9765 45<br />
00002b01 9765 41<br />
00002b08 9765 65<br />
00002b0e 9765 41<br />
00002b24 9765 45<br />
00002b28 9765 61<br />
00002b29 9765 41<br />
00002b3c 9765 40<br />
00002b3d 9765 40<br />
00002b41 9765 47<br />
00002b4c 9765 46<br />
00002b52 9765 57<br />
00002b64 9765 38<br />
00002b87 9765 38<br />
00002b8e 9765 47<br />
00002b8f 9760 38<br />
00002bad 9765 42<br />
00002bbf 9765 44<br />
00002bc6 9765 38<br />
00002be9 9765 47<br />
00002c00 9765 42<br />
00002c06 9765 69<br />
00002c0c 9765 55<br />
00002c13 9765 38<br />
00002c14 9765 38<br />
00002c2a 9765 41<br />
00002c2b 9765 41<br />
00002c30 9765 54<br />
00002c4c 9765 39<br />
00002c53 9765 45<br />
00002c5f 9765 44<br />
00002c7c 9765 38<br />
00002c8f 9765 49<br />
00002c95 9765 38<br />
00002ca4 9764 50<br />
00002ca9 9765 42<br />
00002cad 9763 41<br />
00002cc4 9765 39<br />
00002ccf 9765 43<br />
00002cd3 9765 45<br />
00002cf3 9765 48<br />
00002d19 9765 38<br />
00002d20 9765 38<br />
00002d3a 9765 40<br />
00002d3b 9765 41<br />
00002d41 9765 43<br />
00002d51 9765 39<br />
00002d59 9765 40<br />
00002d79 9765 62<br />
00002d8f 9765 39<br />
00002deb 9765 40<br />
00002e21 9765 39<br />
00002e40 9765 41<br />
00002e41 9765 45<br />
00002e45 9765 40<br />
00002e5b 9765 58<br />
00002e5f 9765 49<br />
00002e60 9765 67<br />
00002e7d 9765 48<br />
00002e95 9765 38<br />
00002eb1 9762 57<br />
00002eb6 9765 53<br />
00002ee1 9765 38<br />
00002ef4 9763 43<br />
00002ef8 9765 41<br />
00002f1a 9765 53<br />
00002f25 9765 65<br />
00002f5a 9765 52<br />
00002f5f 9765 63<br />
00002f66 9765 41<br />
00002f71 9765 42<br />
00002f79 9765 61<br />
00002f87 9765 49<br />
00002f9d 9765 41<br />
00002f9e 9765 42<br />
00002fa3 9765 38<br />
00002fa4 9765 46<br />
00002fb1 9762 55<br />
00002fc1 9765 41<br />
00002fde 9765 38<br />
0000300a 9765 43<br />
0000300e 9762 47<br />
00003026 9765 55<br />
00003027 9765 57<br />
0000303f 9765 63<br />
0000304f 9765 48<br />
00003056 9765 44<br />
0000306c 9765 48<br />
0000306d 9765 42<br />
00003071 9765 45<br />
0000308d 9765 39<br />
000030a1 9765 54<br />
000030a5 9765 51<br />
000030ab 9765 60<br />
000030c2 9765 46<br />
000030d5 9765 43<br />
000030dd 9765 38<br />
000030e3 9765 57<br />
000030e4 9765 55<br />
000030fd 9765 48<br />
0000310c 9764 42<br />
00003110 9765 46<br />
00003119 9765 47<br />
0000315c 9765 48<br />
0000315d 9765 54<br />
00003185 9765 56<br />
00003189 9765 52<br />
0000318a 9765 58<br />
0000319a 9765 46<br />
0000319f 9764 39<br />
000031b5 9764 53<br />
000031b8 9765 40<br />
000031b9 9765 74<br />
000031e2 9765 65<br />
000031fa 9765 58<br />
000031fb 9765 52<br />
00003201 9765 50<br />
00003202 9764 44<br />
00003208 9765 42<br />
00003210 9763 40<br />
00003218 9765 40<br />
00003224 9765 51<br />
00003229 9765 48<br />
00003235 9765 40<br />
0000324e 9765 46<br />
00003266 9765 41<br />
0000326c 9764 47<br />
0000327b 9765 55<br />
00003290 9765 48<br />
00003294 9765 56<br />
000032a7 9765 69<br />
000032ad 9765 62<br />
000032de 9765 47<br />
000032e1 9765 46<br />
000032e2 9765 71<br />
000032e9 9765 45<br />
00003315 9765 42<br />
00003316 9764 54<br />
00003320 9765 43<br />
0000332b 9765 52<br />
0000332c 9765 76<br />
0000336d 9765 69<br />
00003376 9765 72<br />
00003377 9765 46<br />
0000339f 9765 43<br />
000033a0 9765 75<br />
000033a6 9765 46<br />
000033b6 9765 40<br />
000033c0 9764 72<br />
000033d2 9765 38<br />
000033d3 9764 45<br />
000033d9 9765 42<br />
000033da 9765 44<br />
000033ee 9765 41<br />
000033fb 9764 38<br />
00003402 9765 42<br />
00003403 9765 53<br />
0000340c 9765 76<br />
00003428 9765 67<br />
00003431 9765 51<br />
00003436 9765 88<br />
0000343b 9765 85<br />
0000344c 9765 48<br />
00003457 9765 46<br />
0000345b 9765 46<br />
00003460 9765 50<br />
00003465 9765 45<br />
00003466 9762 40<br />
0000347c 9765 54<br />
0000347d 9765 61<br />
00003497 9765 63<br />
000034b3 9765 64<br />
000034bd 9765 54<br />
000034c9 9764 71<br />
000034d6 9765 65<br />
000034d9 9765 86<br />
000034da 9765 46<br />
000034df 9765 51<br />
000034e0 9764 60<br />
00003504 9762 80<br />
00003512 9761 76<br />
0000351b 9765 65<br />
00003525 9765 59<br />
00003532 9765 52<br />
00003540 9764 43<br />
00003553 9765 80<br />
00003558 9765 54<br />
00003580 9762 49<br />
00003586 9765 41<br />
00003587 9763 50<br />
0000359c 9765 40<br />
000035a0 9765 46<br />
000035a1 9765 37<br />
000035a6 9765 96<br />
000035ab 9765 52<br />
000035ac 9765 39<br />
000035c4 9765 61<br />
000035d0 9765 89<br />
000035dc 9765 74<br />
000035dd 9765 43<br />
000035f1 9765 43<br />
000035fd 9765 37<br />
00003604 9765 47<br />
00003609 9764 40<br />
0000360c 9765 59<br />
0000360d 9765 37<br />
00003618 9763 42<br />
00003621 9765 42<br />
00003626 9765 42<br />
00003627 9765 41<br />
0000362a 9765 38<br />
0000362b 9765 73<br />
00003630 9765 68<br />
00003631 9764 42<br />
0000363e 9763 39<br />
00003643 9765 43<br />
00003646 9765 38<br />
0000364c 9762 40<br />
00003659 9765 47<br />
0000365c 9764 44<br />
0000365f 9765 45<br />
00003664 9765 69<br />
00003668 9764 37<br />
00003671 9765 43<br />
00003676 9765 39<br />
00003688 9765 37<br />
00003691 9765 48<br />
00003692 9764 37<br />
00003695 9762 38<br />
000036a3 9764 50<br />
000036aa 9765 47<br />
000036ab 9765 49<br />
000036c2 9765 64<br />
000036c3 9764 57<br />
000036c5 9765 40<br />
000036da 9764 40<br />
000036e1 9765 43<br />
000036ed 9762 39<br />
000036fc 9765 41<br />
000036fd 9765 59<br />
00003701 9764 85<br />
00003708 9765 43<br />
0000370a 9763 49<br />
00003728 9762 41<br />
0000372a 9763 55<br />
0000372d 9765 51<br />
00003732 9763 48<br />
00003734 9765 60<br />
00003736 9765 42<br />
00003740 9765 48<br />
00003741 9765 39<br />
00003742 9765 38<br />
00003749 9764 40<br />
00003756 9763 39<br />
00003758 9762 46<br />
0000375c 9765 67<br />
0000375d 9765 37<br />
00003761 9765 63<br />
00003779 9765 55<br />
00003785 9765 43<br />
00003798 9763 60<br />
0000379b 9765 41<br />
0000379e 9765 37<br />
000037a5 9764 39<br />
000037a8 9764 48<br />
000037ab 9765 38<br />
000037ae 9765 54<br />
000037b3 9764 42<br />
000037b6 9761 36<br />
000037be 9763 46<br />
000037c1 9765 69<br />
000037c3 9765 36<br />
000037c4 9764 37<br />
000037ce 9762 50<br />
000037d1 9761 45<br />
000037d3 9765 45<br />
000037dd 9765 45<br />
000037df 9765 50<br />
000037e0 9763 39<br />
000037e2 9765 13<br />
000037e3 9763 18<br />
000037e7 9765 49<br />
000037e8 9763 41<br />
000037e9 9765 42<br />
000037ea 9763 40<br />
000037eb 9765 39<br />
000037ec 9761 39<br />
000037ed 9763 41<br />
000037ef 9763 36<br />
000037f0 9764 36<br />
000037f1 9764 39<br />
000037f3 9765 41<br />
000037f4 9761 53<br />
000037f5 9765 36<br />
000037f6 9765 54<br />
000037f7 9763 53<br />
000037f8 9765 63<br />
000037f9 9765 60<br />
000037fa 9765 70<br />
000037fb 9765 76<br />
000037fc 9765 72<br />
000037fd 9765 73<br />
000037fe 9765 83<br />
000037ff 9765 87<br />
00003800 9765 89<br />
00003801 9762 86<br />
00003802 9763 88<br />
00003803 9764 90<br />
00003804 8110 76<br />
TOTALS 3757891 49<br />
<br />
* It will tell you for each of the database files (the log segments) how big it is and what the utilization is. The Berkeley DB will try to keep the total utilization above 50%, but for a small pool like this one it cannot do it. I figure that you may find this interesting to know after reading about utilization above.<br />
<br />
* The log segment size and the utilization goal can be adjusted too, but I cannot give any sound advice on whether that’s a good idea and how to determine good values.<br />
<br />
* All of the above is only relevant if you use the Berkeley DB backend for pools.<br />
<br />
== Special thanks ==<br />
* This documentation has been deployed thanks to Gerd Behrmann, you can find more in at: https://gist.github.com/gbehrmann/df1abbe45c03356eff2e</div>Mcaubethttps://pwiki.pic.es/index.php?title=DCACHE_CHECK_POOL_BTREE_MEMORY&diff=226DCACHE CHECK POOL BTREE MEMORY2017-10-06T08:31:01Z<p>Mcaubet: /* For the Admin */</p>
<hr />
<div>= For the MoD =<br />
* Open a ticket to incidents@support.pic.es with the alarm. <br />
* This alarm only affects the duration of the pool start up, nothing has to be done until the next working day where this can be fixed by dCache Administrators.<br />
* Same information as [[DCACHE CHECK POOL METADATA]] as both alarms are strictly related.<br />
<br />
= For the Admin =<br />
<br />
* Berkely DB should be tunned according to the number of meta files that a pool contains. If this not correctly adapted, pool start up will may need several hours to be fully available. Gerd explained how to tune up a pool according to this number. Below I show a modified explanation of what he said, adapting the explanation to the ATLAS pools.<br />
<br />
* The default je.log.fileCacheSize is 100. This could be increased by creating a file je.properties and adding the 'je.log.fileCacheSize=100'.<br />
<br />
* Also, other relevant settings are:<br />
<br />
je.maxMemory=<br />
je.maxMemoryPercent=20<br />
<br />
* These are equivalent. The former sets the number of bytes used for the btree cache, while the latter defines it as a percentage of the max heap size. E.g. if dcache.java.memory.heap is set to 2048m, then 410 MB is used for the btree cache.<br />
<br />
* The question is then how to determine a good size for the cache. There are some hidden utilities you can use to do this.<br />
<br />
* Below I show an example for dc042_1:<br />
<br />
[root@dc042 meta]# java -cp /usr/share/dcache/classes/je-6.3.8.jar com.sleepycat.je.util.DbPrintLog -h . -S<br />
<DbPrintLog><br />
Log statistics:<br />
type total provisional total min max avg entries<br />
count count bytes bytes bytes bytes as % of log<br />
MapLN 1,324 0 12,617,573 76 11,985 9,529 0.3<br />
NameLN 4 0 141 31 38 35 0<br />
FileSummaryLN 231,140 0 55,644,337 24 49,701 240 1.4<br />
IN 84,083 1,080 296,942,059 45 5,709 3,531 7.7<br />
BIN 114,865 114,865 377,569,148 44 6,675 3,287 9.8<br />
DbTree 318 0 1,034,997 2,680 4,136 3,254 0<br />
Commit 3,376,012 0 114,784,408 34 34 34 3<br />
CkptStart 158 0 4,899 30 33 31 0<br />
CkptEnd 226 0 22,560 67 186 99 0<br />
Trace 2,506 0 525,597 48 402 209 0<br />
FileHeader 385 0 14,630 38 38 38 0<br />
DEL_LN_TX 748,055 0 51,409,527 57 69 68 1.3<br />
INS_LN_TX 619,747 0 192,298,301 96 1,443 310 5<br />
UPD_LN_TX 2,008,226 0 398,854,718 32 1,919 198 10.4<br />
UPD_LN 6,636,173 0 1,997,042,458 18 1,819 300 51.9<br />
NewBINDelta 720,540 720,540 348,480,254 55 1,782 483 9.1<br />
key bytes 10,012,201 387,254,450 1 61 38 (10.1)<br />
data bytes 9,264,146 2,036,632,128 1 1,848 219 (52.9)<br />
<br />
Total bytes in portion of log read: 3,847,245,607<br />
Total number of entries: 14,543,762<br />
<br />
Per checkpoint interval info:<br />
lnTxn ln mapLNTxn mapLN end to end end to start start to end maxLNReplay ckptEnd<br />
273 63,082 0 10 30,455,407 3,857,456,063 0 63,355 0x1f2f/0x6f2ef<br />
5,168 130,892 0 24 39,782,369 3,827,000,656 0 136,060 0x226b/0x3a0d0<br />
27,098 132,905 0 15 50,120,541 16,509,395 33,611,146 160,003 0x252c/0x577ad<br />
128 32,773 0 3 9,904,160 3,737,097,746 0 134,154 0x254c/0x4014d<br />
82 31,341 0 12 10,576,583 3,727,193,586 0 31,423 0x2567/0xccd94<br />
0 72 0 4 711,899 51,780 660,119 72 0x2567/0x17aa6f<br />
36,445 61,509 0 0 28,499,838 3,715,905,104 0 97,954 0x25e5/0xc66d<br />
2 108 0 6 1,564,919 56,871 1,508,048 110 0x25e5/0x18a764<br />
10,732 50,299 0 5 18,877,564 3,685,840,347 0 61,055 0x2678/0x786e0<br />
33 190 0 6 1,627,301 94,279 1,533,022 223 0x2678/0x205b85<br />
8,277 84,136 0 3 28,217,133 17,808,699 10,408,434 92,434 0x26be/0x52732<br />
5,401 25,394 0 7 9,824,151 3,637,118,349 0 61,029 0x26cf/0x27849<br />
49,026 92,572 0 3 41,592,807 3,627,294,198 0 141,598 0x2747/0x1ac630<br />
2 46 0 6 357,170 45,107 312,063 48 0x2747/0x203962<br />
20,496 39,154 0 3 18,591,447 3,585,344,221 0 59,654 0x27db/0xabb39<br />
0 154 0 6 1,187,463 83,676 1,103,787 154 0x27db/0x1cd9c0<br />
17,107 49,468 0 3 18,905,722 3,565,565,311 0 66,583 0x285c/0xc273a<br />
58 0 0 6 392,501 62,920 329,581 58 0x285c/0x12246f<br />
113,890 201,969 0 17 79,488,418 3,546,267,088 0 315,874 0x28ed/0xa5611<br />
22,147 98,579 0 8 40,657,515 36,319,235 4,338,280 120,726 0x2952/0x145e7c<br />
0 122 0 6 1,296,043 70,361 1,225,682 28,367 0x2952/0x282527<br />
19,570 65,944 0 5 27,570,660 25,251,492 2,319,168 85,520 0x2962/0x3138b<br />
37,714 28,889 0 3 21,505,514 19,337,884 2,167,630 66,613 0x297d/0x1a0c75<br />
3 332 0 6 2,437,155 124,630 2,312,525 479 0x297d/0x3f3c98<br />
41,458 62,406 0 3 26,385,897 3,373,311,783 0 103,904 0x29c7/0x81701<br />
0 46 0 6 1,404,642 46,045 1,358,597 46 0x29c7/0x1d85e3<br />
41,443 87,747 0 3 39,909,271 14,624,816 25,284,455 129,198 0x29fb/0x1c237a<br />
0 123 0 5 1,185,318 68,652 1,116,666 79,470 0x29fb/0x2e39a0<br />
9,199 12,698 0 4 9,927,451 3,304,426,655 0 21,899 0x2a00/0x2d1e3b<br />
8 0 0 5 118,374 35,364 83,010 8 0x2a00/0x2eeca1<br />
122,697 96,787 0 3 66,966,017 3,294,380,830 0 219,488 0x2a44/0xa122<br />
53,436 12,002 0 3 21,830,901 3,227,414,813 0 65,438 0x2a53/0x1c9117<br />
20 167 0 6 1,308,679 85,964 1,222,715 187 0x2a53/0x30891e<br />
18,552 38,805 0 6 17,501,977 3,204,275,233 0 57,381 0x2a58/0xa6b37<br />
9 105 0 4 678,297 87,871 590,426 114 0x2a58/0x14c4d0<br />
25,434 36,231 0 10 19,064,711 3,186,094,959 0 61,675 0x2a5e/0x67f57<br />
15 159 0 6 1,467,746 83,228 1,384,518 174 0x2a5e/0x1ce4b9<br />
12,236 14,099 0 3 8,375,785 3,165,562,502 0 26,358 0x2a63/0x41c22<br />
6 184 0 6 1,058,988 89,948 969,040 190 0x2a63/0x1444ce<br />
71,844 24,018 0 3 29,188,061 27,937,328 1,250,733 95,870 0x2aac/0x7e12b<br />
9,407 25,608 0 7 11,016,096 3,126,939,668 0 35,029 0x2aba/0x17624b<br />
20 110 0 6 748,510 82,108 666,402 130 0x2aba/0x22ce29<br />
64,997 49,072 0 15 27,822,548 27,438,951 383,597 114,142 0x2af3/0x1947d<br />
10,534 23,197 0 6 12,103,837 3,087,352,514 0 33,737 0x2b01/0x21ae9a<br />
122 0 0 32 879,716 489,740 389,976 122 0x2b01/0x2f1afe<br />
60,029 128,431 0 24 58,556,679 56,338,979 2,217,700 188,485 0x2b3c/0x191505<br />
15 52 0 6 530,374 53,795 476,579 231 0x2b3c/0x212ccb<br />
48,799 11,045 0 4 18,178,427 17,837,860 340,567 59,855 0x2b41/0x56146<br />
45,998 29,564 0 3 20,154,236 2,997,103,481 0 75,573 0x2b52/0x7bbc2<br />
39,570 60,757 0 15 30,273,541 2,976,949,245 0 100,327 0x2b8e/0xbe847<br />
31,631 33,780 0 13 22,348,448 16,815,306 5,533,142 65,411 0x2bad/0x2fbde7<br />
16,831 40,178 0 5 20,571,223 2,924,327,256 0 57,047 0x2bc6/0x38753e<br />
4,292 18,826 0 8 8,363,076 2,903,756,033 0 23,118 0x2be9/0x1f7b02<br />
0 254 0 6 2,043,974 119,900 1,924,074 254 0x2be9/0x3eab48<br />
30 19,902 0 4 7,727,051 2,893,348,983 0 19,954 0x2c00/0x1bfc93<br />
27,984 75,433 0 32 29,896,927 2,885,621,932 0 103,417 0x2c13/0x1a69f2<br />
40,477 36,001 0 10 19,737,838 17,591,303 2,146,535 76,478 0x2c2a/0x1669e0<br />
2 2 0 4 194,668 36,808 157,860 10 0x2c2a/0x19624c<br />
29,246 68,367 0 3 31,325,183 2,835,792,499 0 97,615 0x2c4c/0x2d9acb<br />
0 26 0 5 1,118,302 27,604 1,090,698 26 0x2c4c/0x3eab29<br />
4 19,763 0 3 7,562,665 2,803,349,014 0 19,775 0x2c53/0x197a52<br />
26 194 0 6 1,436,963 103,803 1,333,160 220 0x2c53/0x2f6775<br />
6,255 14,988 0 8 9,972,102 2,794,349,386 0 21,292 0x2c5f/0x2efa7b<br />
39,212 18,305 0 5 18,591,243 2,784,377,284 0 57,517 0x2c8f/0x197b86<br />
0 41 0 6 1,495,215 121,720 1,373,495 41 0x2c8f/0x304c35<br />
347 23,340 0 3 7,036,643 2,764,290,826 0 23,697 0x2c95/0x31498<br />
23,408 9,116 0 3 10,098,528 2,757,254,183 0 32,524 0x2ca4/0x49578<br />
45,631 27,183 0 3 19,972,142 2,747,155,655 0 72,814 0x2cad/0x428a6<br />
0 0 0 1 963,802 69 963,733 0 0x2cad/0x12dd80<br />
276 19,066 0 4 10,705,056 5,926,693 4,778,363 19,342 0x2cc4/0x1d9fa0<br />
0 25,292 0 2 10,568,825 7,122,257 3,446,568 25,292 0x2ccf/0x264d99<br />
722 25,106 0 3 9,376,076 2,704,945,830 0 26,568 0x2cd3/0x1cc865<br />
15 96 0 6 601,987 65,907 536,080 111 0x2cd3/0x25f7e8<br />
50,662 51,942 0 4 29,283,798 2,694,967,767 0 102,611 0x2d20/0x1b0a3e<br />
18,486 16,140 0 4 12,291,893 2,665,683,969 0 34,626 0x2d3a/0x3e02f3<br />
67,481 104,443 0 5 48,233,597 15,261,303 32,972,294 171,924 0x2d79/0x230ef0<br />
19,276 34,062 0 3 18,103,592 2,605,158,479 0 171,476 0x2deb/0x61f18<br />
0 225 0 6 607,824 37,538 570,286 225 0x2deb/0xf6568<br />
14,038 14,130 0 3 13,699,081 2,586,447,063 0 28,385 0x2e21/0x47d6f1<br />
4,147 13,212 0 3 5,363,836 2,572,747,982 0 17,359 0x2e40/0x118ed<br />
0 0 0 4 133,826 37,921 95,905 0 0x2e40/0x323af<br />
31,744 33,396 0 3 22,247,625 2,567,250,320 0 65,140 0x2e45/0x256f78<br />
9 0 0 6 290,601 55,855 234,746 9 0x2e45/0x29dea1<br />
16 25,020 0 4 10,475,420 2,544,712,094 0 25,039 0x2e5b/0x311fbd<br />
1,682 19,008 0 3 8,507,134 2,534,236,674 0 20,690 0x2e5f/0x1a583b<br />
12 0 0 6 354,107 41,047 313,060 12 0x2e5f/0x1fbf76<br />
6,849 54,863 0 3 20,176,980 2,525,375,433 0 61,719 0x2e7d/0x2272ca<br />
30,012 67,588 0 5 28,465,075 27,606,214 858,861 97,600 0x2eb6/0xb06fd<br />
14,048 16,912 0 2 9,587,676 2,476,733,378 0 66,881 0x2ee1/0x4bc59<br />
15,183 12,704 0 6 11,270,153 6,834,792 4,435,361 27,887 0x2ef4/0x181de2<br />
23,012 39,873 0 7 18,610,316 2,455,875,549 0 62,948 0x2f1a/0x2e96e<br />
25,472 49,151 0 6 22,681,935 2,437,265,233 0 74,623 0x2f5a/0x2bd5bd<br />
1,031 60,319 0 6 20,425,359 2,414,583,298 0 61,350 0x2f66/0x32534c<br />
48,185 83,225 0 4 42,656,335 28,813,330 13,843,005 131,410 0x2f9d/0x5adb9b<br />
16,859 32,139 0 3 18,804,637 12,310,056 6,494,581 74,600 0x2fa3/0x489e38<br />
29,050 79,391 0 8 47,784,742 37,879,192 9,905,550 108,511 0x300a/0x26d0de<br />
32 38,306 0 6 19,991,849 14,244,764 5,747,085 117,588 0x3026/0x26b107<br />
0 37 0 6 345,851 39,476 306,375 25,114 0x3026/0x2bf802<br />
17,725 34,199 0 3 19,155,357 2,264,574,525 0 51,955 0x303f/0x1f149f<br />
1,379 23,909 0 6 10,787,679 2,245,419,168 0 25,288 0x304f/0x2b197e<br />
3 12 0 5 297,677 34,135 263,542 15 0x304f/0x2fa44b<br />
17,968 47,908 0 2 17,014,500 2,234,333,812 0 65,891 0x306c/0x2162f<br />
45,151 19,691 0 3 20,592,703 18,998,392 1,594,311 64,842 0x3071/0xb216e<br />
18,246 13,219 0 3 14,900,440 2,196,726,609 0 31,514 0x308d/0x55e7c6<br />
26 12,008 0 5 7,554,365 4,234,132 3,320,233 12,034 0x30a1/0x309683<br />
35,556 69,762 0 20 31,161,221 2,174,271,804 0 105,338 0x30c2/0x424e88<br />
33,203 114,692 0 5 49,168,609 43,804,782 5,363,827 147,895 0x30fd/0x359ee9<br />
182 19,563 0 7 10,363,229 2,093,941,974 0 87,398 0x310c/0x3b29c6<br />
14 19,817 0 4 9,355,037 4,743,642 4,611,395 19,831 0x3110/0x315263<br />
46 37,708 0 8 20,678,814 10,884,622 9,794,192 41,674 0x315c/0x3bae01<br />
88 279 0 5 761,100 44,429 716,671 3,390 0x315c/0x474b0d<br />
40,709 61,572 0 4 26,837,961 2,052,783,794 0 102,607 0x3189/0x170b56<br />
42 0 0 6 814,957 77,642 737,315 42 0x3189/0x237ac3<br />
69,126 107,504 0 5 52,094,116 47,372,898 4,721,218 176,645 0x31b8/0x436ee7<br />
46 0 0 5 393,197 51,476 341,721 82,129 0x31b8/0x496ed4<br />
48,348 117,823 0 3 48,879,763 41,965,088 6,914,675 166,196 0x3201/0x3856e7<br />
16,349 39,063 0 7 17,875,388 14,606,658 3,268,730 62,982 0x3208/0x17eba3<br />
105 0 0 6 3,394,912 60,919 3,333,993 2,571 0x3208/0x4bb903<br />
217 17,310 0 7 5,776,110 1,902,493,500 0 17,617 0x3210/0xb4571<br />
36,657 27,268 0 7 22,160,771 4,617,088 17,543,683 63,925 0x3224/0x2c3df4<br />
24,153 42,022 0 1 22,019,603 1,874,556,619 0 114,113 0x3235/0x4b0f07<br />
16 18,174 0 7 7,706,321 1,852,537,016 0 18,190 0x324e/0x280f58<br />
120 24,379 0 7 10,458,169 1,844,830,695 0 24,499 0x3266/0x2f0d11<br />
14,236 34,916 0 3 17,998,326 12,723,415 5,274,911 49,152 0x327b/0x108207<br />
79 0 0 5 673,666 37,699 635,967 2,808 0x327b/0x1ac989<br />
6,624 22,958 0 3 11,999,224 1,815,700,534 0 29,650 0x3290/0x394b01<br />
19,965 40,810 0 8 17,269,398 1,803,701,310 0 60,775 0x32a7/0xfa097<br />
7,517 58,175 0 5 21,772,528 1,786,431,912 0 65,692 0x32de/0x2aac87<br />
9,415 18,600 0 6 10,786,946 1,764,659,384 0 28,015 0x32e1/0x36ae89<br />
51 0 0 6 622,591 52,177 570,414 51 0x32e1/0x402e88<br />
30,139 66,899 0 4 30,497,693 1,753,249,847 0 97,065 0x3315/0x47c6a5<br />
6 41,284 0 1 16,342,999 12,394,744 3,948,255 41,290 0x3320/0xff97c<br />
21,091 17,059 0 6 11,525,646 1,706,409,155 0 38,984 0x332b/0x27410a<br />
0 94 0 4 892,946 34,897 858,049 94 0x332b/0x34e11c<br />
3 52,035 0 3 18,160,444 1,693,990,563 0 52,132 0x336d/0x18cf58<br />
58 27,035 0 6 8,947,956 1,675,830,119 0 27,093 0x3376/0x8c1cc<br />
3 66 0 4 501,044 49,007 452,037 69 0x3376/0x106700<br />
27,134 44,130 0 4 22,994,532 1,666,381,119 0 71,330 0x339f/0x3e1864<br />
0 108 0 4 439,894 35,044 404,850 108 0x339f/0x44ceba<br />
17 51,506 0 3 20,284,195 1,642,946,693 0 51,631 0x33a6/0x4924dd<br />
28 16,826 0 2 9,954,053 1,622,662,498 0 16,854 0x33b6/0x487162<br />
2,079 43,231 0 2 20,850,856 13,682,453 7,168,403 45,310 0x33d2/0x556d0a<br />
2 30,027 0 3 14,418,091 8,924,719 5,493,372 30,743 0x33d9/0x40b5<br />
0 1,157 0 6 5,510,215 155,394 5,354,821 3,103 0x33d9/0x5454fc<br />
31,517 31,808 0 3 18,110,480 1,571,929,283 0 64,085 0x33ee/0x37800c<br />
30,598 27,193 0 4 18,121,300 12,201,650 5,919,650 57,791 0x3402/0x1ad560<br />
38,905 34,440 0 4 18,443,706 15,100,893 3,342,813 78,249 0x340c/0x3161a<br />
51,887 95,209 0 4 40,262,214 29,791,799 10,470,415 152,283 0x343b/0x71660<br />
1 28,686 0 7 10,081,465 1,476,991,583 0 142,625 0x344c/0x85499<br />
28,401 13,707 0 3 10,169,877 1,466,910,118 0 42,108 0x3457/0xaec2e<br />
5 16,531 0 6 14,562,193 4,402,715 10,159,478 16,536 0x345b/0x50893f<br />
0 15,920 0 7 9,514,322 1,442,178,048 0 19,604 0x3460/0x492011<br />
4 16,264 0 6 10,131,895 1,432,663,726 0 16,268 0x3465/0x4b2348<br />
30,231 28,986 0 4 18,938,103 14,093,138 4,844,965 59,217 0x347c/0x3aef3f<br />
14 0 0 5 512,006 34,432 477,574 120 0x347c/0x42bf45<br />
20,678 38,294 0 3 17,101,301 1,403,081,722 0 58,975 0x3497/0x16843a<br />
9,869 25,099 0 5 12,548,707 1,385,980,421 0 34,968 0x34b3/0x3d681d<br />
36,465 60,605 0 5 26,361,794 24,558,695 1,803,099 97,070 0x34d6/0x5e45f<br />
36 23,345 0 3 9,621,974 6,699,792 2,922,182 23,911 0x34d9/0x1fb5<br />
25,436 49,579 0 7 20,880,498 18,608,524 2,271,974 75,850 0x34df/0xd8f27<br />
32 1,485 0 6 4,197,555 413,166 3,784,389 1,535 0x34df/0x4d9bda<br />
31,535 59,968 0 5 25,248,912 24,728,797 520,115 91,849 0x3512/0x51cea<br />
1,601 24,378 0 6 10,268,411 8,200,696 2,067,715 26,160 0x351b/0x93565<br />
14,147 23,162 0 8 11,300,331 1,276,852,570 0 37,400 0x3525/0x1d0cd0<br />
14,204 20,662 0 3 13,299,117 1,265,552,239 0 34,866 0x3532/0x4f63fd<br />
17,514 24,728 0 3 16,797,576 12,513,187 4,284,389 42,242 0x3553/0x1e8685<br />
7,303 46,343 0 7 23,094,370 14,762,461 8,331,909 54,344 0x3580/0x4dbde7<br />
45,620 41,812 0 7 27,514,220 21,511,990 6,002,230 90,879 0x359c/0x27cfd3<br />
16,861 12,552 0 3 9,619,073 1,184,846,956 0 104,424 0x35a0/0x21ffd4<br />
0 271 0 6 3,731,928 93,043 3,638,885 271 0x35a0/0x5af1ac<br />
30,860 48,187 0 7 26,338,978 24,109,627 2,229,351 79,097 0x35ab/0x2314ce<br />
39,031 24,255 0 4 20,202,284 1,145,156,977 0 63,301 0x35c4/0x262afa<br />
2,576 46,002 0 6 19,184,048 5,580,647 13,603,401 48,578 0x35dc/0x19b7aa<br />
2 0 0 6 174,028 33,089 140,939 31,536 0x35dc/0x1c5f76<br />
111,528 77,161 0 7 62,341,581 50,549,489 11,792,092 188,690 0x360c/0x401a43<br />
28,697 25,910 0 6 18,246,350 13,855,848 4,390,502 56,636 0x3618/0x255811<br />
4,335 12,127 0 6 9,004,926 5,130,460 3,874,466 18,985 0x3621/0x16290f<br />
18,413 8,607 0 5 8,878,516 7,701,363 1,177,153 27,027 0x3626/0x50c43<br />
0 2,910 0 6 6,228,723 848,601 5,380,122 2,910 0x3626/0x641736<br />
28,784 22,689 0 3 21,119,867 14,626,185 6,493,682 52,178 0x362a/0x752db1<br />
41,676 62,528 0 4 34,112,988 29,303,765 4,809,223 105,238 0x363e/0x1b598d<br />
17 735 0 5 12,521,216 31,105 12,490,111 1,300 0x3643/0x41d20d<br />
7,802 34,494 0 7 33,127,169 7,514,191 25,612,978 43,003 0x3659/0x71898e<br />
0 8,734 0 5 3,377,408 900,015,281 0 25,130 0x365c/0xc7c0e<br />
114 9,858 0 7 15,782,161 650,578 15,131,583 9,972 0x365f/0x64b69f<br />
1 11,516 0 7 5,378,418 880,855,712 0 19,548 0x3664/0x1e3191<br />
7 22,005 0 7 8,922,863 6,691,187 2,231,676 22,012 0x3668/0xdc200<br />
13 13,409 0 6 15,952,421 360,887 15,591,534 14,391 0x3671/0x6895a5<br />
1,749 8,568 0 3 3,660,457 850,602,010 0 22,698 0x3676/0x7d9ce<br />
5,390 10,146 0 6 14,882,555 1,855,609 13,026,946 15,536 0x3688/0x525a49<br />
10,230 9,513 0 5 4,816,061 832,058,998 0 28,465 0x3691/0x34086<br />
14,698 31,907 0 7 41,080,361 36,925,183 4,155,178 46,605 0x36aa/0x13bcaf<br />
2 1,755 0 6 4,666,159 345,619 4,320,540 47,630 0x36aa/0x5aefde<br />
25,935 30,385 0 5 38,398,767 781,496,417 0 57,119 0x36c5/0x42810d<br />
9,478 14,696 0 7 21,102,013 743,097,650 0 24,174 0x36e1/0x5351ca<br />
65 12,746 0 6 5,610,587 3,858,013 1,752,574 12,811 0x36ed/0x1057a5<br />
5,806 9,341 0 6 10,353,968 2,690,013 7,663,955 15,343 0x36fc/0x15be55<br />
5 0 0 6 4,894,393 33,744 4,860,649 5,931 0x36fc/0x606d0e<br />
28,754 58,230 0 18 28,401,618 701,136,689 0 86,988 0x3708/0x480960<br />
0 15,897 0 6 10,890,931 3,098,266 7,792,665 15,897 0x370a/0x55a193<br />
126 10,927 0 6 26,068,972 2,214,107 23,854,865 16,100 0x372d/0x19a5ff<br />
9,837 16,155 0 6 11,096,409 6,346,816 4,749,593 29,696 0x3732/0x2a60d8<br />
128 15,250 0 6 8,591,606 3,747,801 4,843,805 16,046 0x3734/0x14e34e<br />
123 28,336 0 6 16,047,724 5,161,792 10,885,932 31,737 0x3736/0x712b3a<br />
24 8,534 0 5 3,423,605 1,552,888 1,870,717 19,982 0x3740/0xcd22f<br />
533 0 0 6 20,325,930 275,776 20,050,154 3,884 0x3742/0x11cb59<br />
0 0 0 4 72,242 36,230 36,012 0 0x3742/0x12e58b<br />
26,449 16,768 0 3 20,777,536 576,217,652 0 43,217 0x3756/0x1ec2cb<br />
11,493 23,349 0 3 21,836,979 13,435,005 8,401,974 34,842 0x375c/0x3aca7e<br />
309 32,239 0 6 17,558,766 9,588,731 7,970,035 56,790 0x3761/0x158a6c<br />
12 26,562 0 7 12,407,639 6,425,279 5,982,360 27,257 0x3779/0x3a4743<br />
6,845 41,908 0 7 33,072,537 22,775,381 10,297,156 51,339 0x379b/0x69295c<br />
6,778 29,990 0 6 25,664,453 17,110,934 8,553,519 41,294 0x37a8/0x2701a1<br />
5 16,509 0 9 19,648,397 8,979,547 10,668,850 44,387 0x37ae/0x21a42e<br />
116,569 137,423 0 12 129,250,694 72,386,145 56,864,549 267,033 0x37e2/0x163534<br />
13 17,546 0 6 28,042,727 34,685 28,008,042 187,464 0x37e7/0x90ee1b<br />
33,326 30,872 0 6 43,125,925 9,805,827 33,320,098 81,756 0x37ec/0x280a40<br />
3 5,332 0 5 19,873,538 37,653 19,835,885 30,103 0x37ef/0x261c42<br />
17 2,306 0 6 19,166,423 244,987 18,921,436 7,657 0x37f1/0x196419<br />
10,753 20,277 0 6 40,648,779 11,079,498 29,569,281 32,618 0x37f6/0x234a64<br />
10 22,876 0 6 18,391,375 164,634 18,226,741 49,034 0x37f8/0xabeb3<br />
30 34,377 0 7 32,006,385 1,775,081 30,231,304 56,545 0x37fb/0x295c24<br />
4 23,090 0 7 38,624,657 2,288,459 36,336,198 52,101 0x37ff/0x145fb5<br />
6 8,689 0 6 27,108,848 949,959 26,158,889 24,449 0x3801/0x80d8a5<br />
6 5,117 0 7 18,644,436 55,620 18,588,816 11,221 0x3803/0x6c2979<br />
837 0 0 6 6,661,670 288,422 6,373,248 5,956 0x3804/0x39391f<br />
5,863 6,917 0 0 3,705,888 3,705,888 0 12,780 0x3804/0x71c53f<br />
</DbPrintLog><br />
<br />
* This prints some statistics about the database. In particular the two rows called key bytes and data bytes are relevant. For the following step you need the values from the avg column - i.e. 38 and 219 in this case. You also need the value of “Total number of entries” (14543762 in this case). Now we take those values and put them into this command:<br />
<br />
[root@dc042 meta]# java -cp /usr/share/dcache/classes/je-*.jar com.sleepycat.je.util.DbCacheSize -records 14543762 -key 38 -data 219<br />
<br />
=== Environment Cache Overhead ===<br />
<br />
3,157,213 minimum bytes<br />
<br />
To account for JE daemon operation and record locks,<br />
a significantly larger amount is needed in practice.<br />
<br />
=== Database Cache Size ===<br />
<br />
Number of Bytes Description<br />
--------------- -----------<br />
1,195,759,832 Internal nodes only<br />
5,124,232,392 Internal nodes and leaf nodes<br />
<br />
* For further information see the DbCacheSize javadoc.<br />
<br />
* For this pool it tells us that we need a bit above 1195 MB to keep all the internal btree nodes cached. Oracle’s recommendation is that if the database is updated often, the cache is at least big enough to contain the internal nodes. Obviously you want to make it a bit bigger to leave room for it to grow. One could take this and configure the cache using the je.maxMemory setting. If the size is lower than the 20% of the max heap size you already use, then you don’t need to do anything (I do not suggest lowering it further).<br />
<br />
* Since this pool has 6000m of JAVA heap max space, it means that 20% = 1200MB, which is exactly cache for Internal btree nodes. So we will not touch this.<br />
<br />
* There is one caveat though: If you increase the cache size, less free space is left on the heap. If the pool is pushed to the limit, this may actually slow down the pool as garbage collection overhead increases. You need to ensure that enough space is left in addition to the cache (possibly by increasing the max heap size). An alternative to adjusting je.maxMemory is of course to adjust the max heap size. If you make it large enough so that 20% is enough to cache the internal btree nodes, then all is well. This may however mean you assign significantly more memory to the pool than it really needs. Yeah, there are lots of things to consider.<br />
<br />
* A little bonus: There is also this command:<br />
<br />
[root@dc042 meta]# java -cp /usr/share/dcache/classes/je-*.jar com.sleepycat.je.util.DbSpace -h .<br />
File Size (KB) % Used<br />
-------- --------- ------<br />
00001857 9765 41<br />
00001f2a 9765 43<br />
00001f2f 9765 39<br />
00001f72 9765 44<br />
0000216d 9765 40<br />
000021cb 9765 40<br />
0000226b 9765 45<br />
000022e8 9763 38<br />
000022fa 9765 39<br />
00002390 9765 38<br />
0000241d 9765 49<br />
0000252c 9765 51<br />
0000254c 9765 47<br />
00002567 9765 39<br />
000025b7 9765 42<br />
000025db 9765 42<br />
000025e5 9765 56<br />
0000265f 9765 39<br />
00002678 9765 61<br />
0000268d 9762 48<br />
00002692 9765 74<br />
000026be 9765 48<br />
000026cf 9765 40<br />
00002700 9765 41<br />
00002706 9765 38<br />
00002718 9765 56<br />
00002747 9765 44<br />
0000274d 9765 48<br />
000027db 9765 62<br />
00002855 9765 39<br />
0000285c 9765 39<br />
00002882 9765 41<br />
0000289d 9765 54<br />
000028ba 9765 52<br />
000028c1 9765 53<br />
000028d0 9765 41<br />
000028d7 9765 41<br />
000028e7 9765 39<br />
000028ed 9765 39<br />
00002926 9765 39<br />
0000292b 9765 43<br />
00002945 9765 41<br />
00002952 9765 41<br />
00002958 9765 62<br />
00002959 9764 38<br />
00002962 9765 53<br />
00002963 9765 47<br />
0000297d 9765 51<br />
0000297e 9765 41<br />
000029b3 9765 45<br />
000029c7 9765 56<br />
000029ce 9765 42<br />
000029d3 9765 43<br />
000029d9 9765 54<br />
000029fb 9765 42<br />
00002a00 9765 54<br />
00002a01 9765 39<br />
00002a06 9765 39<br />
00002a20 9765 45<br />
00002a26 9765 61<br />
00002a2b 9765 69<br />
00002a30 9765 65<br />
00002a44 9765 46<br />
00002a4f 9765 50<br />
00002a53 9765 64<br />
00002a54 9765 47<br />
00002a58 9765 66<br />
00002a59 9765 43<br />
00002a5e 9765 56<br />
00002a63 9765 44<br />
00002a69 9765 47<br />
00002a8f 9765 39<br />
00002aac 9765 54<br />
00002aba 9765 42<br />
00002abb 9765 44<br />
00002ae0 9763 38<br />
00002af3 9765 45<br />
00002b01 9765 41<br />
00002b08 9765 65<br />
00002b0e 9765 41<br />
00002b24 9765 45<br />
00002b28 9765 61<br />
00002b29 9765 41<br />
00002b3c 9765 40<br />
00002b3d 9765 40<br />
00002b41 9765 47<br />
00002b4c 9765 46<br />
00002b52 9765 57<br />
00002b64 9765 38<br />
00002b87 9765 38<br />
00002b8e 9765 47<br />
00002b8f 9760 38<br />
00002bad 9765 42<br />
00002bbf 9765 44<br />
00002bc6 9765 38<br />
00002be9 9765 47<br />
00002c00 9765 42<br />
00002c06 9765 69<br />
00002c0c 9765 55<br />
00002c13 9765 38<br />
00002c14 9765 38<br />
00002c2a 9765 41<br />
00002c2b 9765 41<br />
00002c30 9765 54<br />
00002c4c 9765 39<br />
00002c53 9765 45<br />
00002c5f 9765 44<br />
00002c7c 9765 38<br />
00002c8f 9765 49<br />
00002c95 9765 38<br />
00002ca4 9764 50<br />
00002ca9 9765 42<br />
00002cad 9763 41<br />
00002cc4 9765 39<br />
00002ccf 9765 43<br />
00002cd3 9765 45<br />
00002cf3 9765 48<br />
00002d19 9765 38<br />
00002d20 9765 38<br />
00002d3a 9765 40<br />
00002d3b 9765 41<br />
00002d41 9765 43<br />
00002d51 9765 39<br />
00002d59 9765 40<br />
00002d79 9765 62<br />
00002d8f 9765 39<br />
00002deb 9765 40<br />
00002e21 9765 39<br />
00002e40 9765 41<br />
00002e41 9765 45<br />
00002e45 9765 40<br />
00002e5b 9765 58<br />
00002e5f 9765 49<br />
00002e60 9765 67<br />
00002e7d 9765 48<br />
00002e95 9765 38<br />
00002eb1 9762 57<br />
00002eb6 9765 53<br />
00002ee1 9765 38<br />
00002ef4 9763 43<br />
00002ef8 9765 41<br />
00002f1a 9765 53<br />
00002f25 9765 65<br />
00002f5a 9765 52<br />
00002f5f 9765 63<br />
00002f66 9765 41<br />
00002f71 9765 42<br />
00002f79 9765 61<br />
00002f87 9765 49<br />
00002f9d 9765 41<br />
00002f9e 9765 42<br />
00002fa3 9765 38<br />
00002fa4 9765 46<br />
00002fb1 9762 55<br />
00002fc1 9765 41<br />
00002fde 9765 38<br />
0000300a 9765 43<br />
0000300e 9762 47<br />
00003026 9765 55<br />
00003027 9765 57<br />
0000303f 9765 63<br />
0000304f 9765 48<br />
00003056 9765 44<br />
0000306c 9765 48<br />
0000306d 9765 42<br />
00003071 9765 45<br />
0000308d 9765 39<br />
000030a1 9765 54<br />
000030a5 9765 51<br />
000030ab 9765 60<br />
000030c2 9765 46<br />
000030d5 9765 43<br />
000030dd 9765 38<br />
000030e3 9765 57<br />
000030e4 9765 55<br />
000030fd 9765 48<br />
0000310c 9764 42<br />
00003110 9765 46<br />
00003119 9765 47<br />
0000315c 9765 48<br />
0000315d 9765 54<br />
00003185 9765 56<br />
00003189 9765 52<br />
0000318a 9765 58<br />
0000319a 9765 46<br />
0000319f 9764 39<br />
000031b5 9764 53<br />
000031b8 9765 40<br />
000031b9 9765 74<br />
000031e2 9765 65<br />
000031fa 9765 58<br />
000031fb 9765 52<br />
00003201 9765 50<br />
00003202 9764 44<br />
00003208 9765 42<br />
00003210 9763 40<br />
00003218 9765 40<br />
00003224 9765 51<br />
00003229 9765 48<br />
00003235 9765 40<br />
0000324e 9765 46<br />
00003266 9765 41<br />
0000326c 9764 47<br />
0000327b 9765 55<br />
00003290 9765 48<br />
00003294 9765 56<br />
000032a7 9765 69<br />
000032ad 9765 62<br />
000032de 9765 47<br />
000032e1 9765 46<br />
000032e2 9765 71<br />
000032e9 9765 45<br />
00003315 9765 42<br />
00003316 9764 54<br />
00003320 9765 43<br />
0000332b 9765 52<br />
0000332c 9765 76<br />
0000336d 9765 69<br />
00003376 9765 72<br />
00003377 9765 46<br />
0000339f 9765 43<br />
000033a0 9765 75<br />
000033a6 9765 46<br />
000033b6 9765 40<br />
000033c0 9764 72<br />
000033d2 9765 38<br />
000033d3 9764 45<br />
000033d9 9765 42<br />
000033da 9765 44<br />
000033ee 9765 41<br />
000033fb 9764 38<br />
00003402 9765 42<br />
00003403 9765 53<br />
0000340c 9765 76<br />
00003428 9765 67<br />
00003431 9765 51<br />
00003436 9765 88<br />
0000343b 9765 85<br />
0000344c 9765 48<br />
00003457 9765 46<br />
0000345b 9765 46<br />
00003460 9765 50<br />
00003465 9765 45<br />
00003466 9762 40<br />
0000347c 9765 54<br />
0000347d 9765 61<br />
00003497 9765 63<br />
000034b3 9765 64<br />
000034bd 9765 54<br />
000034c9 9764 71<br />
000034d6 9765 65<br />
000034d9 9765 86<br />
000034da 9765 46<br />
000034df 9765 51<br />
000034e0 9764 60<br />
00003504 9762 80<br />
00003512 9761 76<br />
0000351b 9765 65<br />
00003525 9765 59<br />
00003532 9765 52<br />
00003540 9764 43<br />
00003553 9765 80<br />
00003558 9765 54<br />
00003580 9762 49<br />
00003586 9765 41<br />
00003587 9763 50<br />
0000359c 9765 40<br />
000035a0 9765 46<br />
000035a1 9765 37<br />
000035a6 9765 96<br />
000035ab 9765 52<br />
000035ac 9765 39<br />
000035c4 9765 61<br />
000035d0 9765 89<br />
000035dc 9765 74<br />
000035dd 9765 43<br />
000035f1 9765 43<br />
000035fd 9765 37<br />
00003604 9765 47<br />
00003609 9764 40<br />
0000360c 9765 59<br />
0000360d 9765 37<br />
00003618 9763 42<br />
00003621 9765 42<br />
00003626 9765 42<br />
00003627 9765 41<br />
0000362a 9765 38<br />
0000362b 9765 73<br />
00003630 9765 68<br />
00003631 9764 42<br />
0000363e 9763 39<br />
00003643 9765 43<br />
00003646 9765 38<br />
0000364c 9762 40<br />
00003659 9765 47<br />
0000365c 9764 44<br />
0000365f 9765 45<br />
00003664 9765 69<br />
00003668 9764 37<br />
00003671 9765 43<br />
00003676 9765 39<br />
00003688 9765 37<br />
00003691 9765 48<br />
00003692 9764 37<br />
00003695 9762 38<br />
000036a3 9764 50<br />
000036aa 9765 47<br />
000036ab 9765 49<br />
000036c2 9765 64<br />
000036c3 9764 57<br />
000036c5 9765 40<br />
000036da 9764 40<br />
000036e1 9765 43<br />
000036ed 9762 39<br />
000036fc 9765 41<br />
000036fd 9765 59<br />
00003701 9764 85<br />
00003708 9765 43<br />
0000370a 9763 49<br />
00003728 9762 41<br />
0000372a 9763 55<br />
0000372d 9765 51<br />
00003732 9763 48<br />
00003734 9765 60<br />
00003736 9765 42<br />
00003740 9765 48<br />
00003741 9765 39<br />
00003742 9765 38<br />
00003749 9764 40<br />
00003756 9763 39<br />
00003758 9762 46<br />
0000375c 9765 67<br />
0000375d 9765 37<br />
00003761 9765 63<br />
00003779 9765 55<br />
00003785 9765 43<br />
00003798 9763 60<br />
0000379b 9765 41<br />
0000379e 9765 37<br />
000037a5 9764 39<br />
000037a8 9764 48<br />
000037ab 9765 38<br />
000037ae 9765 54<br />
000037b3 9764 42<br />
000037b6 9761 36<br />
000037be 9763 46<br />
000037c1 9765 69<br />
000037c3 9765 36<br />
000037c4 9764 37<br />
000037ce 9762 50<br />
000037d1 9761 45<br />
000037d3 9765 45<br />
000037dd 9765 45<br />
000037df 9765 50<br />
000037e0 9763 39<br />
000037e2 9765 13<br />
000037e3 9763 18<br />
000037e7 9765 49<br />
000037e8 9763 41<br />
000037e9 9765 42<br />
000037ea 9763 40<br />
000037eb 9765 39<br />
000037ec 9761 39<br />
000037ed 9763 41<br />
000037ef 9763 36<br />
000037f0 9764 36<br />
000037f1 9764 39<br />
000037f3 9765 41<br />
000037f4 9761 53<br />
000037f5 9765 36<br />
000037f6 9765 54<br />
000037f7 9763 53<br />
000037f8 9765 63<br />
000037f9 9765 60<br />
000037fa 9765 70<br />
000037fb 9765 76<br />
000037fc 9765 72<br />
000037fd 9765 73<br />
000037fe 9765 83<br />
000037ff 9765 87<br />
00003800 9765 89<br />
00003801 9762 86<br />
00003802 9763 88<br />
00003803 9764 90<br />
00003804 8110 76<br />
TOTALS 3757891 49<br />
<br />
* It will tell you for each of the database files (the log segments) how big it is and what the utilization is. The Berkeley DB will try to keep the total utilization above 50%, but for a small pool like this one it cannot do it. I figure that you may find this interesting to know after reading about utilization above.<br />
<br />
* The log segment size and the utilization goal can be adjusted too, but I cannot give any sound advice on whether that’s a good idea and how to determine good values.<br />
<br />
* All of the above is only relevant if you use the Berkeley DB backend for pools.<br />
<br />
== Special thanks ==<br />
* This documentation has been deployed thanks to Gerd Behrmann, you can find more in at: https://gist.github.com/gbehrmann/df1abbe45c03356eff2e</div>Mcaubethttps://pwiki.pic.es/index.php?title=DCache_gPlazma:_htpasswd_management&diff=225DCache gPlazma: htpasswd management2017-10-05T09:08:06Z<p>Mcaubet: /* How to generate a new password in htpasswd */</p>
<hr />
<div>= How to generate a new password in htpasswd =<br />
* Log in a door and execute the following command:<br />
[root@door01 ~]# htpasswd -n -m <user><br />
New password: <br />
Re-type new password:<br />
<user>:$apr1$Knkg1eYq$WKIQSP0p0m2XKhAasbIRD1<br />
:where '''MD5''' encryption must be used (option '''-m''') and option '''-n''' just forces to show the output on stdout.<br />
<br />
* Copy the output to the htpasswd file locate in '''/etc/dcache/htpasswd'''. This file is managed in '''puppet'.</div>Mcaubethttps://pwiki.pic.es/index.php?title=DCache_gPlazma:_htpasswd_management&diff=224DCache gPlazma: htpasswd management2017-10-05T09:06:11Z<p>Mcaubet: Created page with "= How to generate a new password in htpasswd = * Log in a door and execute the following command: [root@door01 ~]# htpasswd -n -m <user> New password: Re-type new password..."</p>
<hr />
<div>= How to generate a new password in htpasswd =<br />
* Log in a door and execute the following command:<br />
[root@door01 ~]# htpasswd -n -m <user><br />
New password: <br />
Re-type new password:<br />
<user>:$apr1$Knkg1eYq$WKIQSP0p0m2XKhAasbIRD1<br />
* Copy the output to the htpasswd file locate in '''/etc/dcache/htpasswd'''. This file is managed in '''puppet'.</div>Mcaubethttps://pwiki.pic.es/index.php?title=HowTo:_Configuring_gPlazma&diff=223HowTo: Configuring gPlazma2017-10-05T08:58:01Z<p>Mcaubet: /* htpasswd */</p>
<hr />
<div>= Configuring gPlazma =<br />
== gPlazma configuration file example ==<br />
################################################################################<br />
# Login Phases:<br />
# [auth]<br />
# - auth-plugins are used to read the users public and private<br />
# credentials and ask some authority, if those are valid for accessing <br />
# the system. <br />
# [map]<br />
# - map-plugins map the user information obtained in the auth step to UID<br />
# and GIDs. <br />
# - This may also be done in several steps (e.g., the vorolemap plugin <br />
# maps the users DN+FQAN to a username which is then mapped to UID/GIDs<br />
# by the authzdb plugin)<br />
# [account]<br />
# - account-plugins verify the validity of a possibly mapped of the user <br />
# and may reject the login depending on information gathered within the <br />
# map step.<br />
# [session]<br />
# - session plugins usually enrich the session with additional attributes<br />
# like the user’s home directory.<br />
# [identity]<br />
# - identity plugins are responsible for mapping UID and GID to user<br />
# names and vice versa during the work with dCache. <br />
################################################################################<br />
<br />
################################################################################<br />
# Modifiers:<br />
# [optional]<br />
# - The success or failure of this plugin is only important if it is the <br />
# only plugin in the stack associated with this type. <br />
# [sufficient]<br />
# - Success of such a plugin is enough to satisfy the authentication <br />
# requirements of the stack of plugins (if a prior required plugin has<br />
# failed the success of this one is ignored). <br />
# - A failure of this plugin isn't deemed as fatal for the login attempt.<br />
# - If the plugin succeeds gPlazma2 immediately proceeds with the next <br />
# plugin type or returns control to the door if this was the last stack. <br />
# [required]<br />
# - Failure of such a plugin will ultimately lead to gPlazma2 returning <br />
# failure but only after the remaining plugins for this type have been <br />
# invoked. <br />
# [requisite]<br />
# - Like required, however, in the case that such a plugin returns a<br />
# failure, control is directly returned to the d<br />
################################################################################<br />
auth sufficient htpasswd<br />
auth optional x509<br />
auth sufficient voms<br />
# send username and password to LDAP server<br />
auth optional jaas gplazma.jaas.name=LdapGplazma<br />
map optional vorolemap<br />
map sufficient authzdb<br />
map optional gridmap<br />
# convert LDAP specific result to dCache internal principals<br />
map optional mutator gplazma.mutator.accept=com.sun.security.auth.UserPrincipal gplazma.mutator.produce=username<br />
# use ldap to map user name to uid<br />
# map optional ldap<br />
map requisite nsswitch<br />
identity requisite nsswitch<br />
session sufficient nsswitch<br />
session sufficient authzdb<br />
== Authetincation types ==<br />
=== jaas ===<br />
* ''This is intended for PIC LDAP users accessing to WebDAV.''<br />
* JAAS, custom ''/etc/dcache/jgss.conf''<br />
=== htpasswd ===<br />
* ''This is intended only for the WebDAV protocol.''<br />
* Configuration file can be found here: ''/etc/dcache/htpasswd''<br />
* To generate a new entry in the '''htpasswd''' file proceed as follows: [[dCache gPlazma: htpasswd management]]<br />
<br />
=== x509 ===<br />
* ''This is mostly intented for the WebDAV protocol.''<br />
* Users defined in ''/etc/grid-security/grid-mapfile''.<br />
:* '''/etc/grid-security/grid-mapfile''' is generated with a ''grid-mapfile'' cron (''crontab -l'') which executes the '''edg-mkgridmap''' application.<br />
# Puppet Name: grid-mapfile<br />
0 */6 * * * [ ! -f /var/lock/subsys/edg-mkgridmap ] && /usr/sbin/edg-mkgridmap --output=- > /etc/grid-security/grid-mapfile.new && sleep 3 && cat /etc/grid-security/grid-mapfile.new > /etc/grid-security/grid-mapfile<br />
:* '''edg-mkgridmap''' application can be configured with the ''/etc/edg-mkgridmap.conf'' file where:<br />
::* You can map VO members by specifying a VOMS service and the affected VO. In example:<br />
group vomss://voms2.cern.ch:8443/voms/lhcb?/lhcb lhcb001<br />
::* '''/etc/grid-mapfile-local''' is a file to bootstrap the actual grid-mapfile and can be used to allow extra DNs or override some mappings<br />
=== voms ===<br />
* ''This is intented for GRID Users, mostly using GridFTP, GSI-DCap & SRM protocols.''<br />
* User defined in ''/etc/grid-security/grid-vorolemap''.<br />
=== multimap ===<br />
* Useful for OIDC authentication (i.e. through Google or INDIGO Accounts)<br />
<br />
== dCache Authorization by protocol ==<br />
=== WebDAV ===<br />
* htpasswd<br />
* x509<br />
=== NFSv3/NFSv41 ===<br />
* Authentication is not performed in the dCache system, is made through OpenLDAP server.<br />
=== GridFTP ===<br />
* x509<br />
* voms<br />
=== DCap/GSI-DCap ===<br />
==== Plain DCap ====<br />
* No authentication required, but needs world readable and/or writable files and directories.<br />
* Only recommended its use for prestages.<br />
==== GSI-DCap ====<br />
* voms<br />
=== XRootD ===<br />
* voms</div>Mcaubethttps://pwiki.pic.es/index.php?title=HowTo:_Configuring_gPlazma&diff=222HowTo: Configuring gPlazma2017-09-06T13:24:36Z<p>Mcaubet: /* dCache Authorization by protocol */</p>
<hr />
<div>= Configuring gPlazma =<br />
== gPlazma configuration file example ==<br />
################################################################################<br />
# Login Phases:<br />
# [auth]<br />
# - auth-plugins are used to read the users public and private<br />
# credentials and ask some authority, if those are valid for accessing <br />
# the system. <br />
# [map]<br />
# - map-plugins map the user information obtained in the auth step to UID<br />
# and GIDs. <br />
# - This may also be done in several steps (e.g., the vorolemap plugin <br />
# maps the users DN+FQAN to a username which is then mapped to UID/GIDs<br />
# by the authzdb plugin)<br />
# [account]<br />
# - account-plugins verify the validity of a possibly mapped of the user <br />
# and may reject the login depending on information gathered within the <br />
# map step.<br />
# [session]<br />
# - session plugins usually enrich the session with additional attributes<br />
# like the user’s home directory.<br />
# [identity]<br />
# - identity plugins are responsible for mapping UID and GID to user<br />
# names and vice versa during the work with dCache. <br />
################################################################################<br />
<br />
################################################################################<br />
# Modifiers:<br />
# [optional]<br />
# - The success or failure of this plugin is only important if it is the <br />
# only plugin in the stack associated with this type. <br />
# [sufficient]<br />
# - Success of such a plugin is enough to satisfy the authentication <br />
# requirements of the stack of plugins (if a prior required plugin has<br />
# failed the success of this one is ignored). <br />
# - A failure of this plugin isn't deemed as fatal for the login attempt.<br />
# - If the plugin succeeds gPlazma2 immediately proceeds with the next <br />
# plugin type or returns control to the door if this was the last stack. <br />
# [required]<br />
# - Failure of such a plugin will ultimately lead to gPlazma2 returning <br />
# failure but only after the remaining plugins for this type have been <br />
# invoked. <br />
# [requisite]<br />
# - Like required, however, in the case that such a plugin returns a<br />
# failure, control is directly returned to the d<br />
################################################################################<br />
auth sufficient htpasswd<br />
auth optional x509<br />
auth sufficient voms<br />
# send username and password to LDAP server<br />
auth optional jaas gplazma.jaas.name=LdapGplazma<br />
map optional vorolemap<br />
map sufficient authzdb<br />
map optional gridmap<br />
# convert LDAP specific result to dCache internal principals<br />
map optional mutator gplazma.mutator.accept=com.sun.security.auth.UserPrincipal gplazma.mutator.produce=username<br />
# use ldap to map user name to uid<br />
# map optional ldap<br />
map requisite nsswitch<br />
identity requisite nsswitch<br />
session sufficient nsswitch<br />
session sufficient authzdb<br />
== Authetincation types ==<br />
=== jaas ===<br />
* ''This is intended for PIC LDAP users accessing to WebDAV.''<br />
* JAAS, custom ''/etc/dcache/jgss.conf''<br />
=== htpasswd ===<br />
* ''This is intended only for the WebDAV protocol.''<br />
* Configuration file can be found here: ''/etc/dcache/htpasswd''<br />
=== x509 ===<br />
* ''This is mostly intented for the WebDAV protocol.''<br />
* Users defined in ''/etc/grid-security/grid-mapfile''.<br />
:* '''/etc/grid-security/grid-mapfile''' is generated with a ''grid-mapfile'' cron (''crontab -l'') which executes the '''edg-mkgridmap''' application.<br />
# Puppet Name: grid-mapfile<br />
0 */6 * * * [ ! -f /var/lock/subsys/edg-mkgridmap ] && /usr/sbin/edg-mkgridmap --output=- > /etc/grid-security/grid-mapfile.new && sleep 3 && cat /etc/grid-security/grid-mapfile.new > /etc/grid-security/grid-mapfile<br />
:* '''edg-mkgridmap''' application can be configured with the ''/etc/edg-mkgridmap.conf'' file where:<br />
::* You can map VO members by specifying a VOMS service and the affected VO. In example:<br />
group vomss://voms2.cern.ch:8443/voms/lhcb?/lhcb lhcb001<br />
::* '''/etc/grid-mapfile-local''' is a file to bootstrap the actual grid-mapfile and can be used to allow extra DNs or override some mappings<br />
=== voms ===<br />
* ''This is intented for GRID Users, mostly using GridFTP, GSI-DCap & SRM protocols.''<br />
* User defined in ''/etc/grid-security/grid-vorolemap''.<br />
=== multimap ===<br />
* Useful for OIDC authentication (i.e. through Google or INDIGO Accounts)<br />
<br />
== dCache Authorization by protocol ==<br />
=== WebDAV ===<br />
* htpasswd<br />
* x509<br />
=== NFSv3/NFSv41 ===<br />
* Authentication is not performed in the dCache system, is made through OpenLDAP server.<br />
=== GridFTP ===<br />
* x509<br />
* voms<br />
=== DCap/GSI-DCap ===<br />
==== Plain DCap ====<br />
* No authentication required, but needs world readable and/or writable files and directories.<br />
* Only recommended its use for prestages.<br />
==== GSI-DCap ====<br />
* voms<br />
=== XRootD ===<br />
* voms</div>Mcaubethttps://pwiki.pic.es/index.php?title=HowTo:_Configuring_gPlazma&diff=221HowTo: Configuring gPlazma2017-09-06T13:23:17Z<p>Mcaubet: /* DCap/GSI-DCap */</p>
<hr />
<div>= Configuring gPlazma =<br />
== gPlazma configuration file example ==<br />
################################################################################<br />
# Login Phases:<br />
# [auth]<br />
# - auth-plugins are used to read the users public and private<br />
# credentials and ask some authority, if those are valid for accessing <br />
# the system. <br />
# [map]<br />
# - map-plugins map the user information obtained in the auth step to UID<br />
# and GIDs. <br />
# - This may also be done in several steps (e.g., the vorolemap plugin <br />
# maps the users DN+FQAN to a username which is then mapped to UID/GIDs<br />
# by the authzdb plugin)<br />
# [account]<br />
# - account-plugins verify the validity of a possibly mapped of the user <br />
# and may reject the login depending on information gathered within the <br />
# map step.<br />
# [session]<br />
# - session plugins usually enrich the session with additional attributes<br />
# like the user’s home directory.<br />
# [identity]<br />
# - identity plugins are responsible for mapping UID and GID to user<br />
# names and vice versa during the work with dCache. <br />
################################################################################<br />
<br />
################################################################################<br />
# Modifiers:<br />
# [optional]<br />
# - The success or failure of this plugin is only important if it is the <br />
# only plugin in the stack associated with this type. <br />
# [sufficient]<br />
# - Success of such a plugin is enough to satisfy the authentication <br />
# requirements of the stack of plugins (if a prior required plugin has<br />
# failed the success of this one is ignored). <br />
# - A failure of this plugin isn't deemed as fatal for the login attempt.<br />
# - If the plugin succeeds gPlazma2 immediately proceeds with the next <br />
# plugin type or returns control to the door if this was the last stack. <br />
# [required]<br />
# - Failure of such a plugin will ultimately lead to gPlazma2 returning <br />
# failure but only after the remaining plugins for this type have been <br />
# invoked. <br />
# [requisite]<br />
# - Like required, however, in the case that such a plugin returns a<br />
# failure, control is directly returned to the d<br />
################################################################################<br />
auth sufficient htpasswd<br />
auth optional x509<br />
auth sufficient voms<br />
# send username and password to LDAP server<br />
auth optional jaas gplazma.jaas.name=LdapGplazma<br />
map optional vorolemap<br />
map sufficient authzdb<br />
map optional gridmap<br />
# convert LDAP specific result to dCache internal principals<br />
map optional mutator gplazma.mutator.accept=com.sun.security.auth.UserPrincipal gplazma.mutator.produce=username<br />
# use ldap to map user name to uid<br />
# map optional ldap<br />
map requisite nsswitch<br />
identity requisite nsswitch<br />
session sufficient nsswitch<br />
session sufficient authzdb<br />
== Authetincation types ==<br />
=== jaas ===<br />
* ''This is intended for PIC LDAP users accessing to WebDAV.''<br />
* JAAS, custom ''/etc/dcache/jgss.conf''<br />
=== htpasswd ===<br />
* ''This is intended only for the WebDAV protocol.''<br />
* Configuration file can be found here: ''/etc/dcache/htpasswd''<br />
=== x509 ===<br />
* ''This is mostly intented for the WebDAV protocol.''<br />
* Users defined in ''/etc/grid-security/grid-mapfile''.<br />
:* '''/etc/grid-security/grid-mapfile''' is generated with a ''grid-mapfile'' cron (''crontab -l'') which executes the '''edg-mkgridmap''' application.<br />
# Puppet Name: grid-mapfile<br />
0 */6 * * * [ ! -f /var/lock/subsys/edg-mkgridmap ] && /usr/sbin/edg-mkgridmap --output=- > /etc/grid-security/grid-mapfile.new && sleep 3 && cat /etc/grid-security/grid-mapfile.new > /etc/grid-security/grid-mapfile<br />
:* '''edg-mkgridmap''' application can be configured with the ''/etc/edg-mkgridmap.conf'' file where:<br />
::* You can map VO members by specifying a VOMS service and the affected VO. In example:<br />
group vomss://voms2.cern.ch:8443/voms/lhcb?/lhcb lhcb001<br />
::* '''/etc/grid-mapfile-local''' is a file to bootstrap the actual grid-mapfile and can be used to allow extra DNs or override some mappings<br />
=== voms ===<br />
* ''This is intented for GRID Users, mostly using GridFTP, GSI-DCap & SRM protocols.''<br />
* User defined in ''/etc/grid-security/grid-vorolemap''.<br />
=== multimap ===<br />
* Useful for OIDC authentication (i.e. through Google or INDIGO Accounts)<br />
<br />
== dCache Authorization by protocol ==<br />
=== WebDAV ===<br />
==== Authentication ====<br />
* htpasswd<br />
* x509<br />
==== Mapping ====<br />
* glazma Mutator (to convert LDAP specific result to dCache internal principals) + NSSwitch (to map user name to uid)<br />
=== NFSv3/NFSv41 ===<br />
==== Authentication ====<br />
* Authentication is not performed in the dCache system, is made through OpenLDAP server.<br />
==== Mapping ==== <br />
* Username/Password already authenticated through OpenLDAP, NSSwitch for mapping.<br />
=== GridFTP ===<br />
==== Authentication ====<br />
* x509<br />
* voms<br />
==== Mapping ====<br />
* ''gridmap'' must is map the incoming DN to a local user<br />
=== DCap/GSI-DCap ===<br />
==== Authentication ====<br />
===== Plain DCap =====<br />
* No authentication required, but needs world readable and/or writable files and directories.<br />
* Only recommended its use for prestages.<br />
===== GSI-DCap =====<br />
* voms<br />
==== Mapping ====<br />
* vorolemap<br />
* authzdb<br />
* gridmap<br />
<br />
=== XRootD ===<br />
==== Authentication ====<br />
==== Mapping ====</div>Mcaubethttps://pwiki.pic.es/index.php?title=HowTo:_Configuring_gPlazma&diff=220HowTo: Configuring gPlazma2017-09-06T13:21:53Z<p>Mcaubet: /* GSI-DCap */</p>
<hr />
<div>= Configuring gPlazma =<br />
== gPlazma configuration file example ==<br />
################################################################################<br />
# Login Phases:<br />
# [auth]<br />
# - auth-plugins are used to read the users public and private<br />
# credentials and ask some authority, if those are valid for accessing <br />
# the system. <br />
# [map]<br />
# - map-plugins map the user information obtained in the auth step to UID<br />
# and GIDs. <br />
# - This may also be done in several steps (e.g., the vorolemap plugin <br />
# maps the users DN+FQAN to a username which is then mapped to UID/GIDs<br />
# by the authzdb plugin)<br />
# [account]<br />
# - account-plugins verify the validity of a possibly mapped of the user <br />
# and may reject the login depending on information gathered within the <br />
# map step.<br />
# [session]<br />
# - session plugins usually enrich the session with additional attributes<br />
# like the user’s home directory.<br />
# [identity]<br />
# - identity plugins are responsible for mapping UID and GID to user<br />
# names and vice versa during the work with dCache. <br />
################################################################################<br />
<br />
################################################################################<br />
# Modifiers:<br />
# [optional]<br />
# - The success or failure of this plugin is only important if it is the <br />
# only plugin in the stack associated with this type. <br />
# [sufficient]<br />
# - Success of such a plugin is enough to satisfy the authentication <br />
# requirements of the stack of plugins (if a prior required plugin has<br />
# failed the success of this one is ignored). <br />
# - A failure of this plugin isn't deemed as fatal for the login attempt.<br />
# - If the plugin succeeds gPlazma2 immediately proceeds with the next <br />
# plugin type or returns control to the door if this was the last stack. <br />
# [required]<br />
# - Failure of such a plugin will ultimately lead to gPlazma2 returning <br />
# failure but only after the remaining plugins for this type have been <br />
# invoked. <br />
# [requisite]<br />
# - Like required, however, in the case that such a plugin returns a<br />
# failure, control is directly returned to the d<br />
################################################################################<br />
auth sufficient htpasswd<br />
auth optional x509<br />
auth sufficient voms<br />
# send username and password to LDAP server<br />
auth optional jaas gplazma.jaas.name=LdapGplazma<br />
map optional vorolemap<br />
map sufficient authzdb<br />
map optional gridmap<br />
# convert LDAP specific result to dCache internal principals<br />
map optional mutator gplazma.mutator.accept=com.sun.security.auth.UserPrincipal gplazma.mutator.produce=username<br />
# use ldap to map user name to uid<br />
# map optional ldap<br />
map requisite nsswitch<br />
identity requisite nsswitch<br />
session sufficient nsswitch<br />
session sufficient authzdb<br />
== Authetincation types ==<br />
=== jaas ===<br />
* ''This is intended for PIC LDAP users accessing to WebDAV.''<br />
* JAAS, custom ''/etc/dcache/jgss.conf''<br />
=== htpasswd ===<br />
* ''This is intended only for the WebDAV protocol.''<br />
* Configuration file can be found here: ''/etc/dcache/htpasswd''<br />
=== x509 ===<br />
* ''This is mostly intented for the WebDAV protocol.''<br />
* Users defined in ''/etc/grid-security/grid-mapfile''.<br />
:* '''/etc/grid-security/grid-mapfile''' is generated with a ''grid-mapfile'' cron (''crontab -l'') which executes the '''edg-mkgridmap''' application.<br />
# Puppet Name: grid-mapfile<br />
0 */6 * * * [ ! -f /var/lock/subsys/edg-mkgridmap ] && /usr/sbin/edg-mkgridmap --output=- > /etc/grid-security/grid-mapfile.new && sleep 3 && cat /etc/grid-security/grid-mapfile.new > /etc/grid-security/grid-mapfile<br />
:* '''edg-mkgridmap''' application can be configured with the ''/etc/edg-mkgridmap.conf'' file where:<br />
::* You can map VO members by specifying a VOMS service and the affected VO. In example:<br />
group vomss://voms2.cern.ch:8443/voms/lhcb?/lhcb lhcb001<br />
::* '''/etc/grid-mapfile-local''' is a file to bootstrap the actual grid-mapfile and can be used to allow extra DNs or override some mappings<br />
=== voms ===<br />
* ''This is intented for GRID Users, mostly using GridFTP, GSI-DCap & SRM protocols.''<br />
* User defined in ''/etc/grid-security/grid-vorolemap''.<br />
=== multimap ===<br />
* Useful for OIDC authentication (i.e. through Google or INDIGO Accounts)<br />
<br />
== dCache Authorization by protocol ==<br />
=== WebDAV ===<br />
==== Authentication ====<br />
* htpasswd<br />
* x509<br />
==== Mapping ====<br />
* glazma Mutator (to convert LDAP specific result to dCache internal principals) + NSSwitch (to map user name to uid)<br />
=== NFSv3/NFSv41 ===<br />
==== Authentication ====<br />
* Authentication is not performed in the dCache system, is made through OpenLDAP server.<br />
==== Mapping ==== <br />
* Username/Password already authenticated through OpenLDAP, NSSwitch for mapping.<br />
=== GridFTP ===<br />
==== Authentication ====<br />
* x509<br />
* voms<br />
==== Mapping ====<br />
* ''gridmap'' must is map the incoming DN to a local user<br />
=== DCap/GSI-DCap ===<br />
==== Authentication ====<br />
===== Plain DCap =====<br />
* No authentication required, but needs world readable and/or writable files and directories.<br />
* Only recommended its use for prestages.<br />
===== GSI-DCap =====<br />
* voms<br />
<br />
==== Mapping ====<br />
<br />
=== XRootD ===<br />
==== Authentication ====<br />
==== Mapping ====</div>Mcaubethttps://pwiki.pic.es/index.php?title=HowTo:_Configuring_gPlazma&diff=219HowTo: Configuring gPlazma2017-09-06T13:21:38Z<p>Mcaubet: /* GSI-DCap = */</p>
<hr />
<div>= Configuring gPlazma =<br />
== gPlazma configuration file example ==<br />
################################################################################<br />
# Login Phases:<br />
# [auth]<br />
# - auth-plugins are used to read the users public and private<br />
# credentials and ask some authority, if those are valid for accessing <br />
# the system. <br />
# [map]<br />
# - map-plugins map the user information obtained in the auth step to UID<br />
# and GIDs. <br />
# - This may also be done in several steps (e.g., the vorolemap plugin <br />
# maps the users DN+FQAN to a username which is then mapped to UID/GIDs<br />
# by the authzdb plugin)<br />
# [account]<br />
# - account-plugins verify the validity of a possibly mapped of the user <br />
# and may reject the login depending on information gathered within the <br />
# map step.<br />
# [session]<br />
# - session plugins usually enrich the session with additional attributes<br />
# like the user’s home directory.<br />
# [identity]<br />
# - identity plugins are responsible for mapping UID and GID to user<br />
# names and vice versa during the work with dCache. <br />
################################################################################<br />
<br />
################################################################################<br />
# Modifiers:<br />
# [optional]<br />
# - The success or failure of this plugin is only important if it is the <br />
# only plugin in the stack associated with this type. <br />
# [sufficient]<br />
# - Success of such a plugin is enough to satisfy the authentication <br />
# requirements of the stack of plugins (if a prior required plugin has<br />
# failed the success of this one is ignored). <br />
# - A failure of this plugin isn't deemed as fatal for the login attempt.<br />
# - If the plugin succeeds gPlazma2 immediately proceeds with the next <br />
# plugin type or returns control to the door if this was the last stack. <br />
# [required]<br />
# - Failure of such a plugin will ultimately lead to gPlazma2 returning <br />
# failure but only after the remaining plugins for this type have been <br />
# invoked. <br />
# [requisite]<br />
# - Like required, however, in the case that such a plugin returns a<br />
# failure, control is directly returned to the d<br />
################################################################################<br />
auth sufficient htpasswd<br />
auth optional x509<br />
auth sufficient voms<br />
# send username and password to LDAP server<br />
auth optional jaas gplazma.jaas.name=LdapGplazma<br />
map optional vorolemap<br />
map sufficient authzdb<br />
map optional gridmap<br />
# convert LDAP specific result to dCache internal principals<br />
map optional mutator gplazma.mutator.accept=com.sun.security.auth.UserPrincipal gplazma.mutator.produce=username<br />
# use ldap to map user name to uid<br />
# map optional ldap<br />
map requisite nsswitch<br />
identity requisite nsswitch<br />
session sufficient nsswitch<br />
session sufficient authzdb<br />
== Authetincation types ==<br />
=== jaas ===<br />
* ''This is intended for PIC LDAP users accessing to WebDAV.''<br />
* JAAS, custom ''/etc/dcache/jgss.conf''<br />
=== htpasswd ===<br />
* ''This is intended only for the WebDAV protocol.''<br />
* Configuration file can be found here: ''/etc/dcache/htpasswd''<br />
=== x509 ===<br />
* ''This is mostly intented for the WebDAV protocol.''<br />
* Users defined in ''/etc/grid-security/grid-mapfile''.<br />
:* '''/etc/grid-security/grid-mapfile''' is generated with a ''grid-mapfile'' cron (''crontab -l'') which executes the '''edg-mkgridmap''' application.<br />
# Puppet Name: grid-mapfile<br />
0 */6 * * * [ ! -f /var/lock/subsys/edg-mkgridmap ] && /usr/sbin/edg-mkgridmap --output=- > /etc/grid-security/grid-mapfile.new && sleep 3 && cat /etc/grid-security/grid-mapfile.new > /etc/grid-security/grid-mapfile<br />
:* '''edg-mkgridmap''' application can be configured with the ''/etc/edg-mkgridmap.conf'' file where:<br />
::* You can map VO members by specifying a VOMS service and the affected VO. In example:<br />
group vomss://voms2.cern.ch:8443/voms/lhcb?/lhcb lhcb001<br />
::* '''/etc/grid-mapfile-local''' is a file to bootstrap the actual grid-mapfile and can be used to allow extra DNs or override some mappings<br />
=== voms ===<br />
* ''This is intented for GRID Users, mostly using GridFTP, GSI-DCap & SRM protocols.''<br />
* User defined in ''/etc/grid-security/grid-vorolemap''.<br />
=== multimap ===<br />
* Useful for OIDC authentication (i.e. through Google or INDIGO Accounts)<br />
<br />
== dCache Authorization by protocol ==<br />
=== WebDAV ===<br />
==== Authentication ====<br />
* htpasswd<br />
* x509<br />
==== Mapping ====<br />
* glazma Mutator (to convert LDAP specific result to dCache internal principals) + NSSwitch (to map user name to uid)<br />
=== NFSv3/NFSv41 ===<br />
==== Authentication ====<br />
* Authentication is not performed in the dCache system, is made through OpenLDAP server.<br />
==== Mapping ==== <br />
* Username/Password already authenticated through OpenLDAP, NSSwitch for mapping.<br />
=== GridFTP ===<br />
==== Authentication ====<br />
* x509<br />
* voms<br />
==== Mapping ====<br />
* ''gridmap'' must is map the incoming DN to a local user<br />
=== DCap/GSI-DCap ===<br />
==== Authentication ====<br />
===== Plain DCap =====<br />
* No authentication required, but needs world readable and/or writable files and directories.<br />
* Only recommended its use for prestages.<br />
==== GSI-DCap ====<br />
* voms<br />
<br />
==== Mapping ====<br />
<br />
=== XRootD ===<br />
==== Authentication ====<br />
==== Mapping ====</div>Mcaubethttps://pwiki.pic.es/index.php?title=HowTo:_Configuring_gPlazma&diff=218HowTo: Configuring gPlazma2017-09-06T13:20:16Z<p>Mcaubet: /* DCap/GSI-DCap */</p>
<hr />
<div>= Configuring gPlazma =<br />
== gPlazma configuration file example ==<br />
################################################################################<br />
# Login Phases:<br />
# [auth]<br />
# - auth-plugins are used to read the users public and private<br />
# credentials and ask some authority, if those are valid for accessing <br />
# the system. <br />
# [map]<br />
# - map-plugins map the user information obtained in the auth step to UID<br />
# and GIDs. <br />
# - This may also be done in several steps (e.g., the vorolemap plugin <br />
# maps the users DN+FQAN to a username which is then mapped to UID/GIDs<br />
# by the authzdb plugin)<br />
# [account]<br />
# - account-plugins verify the validity of a possibly mapped of the user <br />
# and may reject the login depending on information gathered within the <br />
# map step.<br />
# [session]<br />
# - session plugins usually enrich the session with additional attributes<br />
# like the user’s home directory.<br />
# [identity]<br />
# - identity plugins are responsible for mapping UID and GID to user<br />
# names and vice versa during the work with dCache. <br />
################################################################################<br />
<br />
################################################################################<br />
# Modifiers:<br />
# [optional]<br />
# - The success or failure of this plugin is only important if it is the <br />
# only plugin in the stack associated with this type. <br />
# [sufficient]<br />
# - Success of such a plugin is enough to satisfy the authentication <br />
# requirements of the stack of plugins (if a prior required plugin has<br />
# failed the success of this one is ignored). <br />
# - A failure of this plugin isn't deemed as fatal for the login attempt.<br />
# - If the plugin succeeds gPlazma2 immediately proceeds with the next <br />
# plugin type or returns control to the door if this was the last stack. <br />
# [required]<br />
# - Failure of such a plugin will ultimately lead to gPlazma2 returning <br />
# failure but only after the remaining plugins for this type have been <br />
# invoked. <br />
# [requisite]<br />
# - Like required, however, in the case that such a plugin returns a<br />
# failure, control is directly returned to the d<br />
################################################################################<br />
auth sufficient htpasswd<br />
auth optional x509<br />
auth sufficient voms<br />
# send username and password to LDAP server<br />
auth optional jaas gplazma.jaas.name=LdapGplazma<br />
map optional vorolemap<br />
map sufficient authzdb<br />
map optional gridmap<br />
# convert LDAP specific result to dCache internal principals<br />
map optional mutator gplazma.mutator.accept=com.sun.security.auth.UserPrincipal gplazma.mutator.produce=username<br />
# use ldap to map user name to uid<br />
# map optional ldap<br />
map requisite nsswitch<br />
identity requisite nsswitch<br />
session sufficient nsswitch<br />
session sufficient authzdb<br />
== Authetincation types ==<br />
=== jaas ===<br />
* ''This is intended for PIC LDAP users accessing to WebDAV.''<br />
* JAAS, custom ''/etc/dcache/jgss.conf''<br />
=== htpasswd ===<br />
* ''This is intended only for the WebDAV protocol.''<br />
* Configuration file can be found here: ''/etc/dcache/htpasswd''<br />
=== x509 ===<br />
* ''This is mostly intented for the WebDAV protocol.''<br />
* Users defined in ''/etc/grid-security/grid-mapfile''.<br />
:* '''/etc/grid-security/grid-mapfile''' is generated with a ''grid-mapfile'' cron (''crontab -l'') which executes the '''edg-mkgridmap''' application.<br />
# Puppet Name: grid-mapfile<br />
0 */6 * * * [ ! -f /var/lock/subsys/edg-mkgridmap ] && /usr/sbin/edg-mkgridmap --output=- > /etc/grid-security/grid-mapfile.new && sleep 3 && cat /etc/grid-security/grid-mapfile.new > /etc/grid-security/grid-mapfile<br />
:* '''edg-mkgridmap''' application can be configured with the ''/etc/edg-mkgridmap.conf'' file where:<br />
::* You can map VO members by specifying a VOMS service and the affected VO. In example:<br />
group vomss://voms2.cern.ch:8443/voms/lhcb?/lhcb lhcb001<br />
::* '''/etc/grid-mapfile-local''' is a file to bootstrap the actual grid-mapfile and can be used to allow extra DNs or override some mappings<br />
=== voms ===<br />
* ''This is intented for GRID Users, mostly using GridFTP, GSI-DCap & SRM protocols.''<br />
* User defined in ''/etc/grid-security/grid-vorolemap''.<br />
=== multimap ===<br />
* Useful for OIDC authentication (i.e. through Google or INDIGO Accounts)<br />
<br />
== dCache Authorization by protocol ==<br />
=== WebDAV ===<br />
==== Authentication ====<br />
* htpasswd<br />
* x509<br />
==== Mapping ====<br />
* glazma Mutator (to convert LDAP specific result to dCache internal principals) + NSSwitch (to map user name to uid)<br />
=== NFSv3/NFSv41 ===<br />
==== Authentication ====<br />
* Authentication is not performed in the dCache system, is made through OpenLDAP server.<br />
==== Mapping ==== <br />
* Username/Password already authenticated through OpenLDAP, NSSwitch for mapping.<br />
=== GridFTP ===<br />
==== Authentication ====<br />
* x509<br />
* voms<br />
==== Mapping ====<br />
* ''gridmap'' must is map the incoming DN to a local user<br />
=== DCap/GSI-DCap ===<br />
==== Authentication ====<br />
===== Plain DCap =====<br />
* No authentication required, but needs world readable and/or writable files and directories.<br />
* Only recommended its use for prestages.<br />
==== GSI-DCap =====<br />
* voms<br />
==== Mapping ====<br />
<br />
=== XRootD ===<br />
==== Authentication ====<br />
==== Mapping ====</div>Mcaubethttps://pwiki.pic.es/index.php?title=HowTo:_Configuring_gPlazma&diff=217HowTo: Configuring gPlazma2017-09-06T13:19:47Z<p>Mcaubet: /* Authetincation types */</p>
<hr />
<div>= Configuring gPlazma =<br />
== gPlazma configuration file example ==<br />
################################################################################<br />
# Login Phases:<br />
# [auth]<br />
# - auth-plugins are used to read the users public and private<br />
# credentials and ask some authority, if those are valid for accessing <br />
# the system. <br />
# [map]<br />
# - map-plugins map the user information obtained in the auth step to UID<br />
# and GIDs. <br />
# - This may also be done in several steps (e.g., the vorolemap plugin <br />
# maps the users DN+FQAN to a username which is then mapped to UID/GIDs<br />
# by the authzdb plugin)<br />
# [account]<br />
# - account-plugins verify the validity of a possibly mapped of the user <br />
# and may reject the login depending on information gathered within the <br />
# map step.<br />
# [session]<br />
# - session plugins usually enrich the session with additional attributes<br />
# like the user’s home directory.<br />
# [identity]<br />
# - identity plugins are responsible for mapping UID and GID to user<br />
# names and vice versa during the work with dCache. <br />
################################################################################<br />
<br />
################################################################################<br />
# Modifiers:<br />
# [optional]<br />
# - The success or failure of this plugin is only important if it is the <br />
# only plugin in the stack associated with this type. <br />
# [sufficient]<br />
# - Success of such a plugin is enough to satisfy the authentication <br />
# requirements of the stack of plugins (if a prior required plugin has<br />
# failed the success of this one is ignored). <br />
# - A failure of this plugin isn't deemed as fatal for the login attempt.<br />
# - If the plugin succeeds gPlazma2 immediately proceeds with the next <br />
# plugin type or returns control to the door if this was the last stack. <br />
# [required]<br />
# - Failure of such a plugin will ultimately lead to gPlazma2 returning <br />
# failure but only after the remaining plugins for this type have been <br />
# invoked. <br />
# [requisite]<br />
# - Like required, however, in the case that such a plugin returns a<br />
# failure, control is directly returned to the d<br />
################################################################################<br />
auth sufficient htpasswd<br />
auth optional x509<br />
auth sufficient voms<br />
# send username and password to LDAP server<br />
auth optional jaas gplazma.jaas.name=LdapGplazma<br />
map optional vorolemap<br />
map sufficient authzdb<br />
map optional gridmap<br />
# convert LDAP specific result to dCache internal principals<br />
map optional mutator gplazma.mutator.accept=com.sun.security.auth.UserPrincipal gplazma.mutator.produce=username<br />
# use ldap to map user name to uid<br />
# map optional ldap<br />
map requisite nsswitch<br />
identity requisite nsswitch<br />
session sufficient nsswitch<br />
session sufficient authzdb<br />
== Authetincation types ==<br />
=== jaas ===<br />
* ''This is intended for PIC LDAP users accessing to WebDAV.''<br />
* JAAS, custom ''/etc/dcache/jgss.conf''<br />
=== htpasswd ===<br />
* ''This is intended only for the WebDAV protocol.''<br />
* Configuration file can be found here: ''/etc/dcache/htpasswd''<br />
=== x509 ===<br />
* ''This is mostly intented for the WebDAV protocol.''<br />
* Users defined in ''/etc/grid-security/grid-mapfile''.<br />
:* '''/etc/grid-security/grid-mapfile''' is generated with a ''grid-mapfile'' cron (''crontab -l'') which executes the '''edg-mkgridmap''' application.<br />
# Puppet Name: grid-mapfile<br />
0 */6 * * * [ ! -f /var/lock/subsys/edg-mkgridmap ] && /usr/sbin/edg-mkgridmap --output=- > /etc/grid-security/grid-mapfile.new && sleep 3 && cat /etc/grid-security/grid-mapfile.new > /etc/grid-security/grid-mapfile<br />
:* '''edg-mkgridmap''' application can be configured with the ''/etc/edg-mkgridmap.conf'' file where:<br />
::* You can map VO members by specifying a VOMS service and the affected VO. In example:<br />
group vomss://voms2.cern.ch:8443/voms/lhcb?/lhcb lhcb001<br />
::* '''/etc/grid-mapfile-local''' is a file to bootstrap the actual grid-mapfile and can be used to allow extra DNs or override some mappings<br />
=== voms ===<br />
* ''This is intented for GRID Users, mostly using GridFTP, GSI-DCap & SRM protocols.''<br />
* User defined in ''/etc/grid-security/grid-vorolemap''.<br />
=== multimap ===<br />
* Useful for OIDC authentication (i.e. through Google or INDIGO Accounts)<br />
<br />
== dCache Authorization by protocol ==<br />
=== WebDAV ===<br />
==== Authentication ====<br />
* htpasswd<br />
* x509<br />
==== Mapping ====<br />
* glazma Mutator (to convert LDAP specific result to dCache internal principals) + NSSwitch (to map user name to uid)<br />
=== NFSv3/NFSv41 ===<br />
==== Authentication ====<br />
* Authentication is not performed in the dCache system, is made through OpenLDAP server.<br />
==== Mapping ==== <br />
* Username/Password already authenticated through OpenLDAP, NSSwitch for mapping.<br />
=== GridFTP ===<br />
==== Authentication ====<br />
* x509<br />
* voms<br />
==== Mapping ====<br />
* ''gridmap'' must is map the incoming DN to a local user<br />
=== DCap/GSI-DCap ===<br />
==== Authentication ====<br />
===== Plain DCap =====<br />
* No authentication required, but needs world readable and/or writable files and directories.<br />
* Only recommended its use for prestages.<br />
===== GSI-DCap ====<br />
* voms<br />
==== Mapping ====<br />
<br />
=== XRootD ===<br />
==== Authentication ====<br />
==== Mapping ====</div>Mcaubethttps://pwiki.pic.es/index.php?title=HowTo:_Configuring_gPlazma&diff=216HowTo: Configuring gPlazma2017-09-06T13:19:26Z<p>Mcaubet: /* dCache Authorization by protocol */</p>
<hr />
<div>= Configuring gPlazma =<br />
== gPlazma configuration file example ==<br />
################################################################################<br />
# Login Phases:<br />
# [auth]<br />
# - auth-plugins are used to read the users public and private<br />
# credentials and ask some authority, if those are valid for accessing <br />
# the system. <br />
# [map]<br />
# - map-plugins map the user information obtained in the auth step to UID<br />
# and GIDs. <br />
# - This may also be done in several steps (e.g., the vorolemap plugin <br />
# maps the users DN+FQAN to a username which is then mapped to UID/GIDs<br />
# by the authzdb plugin)<br />
# [account]<br />
# - account-plugins verify the validity of a possibly mapped of the user <br />
# and may reject the login depending on information gathered within the <br />
# map step.<br />
# [session]<br />
# - session plugins usually enrich the session with additional attributes<br />
# like the user’s home directory.<br />
# [identity]<br />
# - identity plugins are responsible for mapping UID and GID to user<br />
# names and vice versa during the work with dCache. <br />
################################################################################<br />
<br />
################################################################################<br />
# Modifiers:<br />
# [optional]<br />
# - The success or failure of this plugin is only important if it is the <br />
# only plugin in the stack associated with this type. <br />
# [sufficient]<br />
# - Success of such a plugin is enough to satisfy the authentication <br />
# requirements of the stack of plugins (if a prior required plugin has<br />
# failed the success of this one is ignored). <br />
# - A failure of this plugin isn't deemed as fatal for the login attempt.<br />
# - If the plugin succeeds gPlazma2 immediately proceeds with the next <br />
# plugin type or returns control to the door if this was the last stack. <br />
# [required]<br />
# - Failure of such a plugin will ultimately lead to gPlazma2 returning <br />
# failure but only after the remaining plugins for this type have been <br />
# invoked. <br />
# [requisite]<br />
# - Like required, however, in the case that such a plugin returns a<br />
# failure, control is directly returned to the d<br />
################################################################################<br />
auth sufficient htpasswd<br />
auth optional x509<br />
auth sufficient voms<br />
# send username and password to LDAP server<br />
auth optional jaas gplazma.jaas.name=LdapGplazma<br />
map optional vorolemap<br />
map sufficient authzdb<br />
map optional gridmap<br />
# convert LDAP specific result to dCache internal principals<br />
map optional mutator gplazma.mutator.accept=com.sun.security.auth.UserPrincipal gplazma.mutator.produce=username<br />
# use ldap to map user name to uid<br />
# map optional ldap<br />
map requisite nsswitch<br />
identity requisite nsswitch<br />
session sufficient nsswitch<br />
session sufficient authzdb<br />
== Authetincation types ==<br />
=== jaas ===<br />
* ''This is intended for PIC LDAP users accessing to WebDAV.''<br />
* JAAS, custom ''/etc/dcache/jgss.conf''<br />
===== htpasswd =====<br />
* ''This is intended only for the WebDAV protocol.''<br />
* Configuration file can be found here: ''/etc/dcache/htpasswd''<br />
=== x509 ===<br />
* ''This is mostly intented for the WebDAV protocol.''<br />
* Users defined in ''/etc/grid-security/grid-mapfile''.<br />
:* '''/etc/grid-security/grid-mapfile''' is generated with a ''grid-mapfile'' cron (''crontab -l'') which executes the '''edg-mkgridmap''' application.<br />
# Puppet Name: grid-mapfile<br />
0 */6 * * * [ ! -f /var/lock/subsys/edg-mkgridmap ] && /usr/sbin/edg-mkgridmap --output=- > /etc/grid-security/grid-mapfile.new && sleep 3 && cat /etc/grid-security/grid-mapfile.new > /etc/grid-security/grid-mapfile<br />
:* '''edg-mkgridmap''' application can be configured with the ''/etc/edg-mkgridmap.conf'' file where:<br />
::* You can map VO members by specifying a VOMS service and the affected VO. In example:<br />
group vomss://voms2.cern.ch:8443/voms/lhcb?/lhcb lhcb001<br />
::* '''/etc/grid-mapfile-local''' is a file to bootstrap the actual grid-mapfile and can be used to allow extra DNs or override some mappings<br />
===== voms =====<br />
* ''This is intented for GRID Users, mostly using GridFTP, GSI-DCap & SRM protocols.''<br />
* User defined in ''/etc/grid-security/grid-vorolemap''.<br />
=== multimap ===<br />
* Useful for OIDC authentication (i.e. through Google or INDIGO Accounts)<br />
<br />
== dCache Authorization by protocol ==<br />
=== WebDAV ===<br />
==== Authentication ====<br />
* htpasswd<br />
* x509<br />
==== Mapping ====<br />
* glazma Mutator (to convert LDAP specific result to dCache internal principals) + NSSwitch (to map user name to uid)<br />
=== NFSv3/NFSv41 ===<br />
==== Authentication ====<br />
* Authentication is not performed in the dCache system, is made through OpenLDAP server.<br />
==== Mapping ==== <br />
* Username/Password already authenticated through OpenLDAP, NSSwitch for mapping.<br />
=== GridFTP ===<br />
==== Authentication ====<br />
* x509<br />
* voms<br />
==== Mapping ====<br />
* ''gridmap'' must is map the incoming DN to a local user<br />
=== DCap/GSI-DCap ===<br />
==== Authentication ====<br />
===== Plain DCap =====<br />
* No authentication required, but needs world readable and/or writable files and directories.<br />
* Only recommended its use for prestages.<br />
===== GSI-DCap ====<br />
* voms<br />
==== Mapping ====<br />
<br />
=== XRootD ===<br />
==== Authentication ====<br />
==== Mapping ====</div>Mcaubethttps://pwiki.pic.es/index.php?title=HowTo:_Configuring_gPlazma&diff=215HowTo: Configuring gPlazma2017-09-06T13:13:50Z<p>Mcaubet: /* Authentication */</p>
<hr />
<div>= Configuring gPlazma =<br />
== gPlazma configuration file example ==<br />
################################################################################<br />
# Login Phases:<br />
# [auth]<br />
# - auth-plugins are used to read the users public and private<br />
# credentials and ask some authority, if those are valid for accessing <br />
# the system. <br />
# [map]<br />
# - map-plugins map the user information obtained in the auth step to UID<br />
# and GIDs. <br />
# - This may also be done in several steps (e.g., the vorolemap plugin <br />
# maps the users DN+FQAN to a username which is then mapped to UID/GIDs<br />
# by the authzdb plugin)<br />
# [account]<br />
# - account-plugins verify the validity of a possibly mapped of the user <br />
# and may reject the login depending on information gathered within the <br />
# map step.<br />
# [session]<br />
# - session plugins usually enrich the session with additional attributes<br />
# like the user’s home directory.<br />
# [identity]<br />
# - identity plugins are responsible for mapping UID and GID to user<br />
# names and vice versa during the work with dCache. <br />
################################################################################<br />
<br />
################################################################################<br />
# Modifiers:<br />
# [optional]<br />
# - The success or failure of this plugin is only important if it is the <br />
# only plugin in the stack associated with this type. <br />
# [sufficient]<br />
# - Success of such a plugin is enough to satisfy the authentication <br />
# requirements of the stack of plugins (if a prior required plugin has<br />
# failed the success of this one is ignored). <br />
# - A failure of this plugin isn't deemed as fatal for the login attempt.<br />
# - If the plugin succeeds gPlazma2 immediately proceeds with the next <br />
# plugin type or returns control to the door if this was the last stack. <br />
# [required]<br />
# - Failure of such a plugin will ultimately lead to gPlazma2 returning <br />
# failure but only after the remaining plugins for this type have been <br />
# invoked. <br />
# [requisite]<br />
# - Like required, however, in the case that such a plugin returns a<br />
# failure, control is directly returned to the d<br />
################################################################################<br />
auth sufficient htpasswd<br />
auth optional x509<br />
auth sufficient voms<br />
# send username and password to LDAP server<br />
auth optional jaas gplazma.jaas.name=LdapGplazma<br />
map optional vorolemap<br />
map sufficient authzdb<br />
map optional gridmap<br />
# convert LDAP specific result to dCache internal principals<br />
map optional mutator gplazma.mutator.accept=com.sun.security.auth.UserPrincipal gplazma.mutator.produce=username<br />
# use ldap to map user name to uid<br />
# map optional ldap<br />
map requisite nsswitch<br />
identity requisite nsswitch<br />
session sufficient nsswitch<br />
session sufficient authzdb<br />
== dCache Authorization by protocol ==<br />
=== WebDAV ===<br />
* htpasswd<br />
==== Authentication ====<br />
===== jaas =====<br />
* JAAS, custom ''/etc/dcache/jgss.conf''<br />
===== htpasswd =====<br />
* Configuration file can be found here: ''/etc/dcache/htpasswd''<br />
===== x509 =====<br />
<br />
==== Mapping ====<br />
* glazma Mutator (to convert LDAP specific result to dCache internal principals) + NSSwitch (to map user name to uid)<br />
=== NFSv3/NFSv41 ===<br />
==== Authentication ====<br />
* Authentication is not performed in the dCache system, is made through OpenLDAP server.<br />
<br />
==== Mapping ==== <br />
* Username/Password already authenticated through OpenLDAP, NSSwitch for mapping.<br />
=== GridFTP ===<br />
==== Authentication ====<br />
===== x509 =====<br />
* Users defined in ''/etc/grid-security/grid-mapfile''.<br />
:* '''/etc/grid-security/grid-mapfile''' is generated with a ''grid-mapfile'' cron (''crontab -l'') which executes the '''edg-mkgridmap''' application.<br />
# Puppet Name: grid-mapfile<br />
0 */6 * * * [ ! -f /var/lock/subsys/edg-mkgridmap ] && /usr/sbin/edg-mkgridmap --output=- > /etc/grid-security/grid-mapfile.new && sleep 3 && cat /etc/grid-security/grid-mapfile.new > /etc/grid-security/grid-mapfile<br />
:* '''edg-mkgridmap''' application can be configured with the ''/etc/edg-mkgridmap.conf'' file where:<br />
::* You can map VO members by specifying a VOMS service and the affected VO. In example:<br />
group vomss://voms2.cern.ch:8443/voms/lhcb?/lhcb lhcb001<br />
::* '''/etc/grid-mapfile-local''' is a file to bootstrap the actual grid-mapfile and can be used to allow extra DNs or override some mappings<br />
===== voms =====<br />
* User defined in ''/etc/grid-security/grid-vorolemap''.<br />
==== Mapping ====<br />
* ''gridmap'' must is map the incoming DN to a local user<br />
=== DCap/GSI-DCap ===<br />
==== Authentication ====<br />
===== Plain DCap =====<br />
* No authentication required, but needs world readable and/or writable files and directories.<br />
* Only recommended its use for prestages.<br />
===== GSI-DCap ====<br />
* <br />
==== Mapping ====<br />
<br />
=== XRootD ===<br />
<br />
=== HTPasswd ===<br />
* This is intended only for the WebDAV protocol.<br />
=== x509 Certificates ===<br />
* This is mostly intented for the WebDAV protocol.<br />
=== VOMS Certificates ===<br />
* This is intented for GRID Users, mostly using GridFTP, GSI-DCap & SRM protocols.<br />
=== JAAS ===<br />
* This is intended for PIC LDAP users accessing to WebDAV.<br />
=== multimap ===<br />
* Useful for OIDC authentication (i.e. through Google or INDIGO Accounts)</div>Mcaubethttps://pwiki.pic.es/index.php?title=HowTo:_Configuring_gPlazma&diff=214HowTo: Configuring gPlazma2017-09-06T13:10:57Z<p>Mcaubet: /* Authentication */</p>
<hr />
<div>= Configuring gPlazma =<br />
== gPlazma configuration file example ==<br />
################################################################################<br />
# Login Phases:<br />
# [auth]<br />
# - auth-plugins are used to read the users public and private<br />
# credentials and ask some authority, if those are valid for accessing <br />
# the system. <br />
# [map]<br />
# - map-plugins map the user information obtained in the auth step to UID<br />
# and GIDs. <br />
# - This may also be done in several steps (e.g., the vorolemap plugin <br />
# maps the users DN+FQAN to a username which is then mapped to UID/GIDs<br />
# by the authzdb plugin)<br />
# [account]<br />
# - account-plugins verify the validity of a possibly mapped of the user <br />
# and may reject the login depending on information gathered within the <br />
# map step.<br />
# [session]<br />
# - session plugins usually enrich the session with additional attributes<br />
# like the user’s home directory.<br />
# [identity]<br />
# - identity plugins are responsible for mapping UID and GID to user<br />
# names and vice versa during the work with dCache. <br />
################################################################################<br />
<br />
################################################################################<br />
# Modifiers:<br />
# [optional]<br />
# - The success or failure of this plugin is only important if it is the <br />
# only plugin in the stack associated with this type. <br />
# [sufficient]<br />
# - Success of such a plugin is enough to satisfy the authentication <br />
# requirements of the stack of plugins (if a prior required plugin has<br />
# failed the success of this one is ignored). <br />
# - A failure of this plugin isn't deemed as fatal for the login attempt.<br />
# - If the plugin succeeds gPlazma2 immediately proceeds with the next <br />
# plugin type or returns control to the door if this was the last stack. <br />
# [required]<br />
# - Failure of such a plugin will ultimately lead to gPlazma2 returning <br />
# failure but only after the remaining plugins for this type have been <br />
# invoked. <br />
# [requisite]<br />
# - Like required, however, in the case that such a plugin returns a<br />
# failure, control is directly returned to the d<br />
################################################################################<br />
auth sufficient htpasswd<br />
auth optional x509<br />
auth sufficient voms<br />
# send username and password to LDAP server<br />
auth optional jaas gplazma.jaas.name=LdapGplazma<br />
map optional vorolemap<br />
map sufficient authzdb<br />
map optional gridmap<br />
# convert LDAP specific result to dCache internal principals<br />
map optional mutator gplazma.mutator.accept=com.sun.security.auth.UserPrincipal gplazma.mutator.produce=username<br />
# use ldap to map user name to uid<br />
# map optional ldap<br />
map requisite nsswitch<br />
identity requisite nsswitch<br />
session sufficient nsswitch<br />
session sufficient authzdb<br />
== dCache Authorization by protocol ==<br />
=== WebDAV ===<br />
* htpasswd<br />
==== Authentication ====<br />
===== jaas =====<br />
* JAAS, custom ''/etc/dcache/jgss.conf''<br />
===== htpasswd =====<br />
* Configuration file can be found here: ''/etc/dcache/htpasswd''<br />
<br />
==== Mapping ====<br />
* glazma Mutator (to convert LDAP specific result to dCache internal principals) + NSSwitch (to map user name to uid)<br />
=== NFSv3/NFSv41 ===<br />
==== Authentication ====<br />
* Authentication is not performed in the dCache system, is made through OpenLDAP server.<br />
<br />
==== Mapping ==== <br />
* Username/Password already authenticated through OpenLDAP, NSSwitch for mapping.<br />
=== GridFTP ===<br />
==== Authentication ====<br />
===== x509 =====<br />
* Users defined in ''/etc/grid-security/grid-mapfile''.<br />
:* '''/etc/grid-security/grid-mapfile''' is generated with a ''grid-mapfile'' cron (''crontab -l'') which executes the '''edg-mkgridmap''' application.<br />
# Puppet Name: grid-mapfile<br />
0 */6 * * * [ ! -f /var/lock/subsys/edg-mkgridmap ] && /usr/sbin/edg-mkgridmap --output=- > /etc/grid-security/grid-mapfile.new && sleep 3 && cat /etc/grid-security/grid-mapfile.new > /etc/grid-security/grid-mapfile<br />
:* '''edg-mkgridmap''' application can be configured with the ''/etc/edg-mkgridmap.conf'' file where:<br />
::* You can map VO members by specifying a VOMS service and the affected VO. In example:<br />
group vomss://voms2.cern.ch:8443/voms/lhcb?/lhcb lhcb001<br />
::* '''/etc/grid-mapfile-local''' is a file to bootstrap the actual grid-mapfile and can be used to allow extra DNs or override some mappings<br />
===== voms =====<br />
* User defined in ''/etc/grid-security/grid-vorolemap''.<br />
==== Mapping ====<br />
* ''gridmap'' must is map the incoming DN to a local user<br />
=== DCap/GSI-DCap ===<br />
==== Authentication ====<br />
===== Plain DCap =====<br />
* No authentication required, but needs world readable and/or writable files and directories.<br />
* Only recommended its use for prestages.<br />
===== GSI-DCap ====<br />
* <br />
==== Mapping ====<br />
<br />
=== XRootD ===<br />
<br />
=== HTPasswd ===<br />
* This is intended only for the WebDAV protocol.<br />
=== x509 Certificates ===<br />
* This is mostly intented for the WebDAV protocol.<br />
=== VOMS Certificates ===<br />
* This is intented for GRID Users, mostly using GridFTP, GSI-DCap & SRM protocols.<br />
=== JAAS ===<br />
* This is intended for PIC LDAP users accessing to WebDAV.<br />
=== multimap ===<br />
* Useful for OIDC authentication (i.e. through Google or INDIGO Accounts)</div>Mcaubethttps://pwiki.pic.es/index.php?title=HowTo:_Configuring_gPlazma&diff=213HowTo: Configuring gPlazma2017-09-06T13:09:18Z<p>Mcaubet: /* DCap/GSI-DCap */</p>
<hr />
<div>= Configuring gPlazma =<br />
== gPlazma configuration file example ==<br />
################################################################################<br />
# Login Phases:<br />
# [auth]<br />
# - auth-plugins are used to read the users public and private<br />
# credentials and ask some authority, if those are valid for accessing <br />
# the system. <br />
# [map]<br />
# - map-plugins map the user information obtained in the auth step to UID<br />
# and GIDs. <br />
# - This may also be done in several steps (e.g., the vorolemap plugin <br />
# maps the users DN+FQAN to a username which is then mapped to UID/GIDs<br />
# by the authzdb plugin)<br />
# [account]<br />
# - account-plugins verify the validity of a possibly mapped of the user <br />
# and may reject the login depending on information gathered within the <br />
# map step.<br />
# [session]<br />
# - session plugins usually enrich the session with additional attributes<br />
# like the user’s home directory.<br />
# [identity]<br />
# - identity plugins are responsible for mapping UID and GID to user<br />
# names and vice versa during the work with dCache. <br />
################################################################################<br />
<br />
################################################################################<br />
# Modifiers:<br />
# [optional]<br />
# - The success or failure of this plugin is only important if it is the <br />
# only plugin in the stack associated with this type. <br />
# [sufficient]<br />
# - Success of such a plugin is enough to satisfy the authentication <br />
# requirements of the stack of plugins (if a prior required plugin has<br />
# failed the success of this one is ignored). <br />
# - A failure of this plugin isn't deemed as fatal for the login attempt.<br />
# - If the plugin succeeds gPlazma2 immediately proceeds with the next <br />
# plugin type or returns control to the door if this was the last stack. <br />
# [required]<br />
# - Failure of such a plugin will ultimately lead to gPlazma2 returning <br />
# failure but only after the remaining plugins for this type have been <br />
# invoked. <br />
# [requisite]<br />
# - Like required, however, in the case that such a plugin returns a<br />
# failure, control is directly returned to the d<br />
################################################################################<br />
auth sufficient htpasswd<br />
auth optional x509<br />
auth sufficient voms<br />
# send username and password to LDAP server<br />
auth optional jaas gplazma.jaas.name=LdapGplazma<br />
map optional vorolemap<br />
map sufficient authzdb<br />
map optional gridmap<br />
# convert LDAP specific result to dCache internal principals<br />
map optional mutator gplazma.mutator.accept=com.sun.security.auth.UserPrincipal gplazma.mutator.produce=username<br />
# use ldap to map user name to uid<br />
# map optional ldap<br />
map requisite nsswitch<br />
identity requisite nsswitch<br />
session sufficient nsswitch<br />
session sufficient authzdb<br />
== dCache Authorization by protocol ==<br />
=== WebDAV ===<br />
* htpasswd<br />
==== Authentication ====<br />
* JAAS, custom ''/etc/dcache/jgss.conf''<br />
<br />
==== Mapping ====<br />
* glazma Mutator (to convert LDAP specific result to dCache internal principals) + NSSwitch (to map user name to uid)<br />
=== NFSv3/NFSv41 ===<br />
==== Authentication ====<br />
* Authentication is not performed in the dCache system, is made through OpenLDAP server.<br />
<br />
==== Mapping ==== <br />
* Username/Password already authenticated through OpenLDAP, NSSwitch for mapping.<br />
=== GridFTP ===<br />
==== Authentication ====<br />
===== x509 =====<br />
* Users defined in ''/etc/grid-security/grid-mapfile''.<br />
:* '''/etc/grid-security/grid-mapfile''' is generated with a ''grid-mapfile'' cron (''crontab -l'') which executes the '''edg-mkgridmap''' application.<br />
# Puppet Name: grid-mapfile<br />
0 */6 * * * [ ! -f /var/lock/subsys/edg-mkgridmap ] && /usr/sbin/edg-mkgridmap --output=- > /etc/grid-security/grid-mapfile.new && sleep 3 && cat /etc/grid-security/grid-mapfile.new > /etc/grid-security/grid-mapfile<br />
:* '''edg-mkgridmap''' application can be configured with the ''/etc/edg-mkgridmap.conf'' file where:<br />
::* You can map VO members by specifying a VOMS service and the affected VO. In example:<br />
group vomss://voms2.cern.ch:8443/voms/lhcb?/lhcb lhcb001<br />
::* '''/etc/grid-mapfile-local''' is a file to bootstrap the actual grid-mapfile and can be used to allow extra DNs or override some mappings<br />
===== voms =====<br />
* User defined in ''/etc/grid-security/grid-vorolemap''.<br />
==== Mapping ====<br />
* ''gridmap'' must is map the incoming DN to a local user<br />
=== DCap/GSI-DCap ===<br />
==== Authentication ====<br />
===== Plain DCap =====<br />
* No authentication required, but needs world readable and/or writable files and directories.<br />
* Only recommended its use for prestages.<br />
===== GSI-DCap ====<br />
* <br />
==== Mapping ====<br />
<br />
=== XRootD ===<br />
<br />
=== HTPasswd ===<br />
* This is intended only for the WebDAV protocol.<br />
=== x509 Certificates ===<br />
* This is mostly intented for the WebDAV protocol.<br />
=== VOMS Certificates ===<br />
* This is intented for GRID Users, mostly using GridFTP, GSI-DCap & SRM protocols.<br />
=== JAAS ===<br />
* This is intended for PIC LDAP users accessing to WebDAV.<br />
=== multimap ===<br />
* Useful for OIDC authentication (i.e. through Google or INDIGO Accounts)</div>Mcaubethttps://pwiki.pic.es/index.php?title=HowTo:_Configuring_gPlazma&diff=212HowTo: Configuring gPlazma2017-09-06T13:02:06Z<p>Mcaubet: /* Configuring gPlazma */</p>
<hr />
<div>= Configuring gPlazma =<br />
== gPlazma configuration file example ==<br />
################################################################################<br />
# Login Phases:<br />
# [auth]<br />
# - auth-plugins are used to read the users public and private<br />
# credentials and ask some authority, if those are valid for accessing <br />
# the system. <br />
# [map]<br />
# - map-plugins map the user information obtained in the auth step to UID<br />
# and GIDs. <br />
# - This may also be done in several steps (e.g., the vorolemap plugin <br />
# maps the users DN+FQAN to a username which is then mapped to UID/GIDs<br />
# by the authzdb plugin)<br />
# [account]<br />
# - account-plugins verify the validity of a possibly mapped of the user <br />
# and may reject the login depending on information gathered within the <br />
# map step.<br />
# [session]<br />
# - session plugins usually enrich the session with additional attributes<br />
# like the user’s home directory.<br />
# [identity]<br />
# - identity plugins are responsible for mapping UID and GID to user<br />
# names and vice versa during the work with dCache. <br />
################################################################################<br />
<br />
################################################################################<br />
# Modifiers:<br />
# [optional]<br />
# - The success or failure of this plugin is only important if it is the <br />
# only plugin in the stack associated with this type. <br />
# [sufficient]<br />
# - Success of such a plugin is enough to satisfy the authentication <br />
# requirements of the stack of plugins (if a prior required plugin has<br />
# failed the success of this one is ignored). <br />
# - A failure of this plugin isn't deemed as fatal for the login attempt.<br />
# - If the plugin succeeds gPlazma2 immediately proceeds with the next <br />
# plugin type or returns control to the door if this was the last stack. <br />
# [required]<br />
# - Failure of such a plugin will ultimately lead to gPlazma2 returning <br />
# failure but only after the remaining plugins for this type have been <br />
# invoked. <br />
# [requisite]<br />
# - Like required, however, in the case that such a plugin returns a<br />
# failure, control is directly returned to the d<br />
################################################################################<br />
auth sufficient htpasswd<br />
auth optional x509<br />
auth sufficient voms<br />
# send username and password to LDAP server<br />
auth optional jaas gplazma.jaas.name=LdapGplazma<br />
map optional vorolemap<br />
map sufficient authzdb<br />
map optional gridmap<br />
# convert LDAP specific result to dCache internal principals<br />
map optional mutator gplazma.mutator.accept=com.sun.security.auth.UserPrincipal gplazma.mutator.produce=username<br />
# use ldap to map user name to uid<br />
# map optional ldap<br />
map requisite nsswitch<br />
identity requisite nsswitch<br />
session sufficient nsswitch<br />
session sufficient authzdb<br />
== dCache Authorization by protocol ==<br />
=== WebDAV ===<br />
* htpasswd<br />
==== Authentication ====<br />
* JAAS, custom ''/etc/dcache/jgss.conf''<br />
<br />
==== Mapping ====<br />
* glazma Mutator (to convert LDAP specific result to dCache internal principals) + NSSwitch (to map user name to uid)<br />
=== NFSv3/NFSv41 ===<br />
==== Authentication ====<br />
* Authentication is not performed in the dCache system, is made through OpenLDAP server.<br />
<br />
==== Mapping ==== <br />
* Username/Password already authenticated through OpenLDAP, NSSwitch for mapping.<br />
=== GridFTP ===<br />
==== Authentication ====<br />
===== x509 =====<br />
* Users defined in ''/etc/grid-security/grid-mapfile''.<br />
:* '''/etc/grid-security/grid-mapfile''' is generated with a ''grid-mapfile'' cron (''crontab -l'') which executes the '''edg-mkgridmap''' application.<br />
# Puppet Name: grid-mapfile<br />
0 */6 * * * [ ! -f /var/lock/subsys/edg-mkgridmap ] && /usr/sbin/edg-mkgridmap --output=- > /etc/grid-security/grid-mapfile.new && sleep 3 && cat /etc/grid-security/grid-mapfile.new > /etc/grid-security/grid-mapfile<br />
:* '''edg-mkgridmap''' application can be configured with the ''/etc/edg-mkgridmap.conf'' file where:<br />
::* You can map VO members by specifying a VOMS service and the affected VO. In example:<br />
group vomss://voms2.cern.ch:8443/voms/lhcb?/lhcb lhcb001<br />
::* '''/etc/grid-mapfile-local''' is a file to bootstrap the actual grid-mapfile and can be used to allow extra DNs or override some mappings<br />
===== voms =====<br />
* User defined in ''/etc/grid-security/grid-vorolemap''.<br />
==== Mapping ====<br />
* ''gridmap'' must is map the incoming DN to a local user<br />
=== DCap/GSI-DCap ===<br />
:'''Authentication:''' Plain DCap with no authentication, but requires world readable and/or writable. GSI-DCap<br />
:'''Mapping:''' glazma Mutator (to convert LDAP specific result to dCache internal principals) + NSSwitch (to map user name to uid)<br />
=== XRootD ===<br />
<br />
=== HTPasswd ===<br />
* This is intended only for the WebDAV protocol.<br />
=== x509 Certificates ===<br />
* This is mostly intented for the WebDAV protocol.<br />
=== VOMS Certificates ===<br />
* This is intented for GRID Users, mostly using GridFTP, GSI-DCap & SRM protocols.<br />
=== JAAS ===<br />
* This is intended for PIC LDAP users accessing to WebDAV.<br />
=== multimap ===<br />
* Useful for OIDC authentication (i.e. through Google or INDIGO Accounts)</div>Mcaubethttps://pwiki.pic.es/index.php?title=HowTo:_Configuring_gPlazma&diff=211HowTo: Configuring gPlazma2017-09-06T13:00:25Z<p>Mcaubet: /* Authentication */</p>
<hr />
<div>= Configuring gPlazma =<br />
== dCache Authorization by protocol ==<br />
=== WebDAV ===<br />
* htpasswd<br />
==== Authentication ====<br />
* JAAS, custom ''/etc/dcache/jgss.conf''<br />
<br />
==== Mapping ====<br />
* glazma Mutator (to convert LDAP specific result to dCache internal principals) + NSSwitch (to map user name to uid)<br />
=== NFSv3/NFSv41 ===<br />
==== Authentication ====<br />
* Authentication is not performed in the dCache system, is made through OpenLDAP server.<br />
<br />
==== Mapping ==== <br />
* Username/Password already authenticated through OpenLDAP, NSSwitch for mapping.<br />
=== GridFTP ===<br />
==== Authentication ====<br />
===== x509 =====<br />
* Users defined in ''/etc/grid-security/grid-mapfile''.<br />
:* '''/etc/grid-security/grid-mapfile''' is generated with a ''grid-mapfile'' cron (''crontab -l'') which executes the '''edg-mkgridmap''' application.<br />
# Puppet Name: grid-mapfile<br />
0 */6 * * * [ ! -f /var/lock/subsys/edg-mkgridmap ] && /usr/sbin/edg-mkgridmap --output=- > /etc/grid-security/grid-mapfile.new && sleep 3 && cat /etc/grid-security/grid-mapfile.new > /etc/grid-security/grid-mapfile<br />
:* '''edg-mkgridmap''' application can be configured with the ''/etc/edg-mkgridmap.conf'' file where:<br />
::* You can map VO members by specifying a VOMS service and the affected VO. In example:<br />
group vomss://voms2.cern.ch:8443/voms/lhcb?/lhcb lhcb001<br />
::* '''/etc/grid-mapfile-local''' is a file to bootstrap the actual grid-mapfile and can be used to allow extra DNs or override some mappings<br />
===== voms =====<br />
* User defined in ''/etc/grid-security/grid-vorolemap''.<br />
==== Mapping ====<br />
* ''gridmap'' must is map the incoming DN to a local user<br />
=== DCap/GSI-DCap ===<br />
:'''Authentication:''' Plain DCap with no authentication, but requires world readable and/or writable. GSI-DCap<br />
:'''Mapping:''' glazma Mutator (to convert LDAP specific result to dCache internal principals) + NSSwitch (to map user name to uid)<br />
=== XRootD ===<br />
<br />
=== HTPasswd ===<br />
* This is intended only for the WebDAV protocol.<br />
=== x509 Certificates ===<br />
* This is mostly intented for the WebDAV protocol.<br />
=== VOMS Certificates ===<br />
* This is intented for GRID Users, mostly using GridFTP, GSI-DCap & SRM protocols.<br />
=== JAAS ===<br />
* This is intended for PIC LDAP users accessing to WebDAV.<br />
=== multimap ===<br />
* Useful for OIDC authentication (i.e. through Google or INDIGO Accounts)</div>Mcaubethttps://pwiki.pic.es/index.php?title=HowTo:_Configuring_gPlazma&diff=210HowTo: Configuring gPlazma2017-09-06T13:00:00Z<p>Mcaubet: /* Authentication */</p>
<hr />
<div>= Configuring gPlazma =<br />
== dCache Authorization by protocol ==<br />
=== WebDAV ===<br />
* htpasswd<br />
==== Authentication ====<br />
* JAAS, custom ''/etc/dcache/jgss.conf''<br />
==== Mapping ====<br />
* glazma Mutator (to convert LDAP specific result to dCache internal principals) + NSSwitch (to map user name to uid)<br />
=== NFSv3/NFSv41 ===<br />
==== Authentication ====<br />
* Authentication is not performed in the dCache system, is made through OpenLDAP server.<br />
<br />
==== Mapping ==== <br />
* Username/Password already authenticated through OpenLDAP, NSSwitch for mapping.<br />
=== GridFTP ===<br />
==== Authentication ====<br />
===== x509 =====<br />
* Users defined in ''/etc/grid-security/grid-mapfile''.<br />
:* '''/etc/grid-security/grid-mapfile''' is generated with a ''grid-mapfile'' cron (''crontab -l'') which executes the '''edg-mkgridmap''' application.<br />
# Puppet Name: grid-mapfile<br />
0 */6 * * * [ ! -f /var/lock/subsys/edg-mkgridmap ] && /usr/sbin/edg-mkgridmap --output=- > /etc/grid-security/grid-mapfile.new && sleep 3 && cat /etc/grid-security/grid-mapfile.new > /etc/grid-security/grid-mapfile<br />
:* '''edg-mkgridmap''' application can be configured with the ''/etc/edg-mkgridmap.conf'' file where:<br />
::* You can map VO members by specifying a VOMS service and the affected VO. In example:<br />
group vomss://voms2.cern.ch:8443/voms/lhcb?/lhcb lhcb001<br />
::* '''/etc/grid-mapfile-local''' is a file to bootstrap the actual grid-mapfile and can be used to allow extra DNs or override some mappings<br />
===== voms =====<br />
* User defined in ''/etc/grid-security/grid-vorolemap''.<br />
==== Mapping ====<br />
* ''gridmap'' must is map the incoming DN to a local user<br />
=== DCap/GSI-DCap ===<br />
:'''Authentication:''' Plain DCap with no authentication, but requires world readable and/or writable. GSI-DCap<br />
:'''Mapping:''' glazma Mutator (to convert LDAP specific result to dCache internal principals) + NSSwitch (to map user name to uid)<br />
=== XRootD ===<br />
<br />
=== HTPasswd ===<br />
* This is intended only for the WebDAV protocol.<br />
=== x509 Certificates ===<br />
* This is mostly intented for the WebDAV protocol.<br />
=== VOMS Certificates ===<br />
* This is intented for GRID Users, mostly using GridFTP, GSI-DCap & SRM protocols.<br />
=== JAAS ===<br />
* This is intended for PIC LDAP users accessing to WebDAV.<br />
=== multimap ===<br />
* Useful for OIDC authentication (i.e. through Google or INDIGO Accounts)</div>Mcaubethttps://pwiki.pic.es/index.php?title=HowTo:_Configuring_gPlazma&diff=209HowTo: Configuring gPlazma2017-09-06T12:59:06Z<p>Mcaubet: </p>
<hr />
<div>= Configuring gPlazma =<br />
== dCache Authorization by protocol ==<br />
=== WebDAV ===<br />
* htpasswd<br />
==== Authentication ====<br />
* JAAS, custom ''/etc/dcache/jgss.conf''<br />
==== Mapping ====<br />
* glazma Mutator (to convert LDAP specific result to dCache internal principals) + NSSwitch (to map user name to uid)<br />
=== NFSv3/NFSv41 ===<br />
==== Authentication ====<br />
* Not performed in the dCache system, is made through OpenLDAP server.<br />
==== Mapping ==== <br />
* Username/Password already authenticated through OpenLDAP, NSSwitch for mapping.<br />
=== GridFTP ===<br />
==== Authentication ====<br />
===== x509 =====<br />
* Users defined in ''/etc/grid-security/grid-mapfile''.<br />
:* '''/etc/grid-security/grid-mapfile''' is generated with a ''grid-mapfile'' cron (''crontab -l'') which executes the '''edg-mkgridmap''' application.<br />
# Puppet Name: grid-mapfile<br />
0 */6 * * * [ ! -f /var/lock/subsys/edg-mkgridmap ] && /usr/sbin/edg-mkgridmap --output=- > /etc/grid-security/grid-mapfile.new && sleep 3 && cat /etc/grid-security/grid-mapfile.new > /etc/grid-security/grid-mapfile<br />
:* '''edg-mkgridmap''' application can be configured with the ''/etc/edg-mkgridmap.conf'' file where:<br />
::* You can map VO members by specifying a VOMS service and the affected VO. In example:<br />
group vomss://voms2.cern.ch:8443/voms/lhcb?/lhcb lhcb001<br />
::* '''/etc/grid-mapfile-local''' is a file to bootstrap the actual grid-mapfile and can be used to allow extra DNs or override some mappings<br />
===== voms =====<br />
* User defined in ''/etc/grid-security/grid-vorolemap''.<br />
==== Mapping ====<br />
* ''gridmap'' must is map the incoming DN to a local user<br />
=== DCap/GSI-DCap ===<br />
:'''Authentication:''' Plain DCap with no authentication, but requires world readable and/or writable. GSI-DCap<br />
:'''Mapping:''' glazma Mutator (to convert LDAP specific result to dCache internal principals) + NSSwitch (to map user name to uid)<br />
=== XRootD ===<br />
<br />
=== HTPasswd ===<br />
* This is intended only for the WebDAV protocol.<br />
=== x509 Certificates ===<br />
* This is mostly intented for the WebDAV protocol.<br />
=== VOMS Certificates ===<br />
* This is intented for GRID Users, mostly using GridFTP, GSI-DCap & SRM protocols.<br />
=== JAAS ===<br />
* This is intended for PIC LDAP users accessing to WebDAV.<br />
=== multimap ===<br />
* Useful for OIDC authentication (i.e. through Google or INDIGO Accounts)</div>Mcaubethttps://pwiki.pic.es/index.php?title=HowTo:_Configuring_gPlazma&diff=208HowTo: Configuring gPlazma2017-09-06T07:12:41Z<p>Mcaubet: /* dCache Authorization by protocol */</p>
<hr />
<div>= Configuring gPlazma =<br />
== dCache Authorization by protocol ==<br />
=== WebDAV ===<br />
* htpasswd<br />
:'''Authentication:''' JAAS, custom ''/etc/dcache/jgss.conf''<br />
:'''Mapping:''' glazma Mutator (to convert LDAP specific result to dCache internal principals) + NSSwitch (to map user name to uid)<br />
=== NFSv3/NFSv41 ===<br />
* '''Authentication:''' Not performed in the dCache system, is made through OpenLDAP server.<br />
* '''Mapping:''' Username/Password already authenticated through OpenLDAP, NSSwitch for mapping.<br />
=== GridFTP ===<br />
* '''Authentication:''' users configured in ''/etc/grid-security/grid-mapfile''<br />
:* '''/etc/grid-security/grid-mapfile''' is generated with a ''grid-mapfile'' cron (''crontab -l'') which executes the '''edg-mkgridmap''' application.<br />
# Puppet Name: grid-mapfile<br />
0 */6 * * * [ ! -f /var/lock/subsys/edg-mkgridmap ] && /usr/sbin/edg-mkgridmap --output=- > /etc/grid-security/grid-mapfile.new && sleep 3 && cat /etc/grid-security/grid-mapfile.new > /etc/grid-security/grid-mapfile<br />
:* '''edg-mkgridmap''' application can be configured with the ''/etc/edg-mkgridmap.conf'' file where:<br />
::* You can map VO members by specifying a VOMS service and the affected VO. In example:<br />
group vomss://voms2.cern.ch:8443/voms/lhcb?/lhcb lhcb001<br />
::* '''/etc/grid-mapfile-local''' is a file to bootstrap the actual grid-mapfile and can be used to allow extra DNs or override some mappings<br />
* '''Mapping:''' ''gridmap'' must is map the incoming DN to a local user<br />
=== DCap/GSI-DCap ===<br />
=== XRootD ===<br />
<br />
=== HTPasswd ===<br />
* This is intended only for the WebDAV protocol.<br />
=== x509 Certificates ===<br />
* This is mostly intented for the WebDAV protocol.<br />
=== VOMS Certificates ===<br />
* This is intented for GRID Users, mostly using GridFTP, GSI-DCap & SRM protocols.<br />
=== JAAS ===<br />
* This is intended for PIC LDAP users accessing to WebDAV.<br />
=== multimap ===<br />
* Useful for OIDC authentication (i.e. through Google or INDIGO Accounts)</div>Mcaubethttps://pwiki.pic.es/index.php?title=HowTo:_Configuring_gPlazma&diff=207HowTo: Configuring gPlazma2017-09-06T06:43:46Z<p>Mcaubet: /* multimap */</p>
<hr />
<div>= Configuring gPlazma =<br />
== dCache Authorization by protocol ==<br />
=== WebDAV ===<br />
* htpasswd<br />
:'''Authentication:''' JAAS, custom ''/etc/dcache/jgss.conf''<br />
:'''Mapping:''' glazma Mutator (to convert LDAP specific result to dCache internal principals) + NSSwitch (to map user name to uid)<br />
=== NFSv3/NFSv41 ===<br />
* '''Authentication:''' Not performed in the dCache system, is made through OpenLDAP server.<br />
* '''Mapping:''' sername/Password already authenticated through OpenLDAP, NSSwitch for mapping.<br />
=== GridFTP ===<br />
=== DCap/GSI-DCap ===<br />
=== XRootD ===<br />
<br />
=== HTPasswd ===<br />
* This is intended only for the WebDAV protocol.<br />
=== x509 Certificates ===<br />
* This is mostly intented for the WebDAV protocol.<br />
=== VOMS Certificates ===<br />
* This is intented for GRID Users, mostly using GridFTP, GSI-DCap & SRM protocols.<br />
=== JAAS ===<br />
* This is intended for PIC LDAP users accessing to WebDAV.<br />
=== multimap ===<br />
* Useful for OIDC authentication (i.e. through Google or INDIGO Accounts)</div>Mcaubethttps://pwiki.pic.es/index.php?title=HowTo:_Configuring_gPlazma&diff=206HowTo: Configuring gPlazma2017-09-06T06:42:51Z<p>Mcaubet: /* = */</p>
<hr />
<div>= Configuring gPlazma =<br />
== dCache Authorization by protocol ==<br />
=== WebDAV ===<br />
* htpasswd<br />
:'''Authentication:''' JAAS, custom ''/etc/dcache/jgss.conf''<br />
:'''Mapping:''' glazma Mutator (to convert LDAP specific result to dCache internal principals) + NSSwitch (to map user name to uid)<br />
=== NFSv3/NFSv41 ===<br />
* '''Authentication:''' Not performed in the dCache system, is made through OpenLDAP server.<br />
* '''Mapping:''' sername/Password already authenticated through OpenLDAP, NSSwitch for mapping.<br />
=== GridFTP ===<br />
=== DCap/GSI-DCap ===<br />
=== XRootD ===<br />
<br />
=== HTPasswd ===<br />
* This is intended only for the WebDAV protocol.<br />
=== x509 Certificates ===<br />
* This is mostly intented for the WebDAV protocol.<br />
=== VOMS Certificates ===<br />
* This is intented for GRID Users, mostly using GridFTP, GSI-DCap & SRM protocols.<br />
=== JAAS ===<br />
* This is intended for PIC LDAP users accessing to WebDAV.<br />
=== multimap ===</div>Mcaubethttps://pwiki.pic.es/index.php?title=DCache.org&diff=205DCache.org2017-09-06T06:42:07Z<p>Mcaubet: /* Development instance */</p>
<hr />
<div>= Become a dCache Administrator at PIC =<br />
* [[HowTo: dCache Management - GIT + Puppet]]<br />
== Management tools ==<br />
* [[HowTo: Accessing to the dCache Admin Console]]<br />
* [[HowTo: Installing & Configuring PCells to Administrate dCache]]<br />
== Instances ==<br />
<br />
=== Test instance ===<br />
* '''''Version''''': this can be higher than the ''production'' instance.<br />
* '''''Needed for''''': test newer dCache releases. <br />
* '''''When to upgrade''''': <br />
:# Whenever you want, whenever you need.<br />
:# Recommended to test with Enstore before upgrading the Development instance<br />
:# Recommended to backup DBs for rolling back upgrades.<br />
* '''''Upgrade requirements''''':<br />
:# No requirements.<br />
<br />
=== Development instance ===<br />
* '''''Version''''': should be always the same as the ''production'' instance<br />
* '''''Needed for''''': this instance is used in order to test Enstore with the dCache version running in production.<br />
* '''''When to upgrade''''': <br />
:# Always upgrade before upgrading the dCache production instance.<br />
:# Upgrade only as the last validation for the dCache upgrade procedure, just before upgrading the production instance.<br />
:# Only upgrade when no Enstore interventions / changes are expected until the production instance is upgraded.<br />
* '''''Upgrade requirements''''':<br />
:# Always ask to the Enstore admin if upgrades can be performed in the the Development instance.<br />
:# Ensure that there are always DB backups before upgrading the instance.<br />
<br />
=== Production instance ===<br />
* '''''Version''''': Any Golden or Production release.<br />
* '''''Needed for''''': production services.<br />
* '''''When to upgrade''''': <br />
:# Always upgrade after upgrading and validating the upgrade for the dCache development instance.<br />
:# Upgrade always ensuring that Enstore client and service was already tested in the test and development instances.<br />
* '''''Upgrade requirements''''':<br />
:# Always ask to the Enstore admin if upgrade can be performed in the the Production instance.<br />
:# Ensure that there are always DB backups before upgrading the instance.<br />
<br />
=== CMS MWR (MiddleWare Readiness) instance ===<br />
* '''''Version''''': Latest dCache version.<br />
* '''''Needed for''''': CMS MRW tests (Debug)<br />
* '''''When to upgrade''''': <br />
:# Whenever a new release is available is recommended to upgrade. However, there is no a strict requirement and you can wait few days / weeks before upgrading.<br />
* '''''Upgrade requirements''''':<br />
:# For major release upgrades, always inform the CMS Contact and CMS MWR team.<br />
:# Ensure that there are always DB backups before upgrading the instance.<br />
<br />
= Service Administration =<br />
<br />
== gPlazma ==<br />
* [[HowTo: Configuring gPlazma]]<br />
<br />
== Pools Management ==<br />
* Data Migration:<br />
:* [[HowTo: Migrate dCache Pool]]<br />
:* [[HowTo: Permanent Migration to dCache Read Only Pools]]<br />
* [[HowTo: srm22 - Generating poolmanager.conf & LinkGroupAuthorization.conf]]<br />
:* [[HowTo:_srm22_-_Generating_poolmanager.conf_%26_LinkGroupAuthorization.conf#Adding_.2F_Moving_.2F_Removing_pools_from_poolgroups|HowTo: Adding/Removing pools to a VO]]<br />
<br />
== dCacheAdmin ==<br />
* [[HowTo: Configure SSH2 access to the Admin Console]]<br />
<br />
== SRM ==<br />
* [[HowTo: Managing Space Reservation (Tokens)]]<br />
<br />
== Billing ==<br />
* [[HowTo: Reset Billing PostgreSQL]]<br />
* [[HowTo: Clean old Billing PostgreSQL records]]<br />
<br />
== Info Provider ==<br />
* [[HowTo: Publishing dCache Information]]<br />
<br />
== Alarms ==<br />
<br />
== dCache + Enstore Administration ==<br />
* [[HowTo: Managing Tape File Families in dCache]]<br />
<br />
== Doors ==<br />
=== NFSv3/NFSv41 Service in dCache ===<br />
[[HowTo: Exporting a new dCache NFS directory]]<br />
=== WebDAV ===<br />
* [[HowTo: WebDAV properties]]:<br />
:* [[HowTo:_WebDAV_properties#Example:_File_Checksum_via_WebDAV | File Checksum check via WebDAV]]<br />
=== CMS XRootD ===<br />
* [[HowTo: CMS XRootD configuration]]<br />
<br />
== dCache Advanced Administration ==<br />
* [[dCache Metadata Commands]]<br />
* [[dCache Administration Console Commands]]<br />
* [[Monitoring dCache with ELK]]<br />
<br />
== Alarms ==<br />
* [[DCACHE_CHECK_POOL_BTREE_MEMORY]]<br />
* [[DCACHE_CHECK_POOL_METADATA]]</div>Mcaubethttps://pwiki.pic.es/index.php?title=DCache.org&diff=204DCache.org2017-09-06T06:41:15Z<p>Mcaubet: /* Development instance */</p>
<hr />
<div>= Become a dCache Administrator at PIC =<br />
* [[HowTo: dCache Management - GIT + Puppet]]<br />
== Management tools ==<br />
* [[HowTo: Accessing to the dCache Admin Console]]<br />
* [[HowTo: Installing & Configuring PCells to Administrate dCache]]<br />
== Instances ==<br />
<br />
=== Test instance ===<br />
* '''''Version''''': this can be higher than the ''production'' instance.<br />
* '''''Needed for''''': test newer dCache releases. <br />
* '''''When to upgrade''''': <br />
:# Whenever you want, whenever you need.<br />
:# Recommended to test with Enstore before upgrading the Development instance<br />
:# Recommended to backup DBs for rolling back upgrades.<br />
* '''''Upgrade requirements''''':<br />
:# No requirements.<br />
<br />
=== Development instance ===<br />
* '''''Version''''': should be always the same as the ''production'' instance<br />
* '''''Needed for''''': this instance is used in order to test Enstore with the dCache version running in production.<br />
* '''''When to upgrade''''': <br />
:# Always upgrade before upgrading the dCache production instance.<br />
:# Upgrade only as the last validation for the dCache upgrade procedure, just before upgrading the production instance.<br />
:# Only upgrade when no Enstore interventions / changes are expected until the production instance is upgraded.<br />
* '''''Upgrade requirements''''':<br />
:# Always ask to the Enstore admin if upgrade can be performed in the the Development instance.<br />
:# Ensure that there are always DB backups before upgrading the instance.<br />
<br />
=== Production instance ===<br />
* '''''Version''''': Any Golden or Production release.<br />
* '''''Needed for''''': production services.<br />
* '''''When to upgrade''''': <br />
:# Always upgrade after upgrading and validating the upgrade for the dCache development instance.<br />
:# Upgrade always ensuring that Enstore client and service was already tested in the test and development instances.<br />
* '''''Upgrade requirements''''':<br />
:# Always ask to the Enstore admin if upgrade can be performed in the the Production instance.<br />
:# Ensure that there are always DB backups before upgrading the instance.<br />
<br />
=== CMS MWR (MiddleWare Readiness) instance ===<br />
* '''''Version''''': Latest dCache version.<br />
* '''''Needed for''''': CMS MRW tests (Debug)<br />
* '''''When to upgrade''''': <br />
:# Whenever a new release is available is recommended to upgrade. However, there is no a strict requirement and you can wait few days / weeks before upgrading.<br />
* '''''Upgrade requirements''''':<br />
:# For major release upgrades, always inform the CMS Contact and CMS MWR team.<br />
:# Ensure that there are always DB backups before upgrading the instance.<br />
<br />
= Service Administration =<br />
<br />
== gPlazma ==<br />
* [[HowTo: Configuring gPlazma]]<br />
<br />
== Pools Management ==<br />
* Data Migration:<br />
:* [[HowTo: Migrate dCache Pool]]<br />
:* [[HowTo: Permanent Migration to dCache Read Only Pools]]<br />
* [[HowTo: srm22 - Generating poolmanager.conf & LinkGroupAuthorization.conf]]<br />
:* [[HowTo:_srm22_-_Generating_poolmanager.conf_%26_LinkGroupAuthorization.conf#Adding_.2F_Moving_.2F_Removing_pools_from_poolgroups|HowTo: Adding/Removing pools to a VO]]<br />
<br />
== dCacheAdmin ==<br />
* [[HowTo: Configure SSH2 access to the Admin Console]]<br />
<br />
== SRM ==<br />
* [[HowTo: Managing Space Reservation (Tokens)]]<br />
<br />
== Billing ==<br />
* [[HowTo: Reset Billing PostgreSQL]]<br />
* [[HowTo: Clean old Billing PostgreSQL records]]<br />
<br />
== Info Provider ==<br />
* [[HowTo: Publishing dCache Information]]<br />
<br />
== Alarms ==<br />
<br />
== dCache + Enstore Administration ==<br />
* [[HowTo: Managing Tape File Families in dCache]]<br />
<br />
== Doors ==<br />
=== NFSv3/NFSv41 Service in dCache ===<br />
[[HowTo: Exporting a new dCache NFS directory]]<br />
=== WebDAV ===<br />
* [[HowTo: WebDAV properties]]:<br />
:* [[HowTo:_WebDAV_properties#Example:_File_Checksum_via_WebDAV | File Checksum check via WebDAV]]<br />
=== CMS XRootD ===<br />
* [[HowTo: CMS XRootD configuration]]<br />
<br />
== dCache Advanced Administration ==<br />
* [[dCache Metadata Commands]]<br />
* [[dCache Administration Console Commands]]<br />
* [[Monitoring dCache with ELK]]<br />
<br />
== Alarms ==<br />
* [[DCACHE_CHECK_POOL_BTREE_MEMORY]]<br />
* [[DCACHE_CHECK_POOL_METADATA]]</div>Mcaubethttps://pwiki.pic.es/index.php?title=DCache.org&diff=203DCache.org2017-09-05T14:47:10Z<p>Mcaubet: /* Become a dCache Administrator */</p>
<hr />
<div>= Become a dCache Administrator at PIC =<br />
* [[HowTo: dCache Management - GIT + Puppet]]<br />
== Management tools ==<br />
* [[HowTo: Accessing to the dCache Admin Console]]<br />
* [[HowTo: Installing & Configuring PCells to Administrate dCache]]<br />
== Instances ==<br />
<br />
=== Test instance ===<br />
* '''''Version''''': this can be higher than the ''production'' instance.<br />
* '''''Needed for''''': test newer dCache releases. <br />
* '''''When to upgrade''''': <br />
:# Whenever you want, whenever you need.<br />
:# Recommended to test with Enstore before upgrading the Development instance<br />
:# Recommended to backup DBs for rolling back upgrades.<br />
* '''''Upgrade requirements''''':<br />
:# No requirements.<br />
<br />
=== Development instance ===<br />
* '''''Version''''': should be always the same as the ''production'' instance<br />
* '''''Needed for''''': this instance is used in order to test Enstore with the dCache version which is running in production.<br />
* '''''When to upgrade''''': <br />
:# Always upgrade before upgrading the dCache production instance.<br />
:# Upgrade only as the last validation for the dCache upgrade procedure, just before upgrading the production instance.<br />
:# Only upgrade when no Enstore interventions / changes are expected until the production instance is upgraded.<br />
* '''''Upgrade requirements''''':<br />
:# Always ask to the Enstore admin if upgrade can be performed in the the Development instance.<br />
:# Ensure that there are always DB backups before upgrading the instance.<br />
<br />
=== Production instance ===<br />
* '''''Version''''': Any Golden or Production release.<br />
* '''''Needed for''''': production services.<br />
* '''''When to upgrade''''': <br />
:# Always upgrade after upgrading and validating the upgrade for the dCache development instance.<br />
:# Upgrade always ensuring that Enstore client and service was already tested in the test and development instances.<br />
* '''''Upgrade requirements''''':<br />
:# Always ask to the Enstore admin if upgrade can be performed in the the Production instance.<br />
:# Ensure that there are always DB backups before upgrading the instance.<br />
<br />
=== CMS MWR (MiddleWare Readiness) instance ===<br />
* '''''Version''''': Latest dCache version.<br />
* '''''Needed for''''': CMS MRW tests (Debug)<br />
* '''''When to upgrade''''': <br />
:# Whenever a new release is available is recommended to upgrade. However, there is no a strict requirement and you can wait few days / weeks before upgrading.<br />
* '''''Upgrade requirements''''':<br />
:# For major release upgrades, always inform the CMS Contact and CMS MWR team.<br />
:# Ensure that there are always DB backups before upgrading the instance.<br />
<br />
= Service Administration =<br />
<br />
== gPlazma ==<br />
* [[HowTo: Configuring gPlazma]]<br />
<br />
== Pools Management ==<br />
* Data Migration:<br />
:* [[HowTo: Migrate dCache Pool]]<br />
:* [[HowTo: Permanent Migration to dCache Read Only Pools]]<br />
* [[HowTo: srm22 - Generating poolmanager.conf & LinkGroupAuthorization.conf]]<br />
:* [[HowTo:_srm22_-_Generating_poolmanager.conf_%26_LinkGroupAuthorization.conf#Adding_.2F_Moving_.2F_Removing_pools_from_poolgroups|HowTo: Adding/Removing pools to a VO]]<br />
<br />
== dCacheAdmin ==<br />
* [[HowTo: Configure SSH2 access to the Admin Console]]<br />
<br />
== SRM ==<br />
* [[HowTo: Managing Space Reservation (Tokens)]]<br />
<br />
== Billing ==<br />
* [[HowTo: Reset Billing PostgreSQL]]<br />
* [[HowTo: Clean old Billing PostgreSQL records]]<br />
<br />
== Info Provider ==<br />
* [[HowTo: Publishing dCache Information]]<br />
<br />
== Alarms ==<br />
<br />
== dCache + Enstore Administration ==<br />
* [[HowTo: Managing Tape File Families in dCache]]<br />
<br />
== Doors ==<br />
=== NFSv3/NFSv41 Service in dCache ===<br />
[[HowTo: Exporting a new dCache NFS directory]]<br />
=== WebDAV ===<br />
* [[HowTo: WebDAV properties]]:<br />
:* [[HowTo:_WebDAV_properties#Example:_File_Checksum_via_WebDAV | File Checksum check via WebDAV]]<br />
=== CMS XRootD ===<br />
* [[HowTo: CMS XRootD configuration]]<br />
<br />
== dCache Advanced Administration ==<br />
* [[dCache Metadata Commands]]<br />
* [[dCache Administration Console Commands]]<br />
* [[Monitoring dCache with ELK]]<br />
<br />
== Alarms ==<br />
* [[DCACHE_CHECK_POOL_BTREE_MEMORY]]<br />
* [[DCACHE_CHECK_POOL_METADATA]]</div>Mcaubethttps://pwiki.pic.es/index.php?title=DCache_Administration_Console_Commands&diff=202DCache Administration Console Commands2017-07-31T10:02:56Z<p>Mcaubet: /* File / replica management examples */</p>
<hr />
<div>= About this document =<br />
* This document shows several examples of useful commands that can be run in the dCache Administration Console.<br />
* Administration console can be accessed through '''[[https://www.dcache.org/downloads/gui/ PCells GUI]]''' or through '''SSHv2''' as follows (in the example, '''dccore.pic.es''' is the physical machine running the '''admin''' cell):<br />
ssh -2 -x -o StrictHostKeyChecking=no -l admin -c blowfish -p 22224 dccore.pic.es<br />
* In this document we show what we consider the most important commands for the everyday's administration tasks, and basically the ones that may be useful for the Manager On Duty. Maybe some important commands are missing, but will be added as soon as we detect that these may be important for the MoD.<br />
* This document is divided in two main blocks:<br />
# '''Pool commands''': shows useful commands that can be run on pools.<br />
# '''Service commands''': show useful commands that can be run on servers (this will include doors)<br />
= The basics =<br />
* If you access the Administration Console through '''SSHv2''' you should access to the different cells as follows:<br />
\? [command]... # display help for shell commands<br />
\c cell[@domain] [user] # connect to cell<br />
\exception [trace] # controls display of stack traces<br />
\h [command]... # display help for cell commands<br />
\l [cell[@domain]|pool/poolgroup]... # list cells<br />
\q # quit<br />
'''\s (cell[@domain]|pool/poolgroup)[,(cell[@domain]|pool/poolgroup)]... command... # send command'''<br />
\sl pnfsid|path command... # send to locations<br />
\sn command... # send pnfsmanager command<br />
\sp command... # send poolmanager command<br />
\timeout [seconds] # sets the command timeout<br />
* With '''\s''' you will be able to access to any '''cell[@domain]'''s, while branches of this command such like '''\sn''' or '''\sp''' are useful to (respectively) direct access to the ''namespace'' or ''poolmanager'' cells without the need of specifying the ''cell[@domain]''.<br />
<br />
= Pool commands =<br />
== Introduction ==<br />
* Any command run on a pool is a temporary change: on any pool service restart configuration will be restored back to the default value or to the value set in the '''setup''' file (locate in '''/dcpool*/vpool*/setup''').<br />
:* In example, whenever we set a pool to ''rdonly'', on restart, once the ''initialization'' process finishes, pool will switch back to ''enabled''.<br />
* If you want to make permanent changes, you should '''save''' changes. Running the '''save''' command will update the already mentioned '''setup''' file.<br />
:* In example, whenever we set a pool to ''rdonly'' '''and''' we ''save'' the configuration, on a restart, once the ''initialization'' process finishes, pool will remain ''rdonly''.<br />
<br />
== Disable / enable pools ==<br />
=== Command help ===<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? help hsm'''<br />
''pool disable [options] [<errorCode> [<errorMessage>]]'' # suspend sending 'up messages'<br />
OPTIONS :<br />
-fetch # disallows fetch (transfer to client)<br />
-stage # disallows staging (from HSM)<br />
-store # disallows store (transfer from client)<br />
-p2p-client<br />
-rdonly # := store,stage,p2p-client<br />
-strict # := disallows everything<br />
''pool enable'' # start sending 'up messages' again<br />
''pool suppress hsmload on|off''<br />
<br />
=== Disable / enable pool examples ===<br />
* Disallow store + stage + p2p-client (set pool in read only mode)<br />
'''''[dccore01] (local) admin >''''' '''pool disable -rdonly'''<br />
* Enable pool (set pool to RW)<br />
'''''[dccore01] (local) admin >''''' '''pool enable'''<br />
<br />
== HSM configuration ==<br />
=== Command help ===<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? help hsm'''<br />
hsm create <type> [<name> [<provider>]] [-<key>=<value>] ...<br />
hsm ls [<name>] ...<br />
hsm remove <name><br />
hsm set <name> [-<key>=<value>] ...<br />
hsm show providers # show available nearline storage providers<br />
hsm unset <name> [-<key>] ...<br />
=== HSM configuration examples ===<br />
* Create a ''enstore'' HSM with the ''real-encp.sh'' client script (this example can be found in the '''setup''' file):<br />
'''''[dccore01] (local) admin >''''' '''hsm create enstore enstore script -pnfs=/pnfs/fs/usr -command=/opt/enstore/bin/real-encp.sh -c:gets=200 -c:puts=20 -c:removes=2'''<br />
* Show ''enstore'' configuration:<br />
'''''[dccore01] (local) admin >''''' '''hsm ls enstore'''<br />
enstore(enstore):script<br />
pnfs /pnfs/fs/usr<br />
command /opt/enstore/bin/real-encp.sh<br />
c:gets 200<br />
c:puts 20<br />
c:removes 2<br />
* Set maximum restores to 400:<br />
'''''[dccore01] (local) admin >''''' '''hsm set enstore -c:gets=400'''<br />
<br />
== Mover management ==<br />
=== Command help ===<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? help hsm'''<br />
''mover kill <jobid>'' # terminate a file transfer connection<br />
''mover ls [-binary [jobId] ]''<br />
''mover queue ls [-l]'' # list all mover queues in this pool<br />
''mover remove [<jobid>]'' # #OBSOLETE command<br />
''mover set max active [OPTIONS] <maxactivemovers>'' # set the maximum number of active client transfers<br />
<br />
=== Mover management examples ===<br />
* Show current '''ftp''' transfers:<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? mover ls -l -queue=ftp'''<br />
33743014 : RUNNING : 0000D5B91029797545859181EB738C25E93C IoMode=READ h={SU=0;SA=0;S=None} bytes=12113543168 time/sec=3893 LM=0 si={vo-cms.cms}<br />
33743019 : RUNNING : 00008862CFF7A1C74513A617D364A4162DAA IoMode=READ h={SU=0;SA=0;S=None} bytes=10561388544 time/sec=3409 LM=0 si={vo-cms.cms}<br />
33743076 : RUNNING : 0000736CFC0DAF494816BFE7CC32D3192900 IoMode=READ h={SU=0;SA=0;S=None} bytes=2034630656 time/sec=2478 LM=0 si={vo-cms.cms}<br />
33743086 : RUNNING : 0000C5F1B9D6EEC04DEEA077CA6753E30DE0 IoMode=READ h={SU=0;SA=0;S=None} bytes=488898560 time/sec=1357 LM=1 si={vo-cms.cms}<br />
33743103 : RUNNING : 0000BBE684CB4BA44AB5ACB0D13C5DD25C8C IoMode=READ h={SU=0;SA=0;S=None} bytes=135921664 time/sec=239 LM=20 si={vo-cms.cms}<br />
* Show all mover queue properties:<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? mover queue ls -l'''<br />
regular 0 0 0<br />
p2p 0 40 0<br />
ftp 11 500 0<br />
dcap 0 10 0<br />
nfs4 0 100 0<br />
xrootd 406 1500 0<br />
xrootd-wan 0 700 0<br />
webdav 0 0 0<br />
* Set to 400 the maximum active client transfers for the '''ftp''' queue:<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? mover set max active -queue=ftp 400'''<br />
Max Active Io Movers set to 400<br />
<br />
== File / replica management ==<br />
=== Command help ===<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? help hsm'''<br />
''rep ls [-l[=s,l,u,nc,p]] [-s[=kmgt]] [-storage=<glob>] | <pnfsId>...''<br />
Format I : <pnfsId>...<br />
Format II : [-l[=<selectionOptions>]] [-storage=<glob>] [-s]<br />
Options :<br />
-l[=splunc] # selected list<br />
s : sticky files<br />
p : precious files<br />
l : locked files<br />
u : files in use<br />
nc : files which are not cached<br />
e : files which error condition<br />
-storage=<glob> # select only replicas<br />
of files with storage-<br />
info that matches <glob><br />
-s[=kmgt] [-sum] # statistics<br />
k : data amount in KBytes<br />
m : data amount in MBytes<br />
g : data amount in GBytes<br />
t : data amount in TBytes<br />
Output is a list of repository entries, one per line<br />
each line has the followin syntax:<br />
<pnfsid> <state> <size> <storageinfo><br />
state is a sequence of state bits inclosed in angular "<>" brackets <br />
bit 1 is "C" if entry is cached or "-" if not <br />
bit 2 is "P" if entry is precious or "-" if not <br />
bit 3 is "C" if entry is being transfered "from client" or "-" if not <br />
bit 4 is "S" if entry is being transfered "from store" or "-" if not <br />
bit 5 is "c" if entry is being transfered "to client" or "-" if not <br />
bit 6 is "s" if entry is being transfered "to store" or "-" if not <br />
bit 7 is "R" if entry is removed or "-" if not <br />
bit 8 is is always "-" <br />
bit 9 is "X" if entry is sticky or "-" if not <br />
bit 10 is "E" if entry is in error state or "-" if not <br />
bit 11 is "L(x)(y)" if entry is in locked or "-" if not <br />
x is epoch until which the entry is locked, 0 for non expiring lock <br />
y is the link count<br />
''rep rm <pnfsid> [-force]'' # removes the pnfsfile from the cache<br />
''rep rmclass <storageClass>'' # removes the from the cache<br />
''rep set broken <pnfsid>''<br />
''rep set cached <pnfsId>'' # DON'T USE, Potentially dangerous<br />
''rep set precious <pnfsId>''<br />
''rep set sticky [OPTIONS] [<pnfsid>] on|off'' # change sticky flags<br />
''rep sticky ls'' <pnfsid><br />
=== File / replica management examples ===<br />
* Show all ''precious'' replicas:<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? rep ls -l=p'''<br />
00000F754EACA5F343A593669020CA14B742 <-P---------L(0)[1]> 2964602212 si={vo-cms.t1d0_Tau_data_raw}<br />
000094BC5C56FD6A4299A80A9A2D17972F0A <-P---------L(0)[0]> 4008986357 si={vo-cms.t1d0_Tau_data_raw}<br />
...<br />
* Delete replica ('''DO NOT delete non-cached-only files''')<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? rep rm <pnfsid>'''</div>Mcaubethttps://pwiki.pic.es/index.php?title=DCache_Administration_Console_Commands&diff=201DCache Administration Console Commands2017-07-31T09:40:13Z<p>Mcaubet: /* File / replica management */</p>
<hr />
<div>= About this document =<br />
* This document shows several examples of useful commands that can be run in the dCache Administration Console.<br />
* Administration console can be accessed through '''[[https://www.dcache.org/downloads/gui/ PCells GUI]]''' or through '''SSHv2''' as follows (in the example, '''dccore.pic.es''' is the physical machine running the '''admin''' cell):<br />
ssh -2 -x -o StrictHostKeyChecking=no -l admin -c blowfish -p 22224 dccore.pic.es<br />
* In this document we show what we consider the most important commands for the everyday's administration tasks, and basically the ones that may be useful for the Manager On Duty. Maybe some important commands are missing, but will be added as soon as we detect that these may be important for the MoD.<br />
* This document is divided in two main blocks:<br />
# '''Pool commands''': shows useful commands that can be run on pools.<br />
# '''Service commands''': show useful commands that can be run on servers (this will include doors)<br />
= The basics =<br />
* If you access the Administration Console through '''SSHv2''' you should access to the different cells as follows:<br />
\? [command]... # display help for shell commands<br />
\c cell[@domain] [user] # connect to cell<br />
\exception [trace] # controls display of stack traces<br />
\h [command]... # display help for cell commands<br />
\l [cell[@domain]|pool/poolgroup]... # list cells<br />
\q # quit<br />
'''\s (cell[@domain]|pool/poolgroup)[,(cell[@domain]|pool/poolgroup)]... command... # send command'''<br />
\sl pnfsid|path command... # send to locations<br />
\sn command... # send pnfsmanager command<br />
\sp command... # send poolmanager command<br />
\timeout [seconds] # sets the command timeout<br />
* With '''\s''' you will be able to access to any '''cell[@domain]'''s, while branches of this command such like '''\sn''' or '''\sp''' are useful to (respectively) direct access to the ''namespace'' or ''poolmanager'' cells without the need of specifying the ''cell[@domain]''.<br />
<br />
= Pool commands =<br />
== Introduction ==<br />
* Any command run on a pool is a temporary change: on any pool service restart configuration will be restored back to the default value or to the value set in the '''setup''' file (locate in '''/dcpool*/vpool*/setup''').<br />
:* In example, whenever we set a pool to ''rdonly'', on restart, once the ''initialization'' process finishes, pool will switch back to ''enabled''.<br />
* If you want to make permanent changes, you should '''save''' changes. Running the '''save''' command will update the already mentioned '''setup''' file.<br />
:* In example, whenever we set a pool to ''rdonly'' '''and''' we ''save'' the configuration, on a restart, once the ''initialization'' process finishes, pool will remain ''rdonly''.<br />
<br />
== Disable / enable pools ==<br />
=== Command help ===<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? help hsm'''<br />
''pool disable [options] [<errorCode> [<errorMessage>]]'' # suspend sending 'up messages'<br />
OPTIONS :<br />
-fetch # disallows fetch (transfer to client)<br />
-stage # disallows staging (from HSM)<br />
-store # disallows store (transfer from client)<br />
-p2p-client<br />
-rdonly # := store,stage,p2p-client<br />
-strict # := disallows everything<br />
''pool enable'' # start sending 'up messages' again<br />
''pool suppress hsmload on|off''<br />
<br />
=== Disable / enable pool examples ===<br />
* Disallow store + stage + p2p-client (set pool in read only mode)<br />
'''''[dccore01] (local) admin >''''' '''pool disable -rdonly'''<br />
* Enable pool (set pool to RW)<br />
'''''[dccore01] (local) admin >''''' '''pool enable'''<br />
<br />
== HSM configuration ==<br />
=== Command help ===<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? help hsm'''<br />
hsm create <type> [<name> [<provider>]] [-<key>=<value>] ...<br />
hsm ls [<name>] ...<br />
hsm remove <name><br />
hsm set <name> [-<key>=<value>] ...<br />
hsm show providers # show available nearline storage providers<br />
hsm unset <name> [-<key>] ...<br />
=== HSM configuration examples ===<br />
* Create a ''enstore'' HSM with the ''real-encp.sh'' client script (this example can be found in the '''setup''' file):<br />
'''''[dccore01] (local) admin >''''' '''hsm create enstore enstore script -pnfs=/pnfs/fs/usr -command=/opt/enstore/bin/real-encp.sh -c:gets=200 -c:puts=20 -c:removes=2'''<br />
* Show ''enstore'' configuration:<br />
'''''[dccore01] (local) admin >''''' '''hsm ls enstore'''<br />
enstore(enstore):script<br />
pnfs /pnfs/fs/usr<br />
command /opt/enstore/bin/real-encp.sh<br />
c:gets 200<br />
c:puts 20<br />
c:removes 2<br />
* Set maximum restores to 400:<br />
'''''[dccore01] (local) admin >''''' '''hsm set enstore -c:gets=400'''<br />
<br />
== Mover management ==<br />
=== Command help ===<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? help hsm'''<br />
''mover kill <jobid>'' # terminate a file transfer connection<br />
''mover ls [-binary [jobId] ]''<br />
''mover queue ls [-l]'' # list all mover queues in this pool<br />
''mover remove [<jobid>]'' # #OBSOLETE command<br />
''mover set max active [OPTIONS] <maxactivemovers>'' # set the maximum number of active client transfers<br />
<br />
=== Mover management examples ===<br />
* Show current '''ftp''' transfers:<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? mover ls -l -queue=ftp'''<br />
33743014 : RUNNING : 0000D5B91029797545859181EB738C25E93C IoMode=READ h={SU=0;SA=0;S=None} bytes=12113543168 time/sec=3893 LM=0 si={vo-cms.cms}<br />
33743019 : RUNNING : 00008862CFF7A1C74513A617D364A4162DAA IoMode=READ h={SU=0;SA=0;S=None} bytes=10561388544 time/sec=3409 LM=0 si={vo-cms.cms}<br />
33743076 : RUNNING : 0000736CFC0DAF494816BFE7CC32D3192900 IoMode=READ h={SU=0;SA=0;S=None} bytes=2034630656 time/sec=2478 LM=0 si={vo-cms.cms}<br />
33743086 : RUNNING : 0000C5F1B9D6EEC04DEEA077CA6753E30DE0 IoMode=READ h={SU=0;SA=0;S=None} bytes=488898560 time/sec=1357 LM=1 si={vo-cms.cms}<br />
33743103 : RUNNING : 0000BBE684CB4BA44AB5ACB0D13C5DD25C8C IoMode=READ h={SU=0;SA=0;S=None} bytes=135921664 time/sec=239 LM=20 si={vo-cms.cms}<br />
* Show all mover queue properties:<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? mover queue ls -l'''<br />
regular 0 0 0<br />
p2p 0 40 0<br />
ftp 11 500 0<br />
dcap 0 10 0<br />
nfs4 0 100 0<br />
xrootd 406 1500 0<br />
xrootd-wan 0 700 0<br />
webdav 0 0 0<br />
* Set to 400 the maximum active client transfers for the '''ftp''' queue:<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? mover set max active -queue=ftp 400'''<br />
Max Active Io Movers set to 400<br />
<br />
== File / replica management ==<br />
=== Command help ===<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? help hsm'''<br />
''rep ls [-l[=s,l,u,nc,p]] [-s[=kmgt]] [-storage=<glob>] | <pnfsId>...''<br />
Format I : <pnfsId>...<br />
Format II : [-l[=<selectionOptions>]] [-storage=<glob>] [-s]<br />
Options :<br />
-l[=splunc] # selected list<br />
s : sticky files<br />
p : precious files<br />
l : locked files<br />
u : files in use<br />
nc : files which are not cached<br />
e : files which error condition<br />
-storage=<glob> # select only replicas<br />
of files with storage-<br />
info that matches <glob><br />
-s[=kmgt] [-sum] # statistics<br />
k : data amount in KBytes<br />
m : data amount in MBytes<br />
g : data amount in GBytes<br />
t : data amount in TBytes<br />
Output is a list of repository entries, one per line<br />
each line has the followin syntax:<br />
<pnfsid> <state> <size> <storageinfo><br />
state is a sequence of state bits inclosed in angular "<>" brackets <br />
bit 1 is "C" if entry is cached or "-" if not <br />
bit 2 is "P" if entry is precious or "-" if not <br />
bit 3 is "C" if entry is being transfered "from client" or "-" if not <br />
bit 4 is "S" if entry is being transfered "from store" or "-" if not <br />
bit 5 is "c" if entry is being transfered "to client" or "-" if not <br />
bit 6 is "s" if entry is being transfered "to store" or "-" if not <br />
bit 7 is "R" if entry is removed or "-" if not <br />
bit 8 is is always "-" <br />
bit 9 is "X" if entry is sticky or "-" if not <br />
bit 10 is "E" if entry is in error state or "-" if not <br />
bit 11 is "L(x)(y)" if entry is in locked or "-" if not <br />
x is epoch until which the entry is locked, 0 for non expiring lock <br />
y is the link count<br />
''rep rm <pnfsid> [-force]'' # removes the pnfsfile from the cache<br />
''rep rmclass <storageClass>'' # removes the from the cache<br />
''rep set broken <pnfsid>''<br />
''rep set cached <pnfsId>'' # DON'T USE, Potentially dangerous<br />
''rep set precious <pnfsId>''<br />
''rep set sticky [OPTIONS] [<pnfsid>] on|off'' # change sticky flags<br />
''rep sticky ls'' <pnfsid><br />
=== File / replica management examples ===</div>Mcaubethttps://pwiki.pic.es/index.php?title=DCache_Administration_Console_Commands&diff=200DCache Administration Console Commands2017-07-31T09:39:43Z<p>Mcaubet: /* Command help */</p>
<hr />
<div>= About this document =<br />
* This document shows several examples of useful commands that can be run in the dCache Administration Console.<br />
* Administration console can be accessed through '''[[https://www.dcache.org/downloads/gui/ PCells GUI]]''' or through '''SSHv2''' as follows (in the example, '''dccore.pic.es''' is the physical machine running the '''admin''' cell):<br />
ssh -2 -x -o StrictHostKeyChecking=no -l admin -c blowfish -p 22224 dccore.pic.es<br />
* In this document we show what we consider the most important commands for the everyday's administration tasks, and basically the ones that may be useful for the Manager On Duty. Maybe some important commands are missing, but will be added as soon as we detect that these may be important for the MoD.<br />
* This document is divided in two main blocks:<br />
# '''Pool commands''': shows useful commands that can be run on pools.<br />
# '''Service commands''': show useful commands that can be run on servers (this will include doors)<br />
= The basics =<br />
* If you access the Administration Console through '''SSHv2''' you should access to the different cells as follows:<br />
\? [command]... # display help for shell commands<br />
\c cell[@domain] [user] # connect to cell<br />
\exception [trace] # controls display of stack traces<br />
\h [command]... # display help for cell commands<br />
\l [cell[@domain]|pool/poolgroup]... # list cells<br />
\q # quit<br />
'''\s (cell[@domain]|pool/poolgroup)[,(cell[@domain]|pool/poolgroup)]... command... # send command'''<br />
\sl pnfsid|path command... # send to locations<br />
\sn command... # send pnfsmanager command<br />
\sp command... # send poolmanager command<br />
\timeout [seconds] # sets the command timeout<br />
* With '''\s''' you will be able to access to any '''cell[@domain]'''s, while branches of this command such like '''\sn''' or '''\sp''' are useful to (respectively) direct access to the ''namespace'' or ''poolmanager'' cells without the need of specifying the ''cell[@domain]''.<br />
<br />
= Pool commands =<br />
== Introduction ==<br />
* Any command run on a pool is a temporary change: on any pool service restart configuration will be restored back to the default value or to the value set in the '''setup''' file (locate in '''/dcpool*/vpool*/setup''').<br />
:* In example, whenever we set a pool to ''rdonly'', on restart, once the ''initialization'' process finishes, pool will switch back to ''enabled''.<br />
* If you want to make permanent changes, you should '''save''' changes. Running the '''save''' command will update the already mentioned '''setup''' file.<br />
:* In example, whenever we set a pool to ''rdonly'' '''and''' we ''save'' the configuration, on a restart, once the ''initialization'' process finishes, pool will remain ''rdonly''.<br />
<br />
== Disable / enable pools ==<br />
=== Command help ===<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? help hsm'''<br />
''pool disable [options] [<errorCode> [<errorMessage>]]'' # suspend sending 'up messages'<br />
OPTIONS :<br />
-fetch # disallows fetch (transfer to client)<br />
-stage # disallows staging (from HSM)<br />
-store # disallows store (transfer from client)<br />
-p2p-client<br />
-rdonly # := store,stage,p2p-client<br />
-strict # := disallows everything<br />
''pool enable'' # start sending 'up messages' again<br />
''pool suppress hsmload on|off''<br />
<br />
=== Disable / enable pool examples ===<br />
* Disallow store + stage + p2p-client (set pool in read only mode)<br />
'''''[dccore01] (local) admin >''''' '''pool disable -rdonly'''<br />
* Enable pool (set pool to RW)<br />
'''''[dccore01] (local) admin >''''' '''pool enable'''<br />
<br />
== HSM configuration ==<br />
=== Command help ===<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? help hsm'''<br />
hsm create <type> [<name> [<provider>]] [-<key>=<value>] ...<br />
hsm ls [<name>] ...<br />
hsm remove <name><br />
hsm set <name> [-<key>=<value>] ...<br />
hsm show providers # show available nearline storage providers<br />
hsm unset <name> [-<key>] ...<br />
=== HSM configuration examples ===<br />
* Create a ''enstore'' HSM with the ''real-encp.sh'' client script (this example can be found in the '''setup''' file):<br />
'''''[dccore01] (local) admin >''''' '''hsm create enstore enstore script -pnfs=/pnfs/fs/usr -command=/opt/enstore/bin/real-encp.sh -c:gets=200 -c:puts=20 -c:removes=2'''<br />
* Show ''enstore'' configuration:<br />
'''''[dccore01] (local) admin >''''' '''hsm ls enstore'''<br />
enstore(enstore):script<br />
pnfs /pnfs/fs/usr<br />
command /opt/enstore/bin/real-encp.sh<br />
c:gets 200<br />
c:puts 20<br />
c:removes 2<br />
* Set maximum restores to 400:<br />
'''''[dccore01] (local) admin >''''' '''hsm set enstore -c:gets=400'''<br />
<br />
== Mover management ==<br />
=== Command help ===<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? help hsm'''<br />
''mover kill <jobid>'' # terminate a file transfer connection<br />
''mover ls [-binary [jobId] ]''<br />
''mover queue ls [-l]'' # list all mover queues in this pool<br />
''mover remove [<jobid>]'' # #OBSOLETE command<br />
''mover set max active [OPTIONS] <maxactivemovers>'' # set the maximum number of active client transfers<br />
<br />
=== Mover management examples ===<br />
* Show current '''ftp''' transfers:<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? mover ls -l -queue=ftp'''<br />
33743014 : RUNNING : 0000D5B91029797545859181EB738C25E93C IoMode=READ h={SU=0;SA=0;S=None} bytes=12113543168 time/sec=3893 LM=0 si={vo-cms.cms}<br />
33743019 : RUNNING : 00008862CFF7A1C74513A617D364A4162DAA IoMode=READ h={SU=0;SA=0;S=None} bytes=10561388544 time/sec=3409 LM=0 si={vo-cms.cms}<br />
33743076 : RUNNING : 0000736CFC0DAF494816BFE7CC32D3192900 IoMode=READ h={SU=0;SA=0;S=None} bytes=2034630656 time/sec=2478 LM=0 si={vo-cms.cms}<br />
33743086 : RUNNING : 0000C5F1B9D6EEC04DEEA077CA6753E30DE0 IoMode=READ h={SU=0;SA=0;S=None} bytes=488898560 time/sec=1357 LM=1 si={vo-cms.cms}<br />
33743103 : RUNNING : 0000BBE684CB4BA44AB5ACB0D13C5DD25C8C IoMode=READ h={SU=0;SA=0;S=None} bytes=135921664 time/sec=239 LM=20 si={vo-cms.cms}<br />
* Show all mover queue properties:<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? mover queue ls -l'''<br />
regular 0 0 0<br />
p2p 0 40 0<br />
ftp 11 500 0<br />
dcap 0 10 0<br />
nfs4 0 100 0<br />
xrootd 406 1500 0<br />
xrootd-wan 0 700 0<br />
webdav 0 0 0<br />
* Set to 400 the maximum active client transfers for the '''ftp''' queue:<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? mover set max active -queue=ftp 400'''<br />
Max Active Io Movers set to 400<br />
<br />
== File / replica management ==<br />
=== Command help ===<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? help hsm'''<br />
''rep ls [-l[=s,l,u,nc,p]] [-s[=kmgt]] [-storage=<glob>] | <pnfsId>...''<br />
Format I : <pnfsId>...<br />
Format II : [-l[=<selectionOptions>]] [-storage=<glob>] [-s]<br />
Options :<br />
-l[=splunc] # selected list<br />
s : sticky files<br />
p : precious files<br />
l : locked files<br />
u : files in use<br />
nc : files which are not cached<br />
e : files which error condition<br />
-storage=<glob> # select only replicas<br />
of files with storage-<br />
info that matches <glob><br />
-s[=kmgt] [-sum] # statistics<br />
k : data amount in KBytes<br />
m : data amount in MBytes<br />
g : data amount in GBytes<br />
t : data amount in TBytes<br />
Output is a list of repository entries, one per line<br />
each line has the followin syntax:<br />
<pnfsid> <state> <size> <storageinfo><br />
state is a sequence of state bits inclosed in angular "<>" brackets <br />
bit 1 is "C" if entry is cached or "-" if not <br />
bit 2 is "P" if entry is precious or "-" if not <br />
bit 3 is "C" if entry is being transfered "from client" or "-" if not <br />
bit 4 is "S" if entry is being transfered "from store" or "-" if not <br />
bit 5 is "c" if entry is being transfered "to client" or "-" if not <br />
bit 6 is "s" if entry is being transfered "to store" or "-" if not <br />
bit 7 is "R" if entry is removed or "-" if not <br />
bit 8 is is always "-" <br />
bit 9 is "X" if entry is sticky or "-" if not <br />
bit 10 is "E" if entry is in error state or "-" if not <br />
bit 11 is "L(x)(y)" if entry is in locked or "-" if not <br />
x is epoch until which the entry is locked, 0 for non expiring lock <br />
y is the link count<br />
''rep rm <pnfsid> [-force]'' # removes the pnfsfile from the cache<br />
''rep rmclass <storageClass>'' # removes the from the cache<br />
''rep set broken <pnfsid>''<br />
''rep set cached <pnfsId>'' # DON'T USE, Potentially dangerous<br />
''rep set precious <pnfsId>''<br />
''rep set sticky [OPTIONS] [<pnfsid>] on|off'' # change sticky flags<br />
''rep sticky ls'' <pnfsid></div>Mcaubethttps://pwiki.pic.es/index.php?title=DCache_Administration_Console_Commands&diff=199DCache Administration Console Commands2017-07-31T09:39:23Z<p>Mcaubet: /* Command help */</p>
<hr />
<div>= About this document =<br />
* This document shows several examples of useful commands that can be run in the dCache Administration Console.<br />
* Administration console can be accessed through '''[[https://www.dcache.org/downloads/gui/ PCells GUI]]''' or through '''SSHv2''' as follows (in the example, '''dccore.pic.es''' is the physical machine running the '''admin''' cell):<br />
ssh -2 -x -o StrictHostKeyChecking=no -l admin -c blowfish -p 22224 dccore.pic.es<br />
* In this document we show what we consider the most important commands for the everyday's administration tasks, and basically the ones that may be useful for the Manager On Duty. Maybe some important commands are missing, but will be added as soon as we detect that these may be important for the MoD.<br />
* This document is divided in two main blocks:<br />
# '''Pool commands''': shows useful commands that can be run on pools.<br />
# '''Service commands''': show useful commands that can be run on servers (this will include doors)<br />
= The basics =<br />
* If you access the Administration Console through '''SSHv2''' you should access to the different cells as follows:<br />
\? [command]... # display help for shell commands<br />
\c cell[@domain] [user] # connect to cell<br />
\exception [trace] # controls display of stack traces<br />
\h [command]... # display help for cell commands<br />
\l [cell[@domain]|pool/poolgroup]... # list cells<br />
\q # quit<br />
'''\s (cell[@domain]|pool/poolgroup)[,(cell[@domain]|pool/poolgroup)]... command... # send command'''<br />
\sl pnfsid|path command... # send to locations<br />
\sn command... # send pnfsmanager command<br />
\sp command... # send poolmanager command<br />
\timeout [seconds] # sets the command timeout<br />
* With '''\s''' you will be able to access to any '''cell[@domain]'''s, while branches of this command such like '''\sn''' or '''\sp''' are useful to (respectively) direct access to the ''namespace'' or ''poolmanager'' cells without the need of specifying the ''cell[@domain]''.<br />
<br />
= Pool commands =<br />
== Introduction ==<br />
* Any command run on a pool is a temporary change: on any pool service restart configuration will be restored back to the default value or to the value set in the '''setup''' file (locate in '''/dcpool*/vpool*/setup''').<br />
:* In example, whenever we set a pool to ''rdonly'', on restart, once the ''initialization'' process finishes, pool will switch back to ''enabled''.<br />
* If you want to make permanent changes, you should '''save''' changes. Running the '''save''' command will update the already mentioned '''setup''' file.<br />
:* In example, whenever we set a pool to ''rdonly'' '''and''' we ''save'' the configuration, on a restart, once the ''initialization'' process finishes, pool will remain ''rdonly''.<br />
<br />
== Disable / enable pools ==<br />
=== Command help ===<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? help hsm'''<br />
pool disable [options] [<errorCode> [<errorMessage>]] # suspend sending 'up messages'<br />
OPTIONS :<br />
-fetch # disallows fetch (transfer to client)<br />
-stage # disallows staging (from HSM)<br />
-store # disallows store (transfer from client)<br />
-p2p-client<br />
-rdonly # := store,stage,p2p-client<br />
-strict # := disallows everything<br />
pool enable # start sending 'up messages' again<br />
pool suppress hsmload on|off<br />
<br />
=== Disable / enable pool examples ===<br />
* Disallow store + stage + p2p-client (set pool in read only mode)<br />
'''''[dccore01] (local) admin >''''' '''pool disable -rdonly'''<br />
* Enable pool (set pool to RW)<br />
'''''[dccore01] (local) admin >''''' '''pool enable'''<br />
<br />
== HSM configuration ==<br />
=== Command help ===<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? help hsm'''<br />
hsm create <type> [<name> [<provider>]] [-<key>=<value>] ...<br />
hsm ls [<name>] ...<br />
hsm remove <name><br />
hsm set <name> [-<key>=<value>] ...<br />
hsm show providers # show available nearline storage providers<br />
hsm unset <name> [-<key>] ...<br />
=== HSM configuration examples ===<br />
* Create a ''enstore'' HSM with the ''real-encp.sh'' client script (this example can be found in the '''setup''' file):<br />
'''''[dccore01] (local) admin >''''' '''hsm create enstore enstore script -pnfs=/pnfs/fs/usr -command=/opt/enstore/bin/real-encp.sh -c:gets=200 -c:puts=20 -c:removes=2'''<br />
* Show ''enstore'' configuration:<br />
'''''[dccore01] (local) admin >''''' '''hsm ls enstore'''<br />
enstore(enstore):script<br />
pnfs /pnfs/fs/usr<br />
command /opt/enstore/bin/real-encp.sh<br />
c:gets 200<br />
c:puts 20<br />
c:removes 2<br />
* Set maximum restores to 400:<br />
'''''[dccore01] (local) admin >''''' '''hsm set enstore -c:gets=400'''<br />
<br />
== Mover management ==<br />
=== Command help ===<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? help hsm'''<br />
''mover kill <jobid>'' # terminate a file transfer connection<br />
''mover ls [-binary [jobId] ]''<br />
''mover queue ls [-l]'' # list all mover queues in this pool<br />
''mover remove [<jobid>]'' # #OBSOLETE command<br />
''mover set max active [OPTIONS] <maxactivemovers>'' # set the maximum number of active client transfers<br />
<br />
=== Mover management examples ===<br />
* Show current '''ftp''' transfers:<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? mover ls -l -queue=ftp'''<br />
33743014 : RUNNING : 0000D5B91029797545859181EB738C25E93C IoMode=READ h={SU=0;SA=0;S=None} bytes=12113543168 time/sec=3893 LM=0 si={vo-cms.cms}<br />
33743019 : RUNNING : 00008862CFF7A1C74513A617D364A4162DAA IoMode=READ h={SU=0;SA=0;S=None} bytes=10561388544 time/sec=3409 LM=0 si={vo-cms.cms}<br />
33743076 : RUNNING : 0000736CFC0DAF494816BFE7CC32D3192900 IoMode=READ h={SU=0;SA=0;S=None} bytes=2034630656 time/sec=2478 LM=0 si={vo-cms.cms}<br />
33743086 : RUNNING : 0000C5F1B9D6EEC04DEEA077CA6753E30DE0 IoMode=READ h={SU=0;SA=0;S=None} bytes=488898560 time/sec=1357 LM=1 si={vo-cms.cms}<br />
33743103 : RUNNING : 0000BBE684CB4BA44AB5ACB0D13C5DD25C8C IoMode=READ h={SU=0;SA=0;S=None} bytes=135921664 time/sec=239 LM=20 si={vo-cms.cms}<br />
* Show all mover queue properties:<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? mover queue ls -l'''<br />
regular 0 0 0<br />
p2p 0 40 0<br />
ftp 11 500 0<br />
dcap 0 10 0<br />
nfs4 0 100 0<br />
xrootd 406 1500 0<br />
xrootd-wan 0 700 0<br />
webdav 0 0 0<br />
* Set to 400 the maximum active client transfers for the '''ftp''' queue:<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? mover set max active -queue=ftp 400'''<br />
Max Active Io Movers set to 400<br />
<br />
== File / replica management ==<br />
=== Command help ===<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? help hsm'''<br />
''rep ls [-l[=s,l,u,nc,p]] [-s[=kmgt]] [-storage=<glob>] | <pnfsId>...''<br />
Format I : <pnfsId>...<br />
Format II : [-l[=<selectionOptions>]] [-storage=<glob>] [-s]<br />
Options :<br />
-l[=splunc] # selected list<br />
s : sticky files<br />
p : precious files<br />
l : locked files<br />
u : files in use<br />
nc : files which are not cached<br />
e : files which error condition<br />
-storage=<glob> # select only replicas<br />
of files with storage-<br />
info that matches <glob><br />
-s[=kmgt] [-sum] # statistics<br />
k : data amount in KBytes<br />
m : data amount in MBytes<br />
g : data amount in GBytes<br />
t : data amount in TBytes<br />
Output is a list of repository entries, one per line<br />
each line has the followin syntax:<br />
<pnfsid> <state> <size> <storageinfo><br />
state is a sequence of state bits inclosed in angular "<>" brackets <br />
bit 1 is "C" if entry is cached or "-" if not <br />
bit 2 is "P" if entry is precious or "-" if not <br />
bit 3 is "C" if entry is being transfered "from client" or "-" if not <br />
bit 4 is "S" if entry is being transfered "from store" or "-" if not <br />
bit 5 is "c" if entry is being transfered "to client" or "-" if not <br />
bit 6 is "s" if entry is being transfered "to store" or "-" if not <br />
bit 7 is "R" if entry is removed or "-" if not <br />
bit 8 is is always "-" <br />
bit 9 is "X" if entry is sticky or "-" if not <br />
bit 10 is "E" if entry is in error state or "-" if not <br />
bit 11 is "L(x)(y)" if entry is in locked or "-" if not <br />
x is epoch until which the entry is locked, 0 for non expiring lock <br />
y is the link count<br />
''rep rm <pnfsid> [-force]'' # removes the pnfsfile from the cache<br />
''rep rmclass <storageClass>'' # removes the from the cache<br />
''rep set broken <pnfsid>''<br />
''rep set cached <pnfsId>'' # DON'T USE, Potentially dangerous<br />
''rep set precious <pnfsId>''<br />
''rep set sticky [OPTIONS] [<pnfsid>] on|off'' # change sticky flags<br />
''rep sticky ls'' <pnfsid></div>Mcaubethttps://pwiki.pic.es/index.php?title=DCache_Administration_Console_Commands&diff=198DCache Administration Console Commands2017-07-31T09:38:55Z<p>Mcaubet: /* Command help */</p>
<hr />
<div>= About this document =<br />
* This document shows several examples of useful commands that can be run in the dCache Administration Console.<br />
* Administration console can be accessed through '''[[https://www.dcache.org/downloads/gui/ PCells GUI]]''' or through '''SSHv2''' as follows (in the example, '''dccore.pic.es''' is the physical machine running the '''admin''' cell):<br />
ssh -2 -x -o StrictHostKeyChecking=no -l admin -c blowfish -p 22224 dccore.pic.es<br />
* In this document we show what we consider the most important commands for the everyday's administration tasks, and basically the ones that may be useful for the Manager On Duty. Maybe some important commands are missing, but will be added as soon as we detect that these may be important for the MoD.<br />
* This document is divided in two main blocks:<br />
# '''Pool commands''': shows useful commands that can be run on pools.<br />
# '''Service commands''': show useful commands that can be run on servers (this will include doors)<br />
= The basics =<br />
* If you access the Administration Console through '''SSHv2''' you should access to the different cells as follows:<br />
\? [command]... # display help for shell commands<br />
\c cell[@domain] [user] # connect to cell<br />
\exception [trace] # controls display of stack traces<br />
\h [command]... # display help for cell commands<br />
\l [cell[@domain]|pool/poolgroup]... # list cells<br />
\q # quit<br />
'''\s (cell[@domain]|pool/poolgroup)[,(cell[@domain]|pool/poolgroup)]... command... # send command'''<br />
\sl pnfsid|path command... # send to locations<br />
\sn command... # send pnfsmanager command<br />
\sp command... # send poolmanager command<br />
\timeout [seconds] # sets the command timeout<br />
* With '''\s''' you will be able to access to any '''cell[@domain]'''s, while branches of this command such like '''\sn''' or '''\sp''' are useful to (respectively) direct access to the ''namespace'' or ''poolmanager'' cells without the need of specifying the ''cell[@domain]''.<br />
<br />
= Pool commands =<br />
== Introduction ==<br />
* Any command run on a pool is a temporary change: on any pool service restart configuration will be restored back to the default value or to the value set in the '''setup''' file (locate in '''/dcpool*/vpool*/setup''').<br />
:* In example, whenever we set a pool to ''rdonly'', on restart, once the ''initialization'' process finishes, pool will switch back to ''enabled''.<br />
* If you want to make permanent changes, you should '''save''' changes. Running the '''save''' command will update the already mentioned '''setup''' file.<br />
:* In example, whenever we set a pool to ''rdonly'' '''and''' we ''save'' the configuration, on a restart, once the ''initialization'' process finishes, pool will remain ''rdonly''.<br />
<br />
== Disable / enable pools ==<br />
=== Command help ===<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? help hsm'''<br />
pool disable [options] [<errorCode> [<errorMessage>]] # suspend sending 'up messages'<br />
OPTIONS :<br />
-fetch # disallows fetch (transfer to client)<br />
-stage # disallows staging (from HSM)<br />
-store # disallows store (transfer from client)<br />
-p2p-client<br />
-rdonly # := store,stage,p2p-client<br />
-strict # := disallows everything<br />
pool enable # start sending 'up messages' again<br />
pool suppress hsmload on|off<br />
<br />
=== Disable / enable pool examples ===<br />
* Disallow store + stage + p2p-client (set pool in read only mode)<br />
'''''[dccore01] (local) admin >''''' '''pool disable -rdonly'''<br />
* Enable pool (set pool to RW)<br />
'''''[dccore01] (local) admin >''''' '''pool enable'''<br />
<br />
== HSM configuration ==<br />
=== Command help ===<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? help hsm'''<br />
hsm create <type> [<name> [<provider>]] [-<key>=<value>] ...<br />
hsm ls [<name>] ...<br />
hsm remove <name><br />
hsm set <name> [-<key>=<value>] ...<br />
hsm show providers # show available nearline storage providers<br />
hsm unset <name> [-<key>] ...<br />
=== HSM configuration examples ===<br />
* Create a ''enstore'' HSM with the ''real-encp.sh'' client script (this example can be found in the '''setup''' file):<br />
'''''[dccore01] (local) admin >''''' '''hsm create enstore enstore script -pnfs=/pnfs/fs/usr -command=/opt/enstore/bin/real-encp.sh -c:gets=200 -c:puts=20 -c:removes=2'''<br />
* Show ''enstore'' configuration:<br />
'''''[dccore01] (local) admin >''''' '''hsm ls enstore'''<br />
enstore(enstore):script<br />
pnfs /pnfs/fs/usr<br />
command /opt/enstore/bin/real-encp.sh<br />
c:gets 200<br />
c:puts 20<br />
c:removes 2<br />
* Set maximum restores to 400:<br />
'''''[dccore01] (local) admin >''''' '''hsm set enstore -c:gets=400'''<br />
<br />
== Mover management ==<br />
=== Command help ===<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? help hsm'''<br />
mover kill <jobid> # terminate a file transfer connection<br />
mover ls [-binary [jobId] ]<br />
mover queue ls [-l] # list all mover queues in this pool<br />
mover remove [<jobid>] # #OBSOLETE command<br />
mover set max active [OPTIONS] <maxactivemovers> # set the maximum number of active client transfers<br />
=== Mover management examples ===<br />
* Show current '''ftp''' transfers:<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? mover ls -l -queue=ftp'''<br />
33743014 : RUNNING : 0000D5B91029797545859181EB738C25E93C IoMode=READ h={SU=0;SA=0;S=None} bytes=12113543168 time/sec=3893 LM=0 si={vo-cms.cms}<br />
33743019 : RUNNING : 00008862CFF7A1C74513A617D364A4162DAA IoMode=READ h={SU=0;SA=0;S=None} bytes=10561388544 time/sec=3409 LM=0 si={vo-cms.cms}<br />
33743076 : RUNNING : 0000736CFC0DAF494816BFE7CC32D3192900 IoMode=READ h={SU=0;SA=0;S=None} bytes=2034630656 time/sec=2478 LM=0 si={vo-cms.cms}<br />
33743086 : RUNNING : 0000C5F1B9D6EEC04DEEA077CA6753E30DE0 IoMode=READ h={SU=0;SA=0;S=None} bytes=488898560 time/sec=1357 LM=1 si={vo-cms.cms}<br />
33743103 : RUNNING : 0000BBE684CB4BA44AB5ACB0D13C5DD25C8C IoMode=READ h={SU=0;SA=0;S=None} bytes=135921664 time/sec=239 LM=20 si={vo-cms.cms}<br />
* Show all mover queue properties:<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? mover queue ls -l'''<br />
regular 0 0 0<br />
p2p 0 40 0<br />
ftp 11 500 0<br />
dcap 0 10 0<br />
nfs4 0 100 0<br />
xrootd 406 1500 0<br />
xrootd-wan 0 700 0<br />
webdav 0 0 0<br />
* Set to 400 the maximum active client transfers for the '''ftp''' queue:<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? mover set max active -queue=ftp 400'''<br />
Max Active Io Movers set to 400<br />
<br />
== File / replica management ==<br />
=== Command help ===<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? help hsm'''<br />
''rep ls [-l[=s,l,u,nc,p]] [-s[=kmgt]] [-storage=<glob>] | <pnfsId>...''<br />
Format I : <pnfsId>...<br />
Format II : [-l[=<selectionOptions>]] [-storage=<glob>] [-s]<br />
Options :<br />
-l[=splunc] # selected list<br />
s : sticky files<br />
p : precious files<br />
l : locked files<br />
u : files in use<br />
nc : files which are not cached<br />
e : files which error condition<br />
-storage=<glob> # select only replicas<br />
of files with storage-<br />
info that matches <glob><br />
-s[=kmgt] [-sum] # statistics<br />
k : data amount in KBytes<br />
m : data amount in MBytes<br />
g : data amount in GBytes<br />
t : data amount in TBytes<br />
Output is a list of repository entries, one per line<br />
each line has the followin syntax:<br />
<pnfsid> <state> <size> <storageinfo><br />
state is a sequence of state bits inclosed in angular "<>" brackets <br />
bit 1 is "C" if entry is cached or "-" if not <br />
bit 2 is "P" if entry is precious or "-" if not <br />
bit 3 is "C" if entry is being transfered "from client" or "-" if not <br />
bit 4 is "S" if entry is being transfered "from store" or "-" if not <br />
bit 5 is "c" if entry is being transfered "to client" or "-" if not <br />
bit 6 is "s" if entry is being transfered "to store" or "-" if not <br />
bit 7 is "R" if entry is removed or "-" if not <br />
bit 8 is is always "-" <br />
bit 9 is "X" if entry is sticky or "-" if not <br />
bit 10 is "E" if entry is in error state or "-" if not <br />
bit 11 is "L(x)(y)" if entry is in locked or "-" if not <br />
x is epoch until which the entry is locked, 0 for non expiring lock <br />
y is the link count<br />
''rep rm <pnfsid> [-force]'' # removes the pnfsfile from the cache<br />
''rep rmclass <storageClass>'' # removes the from the cache<br />
''rep set broken <pnfsid>''<br />
''rep set cached <pnfsId>'' # DON'T USE, Potentially dangerous<br />
''rep set precious <pnfsId>''<br />
''rep set sticky [OPTIONS] [<pnfsid>] on|off'' # change sticky flags<br />
''rep sticky ls'' <pnfsid></div>Mcaubethttps://pwiki.pic.es/index.php?title=DCache_Administration_Console_Commands&diff=197DCache Administration Console Commands2017-07-31T09:38:17Z<p>Mcaubet: /* Command help */</p>
<hr />
<div>= About this document =<br />
* This document shows several examples of useful commands that can be run in the dCache Administration Console.<br />
* Administration console can be accessed through '''[[https://www.dcache.org/downloads/gui/ PCells GUI]]''' or through '''SSHv2''' as follows (in the example, '''dccore.pic.es''' is the physical machine running the '''admin''' cell):<br />
ssh -2 -x -o StrictHostKeyChecking=no -l admin -c blowfish -p 22224 dccore.pic.es<br />
* In this document we show what we consider the most important commands for the everyday's administration tasks, and basically the ones that may be useful for the Manager On Duty. Maybe some important commands are missing, but will be added as soon as we detect that these may be important for the MoD.<br />
* This document is divided in two main blocks:<br />
# '''Pool commands''': shows useful commands that can be run on pools.<br />
# '''Service commands''': show useful commands that can be run on servers (this will include doors)<br />
= The basics =<br />
* If you access the Administration Console through '''SSHv2''' you should access to the different cells as follows:<br />
\? [command]... # display help for shell commands<br />
\c cell[@domain] [user] # connect to cell<br />
\exception [trace] # controls display of stack traces<br />
\h [command]... # display help for cell commands<br />
\l [cell[@domain]|pool/poolgroup]... # list cells<br />
\q # quit<br />
'''\s (cell[@domain]|pool/poolgroup)[,(cell[@domain]|pool/poolgroup)]... command... # send command'''<br />
\sl pnfsid|path command... # send to locations<br />
\sn command... # send pnfsmanager command<br />
\sp command... # send poolmanager command<br />
\timeout [seconds] # sets the command timeout<br />
* With '''\s''' you will be able to access to any '''cell[@domain]'''s, while branches of this command such like '''\sn''' or '''\sp''' are useful to (respectively) direct access to the ''namespace'' or ''poolmanager'' cells without the need of specifying the ''cell[@domain]''.<br />
<br />
= Pool commands =<br />
== Introduction ==<br />
* Any command run on a pool is a temporary change: on any pool service restart configuration will be restored back to the default value or to the value set in the '''setup''' file (locate in '''/dcpool*/vpool*/setup''').<br />
:* In example, whenever we set a pool to ''rdonly'', on restart, once the ''initialization'' process finishes, pool will switch back to ''enabled''.<br />
* If you want to make permanent changes, you should '''save''' changes. Running the '''save''' command will update the already mentioned '''setup''' file.<br />
:* In example, whenever we set a pool to ''rdonly'' '''and''' we ''save'' the configuration, on a restart, once the ''initialization'' process finishes, pool will remain ''rdonly''.<br />
<br />
== Disable / enable pools ==<br />
=== Command help ===<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? help hsm'''<br />
pool disable [options] [<errorCode> [<errorMessage>]] # suspend sending 'up messages'<br />
OPTIONS :<br />
-fetch # disallows fetch (transfer to client)<br />
-stage # disallows staging (from HSM)<br />
-store # disallows store (transfer from client)<br />
-p2p-client<br />
-rdonly # := store,stage,p2p-client<br />
-strict # := disallows everything<br />
pool enable # start sending 'up messages' again<br />
pool suppress hsmload on|off<br />
<br />
=== Disable / enable pool examples ===<br />
* Disallow store + stage + p2p-client (set pool in read only mode)<br />
'''''[dccore01] (local) admin >''''' '''pool disable -rdonly'''<br />
* Enable pool (set pool to RW)<br />
'''''[dccore01] (local) admin >''''' '''pool enable'''<br />
<br />
== HSM configuration ==<br />
=== Command help ===<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? help hsm'''<br />
hsm create <type> [<name> [<provider>]] [-<key>=<value>] ...<br />
hsm ls [<name>] ...<br />
hsm remove <name><br />
hsm set <name> [-<key>=<value>] ...<br />
hsm show providers # show available nearline storage providers<br />
hsm unset <name> [-<key>] ...<br />
=== HSM configuration examples ===<br />
* Create a ''enstore'' HSM with the ''real-encp.sh'' client script (this example can be found in the '''setup''' file):<br />
'''''[dccore01] (local) admin >''''' '''hsm create enstore enstore script -pnfs=/pnfs/fs/usr -command=/opt/enstore/bin/real-encp.sh -c:gets=200 -c:puts=20 -c:removes=2'''<br />
* Show ''enstore'' configuration:<br />
'''''[dccore01] (local) admin >''''' '''hsm ls enstore'''<br />
enstore(enstore):script<br />
pnfs /pnfs/fs/usr<br />
command /opt/enstore/bin/real-encp.sh<br />
c:gets 200<br />
c:puts 20<br />
c:removes 2<br />
* Set maximum restores to 400:<br />
'''''[dccore01] (local) admin >''''' '''hsm set enstore -c:gets=400'''<br />
<br />
== Mover management ==<br />
=== Command help ===<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? help hsm'''<br />
mover kill <jobid> # terminate a file transfer connection<br />
mover ls [-binary [jobId] ]<br />
mover queue ls [-l] # list all mover queues in this pool<br />
mover remove [<jobid>] # #OBSOLETE command<br />
mover set max active [OPTIONS] <maxactivemovers> # set the maximum number of active client transfers<br />
=== Mover management examples ===<br />
* Show current '''ftp''' transfers:<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? mover ls -l -queue=ftp'''<br />
33743014 : RUNNING : 0000D5B91029797545859181EB738C25E93C IoMode=READ h={SU=0;SA=0;S=None} bytes=12113543168 time/sec=3893 LM=0 si={vo-cms.cms}<br />
33743019 : RUNNING : 00008862CFF7A1C74513A617D364A4162DAA IoMode=READ h={SU=0;SA=0;S=None} bytes=10561388544 time/sec=3409 LM=0 si={vo-cms.cms}<br />
33743076 : RUNNING : 0000736CFC0DAF494816BFE7CC32D3192900 IoMode=READ h={SU=0;SA=0;S=None} bytes=2034630656 time/sec=2478 LM=0 si={vo-cms.cms}<br />
33743086 : RUNNING : 0000C5F1B9D6EEC04DEEA077CA6753E30DE0 IoMode=READ h={SU=0;SA=0;S=None} bytes=488898560 time/sec=1357 LM=1 si={vo-cms.cms}<br />
33743103 : RUNNING : 0000BBE684CB4BA44AB5ACB0D13C5DD25C8C IoMode=READ h={SU=0;SA=0;S=None} bytes=135921664 time/sec=239 LM=20 si={vo-cms.cms}<br />
* Show all mover queue properties:<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? mover queue ls -l'''<br />
regular 0 0 0<br />
p2p 0 40 0<br />
ftp 11 500 0<br />
dcap 0 10 0<br />
nfs4 0 100 0<br />
xrootd 406 1500 0<br />
xrootd-wan 0 700 0<br />
webdav 0 0 0<br />
* Set to 400 the maximum active client transfers for the '''ftp''' queue:<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? mover set max active -queue=ftp 400'''<br />
Max Active Io Movers set to 400<br />
<br />
== File / replica management ==<br />
=== Command help ===<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? help hsm'''<br />
rep ls [-l[=s,l,u,nc,p]] [-s[=kmgt]] [-storage=<glob>] | <pnfsId>...<br />
Format I : <pnfsId>...<br />
Format II : [-l[=<selectionOptions>]] [-storage=<glob>] [-s]<br />
Options :<br />
-l[=splunc] # selected list<br />
s : sticky files<br />
p : precious files<br />
l : locked files<br />
u : files in use<br />
nc : files which are not cached<br />
e : files which error condition<br />
-storage=<glob> # select only replicas<br />
of files with storage-<br />
info that matches <glob><br />
-s[=kmgt] [-sum] # statistics<br />
k : data amount in KBytes<br />
m : data amount in MBytes<br />
g : data amount in GBytes<br />
t : data amount in TBytes<br />
Output is a list of repository entries, one per line<br />
each line has the followin syntax:<br />
<pnfsid> <state> <size> <storageinfo><br />
state is a sequence of state bits inclosed in angular "<>" brackets <br />
bit 1 is "C" if entry is cached or "-" if not <br />
bit 2 is "P" if entry is precious or "-" if not <br />
bit 3 is "C" if entry is being transfered "from client" or "-" if not <br />
bit 4 is "S" if entry is being transfered "from store" or "-" if not <br />
bit 5 is "c" if entry is being transfered "to client" or "-" if not <br />
bit 6 is "s" if entry is being transfered "to store" or "-" if not <br />
bit 7 is "R" if entry is removed or "-" if not <br />
bit 8 is is always "-" <br />
bit 9 is "X" if entry is sticky or "-" if not <br />
bit 10 is "E" if entry is in error state or "-" if not <br />
bit 11 is "L(x)(y)" if entry is in locked or "-" if not <br />
x is epoch until which the entry is locked, 0 for non expiring lock <br />
y is the link count<br />
rep rm <pnfsid> [-force] # removes the pnfsfile from the cache<br />
rep rmclass <storageClass> # removes the from the cache<br />
rep set broken <pnfsid><br />
rep set cached <pnfsId> # DON'T USE, Potentially dangerous<br />
rep set precious <pnfsId><br />
rep set sticky [OPTIONS] [<pnfsid>] on|off # change sticky flags<br />
rep sticky ls <pnfsid></div>Mcaubethttps://pwiki.pic.es/index.php?title=DCache_Administration_Console_Commands&diff=196DCache Administration Console Commands2017-07-31T09:38:02Z<p>Mcaubet: /* Command help */</p>
<hr />
<div>= About this document =<br />
* This document shows several examples of useful commands that can be run in the dCache Administration Console.<br />
* Administration console can be accessed through '''[[https://www.dcache.org/downloads/gui/ PCells GUI]]''' or through '''SSHv2''' as follows (in the example, '''dccore.pic.es''' is the physical machine running the '''admin''' cell):<br />
ssh -2 -x -o StrictHostKeyChecking=no -l admin -c blowfish -p 22224 dccore.pic.es<br />
* In this document we show what we consider the most important commands for the everyday's administration tasks, and basically the ones that may be useful for the Manager On Duty. Maybe some important commands are missing, but will be added as soon as we detect that these may be important for the MoD.<br />
* This document is divided in two main blocks:<br />
# '''Pool commands''': shows useful commands that can be run on pools.<br />
# '''Service commands''': show useful commands that can be run on servers (this will include doors)<br />
= The basics =<br />
* If you access the Administration Console through '''SSHv2''' you should access to the different cells as follows:<br />
\? [command]... # display help for shell commands<br />
\c cell[@domain] [user] # connect to cell<br />
\exception [trace] # controls display of stack traces<br />
\h [command]... # display help for cell commands<br />
\l [cell[@domain]|pool/poolgroup]... # list cells<br />
\q # quit<br />
'''\s (cell[@domain]|pool/poolgroup)[,(cell[@domain]|pool/poolgroup)]... command... # send command'''<br />
\sl pnfsid|path command... # send to locations<br />
\sn command... # send pnfsmanager command<br />
\sp command... # send poolmanager command<br />
\timeout [seconds] # sets the command timeout<br />
* With '''\s''' you will be able to access to any '''cell[@domain]'''s, while branches of this command such like '''\sn''' or '''\sp''' are useful to (respectively) direct access to the ''namespace'' or ''poolmanager'' cells without the need of specifying the ''cell[@domain]''.<br />
<br />
= Pool commands =<br />
== Introduction ==<br />
* Any command run on a pool is a temporary change: on any pool service restart configuration will be restored back to the default value or to the value set in the '''setup''' file (locate in '''/dcpool*/vpool*/setup''').<br />
:* In example, whenever we set a pool to ''rdonly'', on restart, once the ''initialization'' process finishes, pool will switch back to ''enabled''.<br />
* If you want to make permanent changes, you should '''save''' changes. Running the '''save''' command will update the already mentioned '''setup''' file.<br />
:* In example, whenever we set a pool to ''rdonly'' '''and''' we ''save'' the configuration, on a restart, once the ''initialization'' process finishes, pool will remain ''rdonly''.<br />
<br />
== Disable / enable pools ==<br />
=== Command help ===<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? help hsm'''<br />
'pool disable [options] [<errorCode> [<errorMessage>]] # suspend sending 'up messages''<br />
OPTIONS :<br />
-fetch # disallows fetch (transfer to client)<br />
-stage # disallows staging (from HSM)<br />
-store # disallows store (transfer from client)<br />
-p2p-client<br />
-rdonly # := store,stage,p2p-client<br />
-strict # := disallows everything<br />
'pool enable # start sending 'up messages' again'<br />
'pool suppress hsmload on|off'<br />
<br />
=== Disable / enable pool examples ===<br />
* Disallow store + stage + p2p-client (set pool in read only mode)<br />
'''''[dccore01] (local) admin >''''' '''pool disable -rdonly'''<br />
* Enable pool (set pool to RW)<br />
'''''[dccore01] (local) admin >''''' '''pool enable'''<br />
<br />
== HSM configuration ==<br />
=== Command help ===<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? help hsm'''<br />
hsm create <type> [<name> [<provider>]] [-<key>=<value>] ...<br />
hsm ls [<name>] ...<br />
hsm remove <name><br />
hsm set <name> [-<key>=<value>] ...<br />
hsm show providers # show available nearline storage providers<br />
hsm unset <name> [-<key>] ...<br />
=== HSM configuration examples ===<br />
* Create a ''enstore'' HSM with the ''real-encp.sh'' client script (this example can be found in the '''setup''' file):<br />
'''''[dccore01] (local) admin >''''' '''hsm create enstore enstore script -pnfs=/pnfs/fs/usr -command=/opt/enstore/bin/real-encp.sh -c:gets=200 -c:puts=20 -c:removes=2'''<br />
* Show ''enstore'' configuration:<br />
'''''[dccore01] (local) admin >''''' '''hsm ls enstore'''<br />
enstore(enstore):script<br />
pnfs /pnfs/fs/usr<br />
command /opt/enstore/bin/real-encp.sh<br />
c:gets 200<br />
c:puts 20<br />
c:removes 2<br />
* Set maximum restores to 400:<br />
'''''[dccore01] (local) admin >''''' '''hsm set enstore -c:gets=400'''<br />
<br />
== Mover management ==<br />
=== Command help ===<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? help hsm'''<br />
mover kill <jobid> # terminate a file transfer connection<br />
mover ls [-binary [jobId] ]<br />
mover queue ls [-l] # list all mover queues in this pool<br />
mover remove [<jobid>] # #OBSOLETE command<br />
mover set max active [OPTIONS] <maxactivemovers> # set the maximum number of active client transfers<br />
=== Mover management examples ===<br />
* Show current '''ftp''' transfers:<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? mover ls -l -queue=ftp'''<br />
33743014 : RUNNING : 0000D5B91029797545859181EB738C25E93C IoMode=READ h={SU=0;SA=0;S=None} bytes=12113543168 time/sec=3893 LM=0 si={vo-cms.cms}<br />
33743019 : RUNNING : 00008862CFF7A1C74513A617D364A4162DAA IoMode=READ h={SU=0;SA=0;S=None} bytes=10561388544 time/sec=3409 LM=0 si={vo-cms.cms}<br />
33743076 : RUNNING : 0000736CFC0DAF494816BFE7CC32D3192900 IoMode=READ h={SU=0;SA=0;S=None} bytes=2034630656 time/sec=2478 LM=0 si={vo-cms.cms}<br />
33743086 : RUNNING : 0000C5F1B9D6EEC04DEEA077CA6753E30DE0 IoMode=READ h={SU=0;SA=0;S=None} bytes=488898560 time/sec=1357 LM=1 si={vo-cms.cms}<br />
33743103 : RUNNING : 0000BBE684CB4BA44AB5ACB0D13C5DD25C8C IoMode=READ h={SU=0;SA=0;S=None} bytes=135921664 time/sec=239 LM=20 si={vo-cms.cms}<br />
* Show all mover queue properties:<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? mover queue ls -l'''<br />
regular 0 0 0<br />
p2p 0 40 0<br />
ftp 11 500 0<br />
dcap 0 10 0<br />
nfs4 0 100 0<br />
xrootd 406 1500 0<br />
xrootd-wan 0 700 0<br />
webdav 0 0 0<br />
* Set to 400 the maximum active client transfers for the '''ftp''' queue:<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? mover set max active -queue=ftp 400'''<br />
Max Active Io Movers set to 400<br />
<br />
== File / replica management ==<br />
=== Command help ===<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? help hsm'''<br />
rep ls [-l[=s,l,u,nc,p]] [-s[=kmgt]] [-storage=<glob>] | <pnfsId>...<br />
Format I : <pnfsId>...<br />
Format II : [-l[=<selectionOptions>]] [-storage=<glob>] [-s]<br />
Options :<br />
-l[=splunc] # selected list<br />
s : sticky files<br />
p : precious files<br />
l : locked files<br />
u : files in use<br />
nc : files which are not cached<br />
e : files which error condition<br />
-storage=<glob> # select only replicas<br />
of files with storage-<br />
info that matches <glob><br />
-s[=kmgt] [-sum] # statistics<br />
k : data amount in KBytes<br />
m : data amount in MBytes<br />
g : data amount in GBytes<br />
t : data amount in TBytes<br />
Output is a list of repository entries, one per line<br />
each line has the followin syntax:<br />
<pnfsid> <state> <size> <storageinfo><br />
state is a sequence of state bits inclosed in angular "<>" brackets <br />
bit 1 is "C" if entry is cached or "-" if not <br />
bit 2 is "P" if entry is precious or "-" if not <br />
bit 3 is "C" if entry is being transfered "from client" or "-" if not <br />
bit 4 is "S" if entry is being transfered "from store" or "-" if not <br />
bit 5 is "c" if entry is being transfered "to client" or "-" if not <br />
bit 6 is "s" if entry is being transfered "to store" or "-" if not <br />
bit 7 is "R" if entry is removed or "-" if not <br />
bit 8 is is always "-" <br />
bit 9 is "X" if entry is sticky or "-" if not <br />
bit 10 is "E" if entry is in error state or "-" if not <br />
bit 11 is "L(x)(y)" if entry is in locked or "-" if not <br />
x is epoch until which the entry is locked, 0 for non expiring lock <br />
y is the link count<br />
rep rm <pnfsid> [-force] # removes the pnfsfile from the cache<br />
rep rmclass <storageClass> # removes the from the cache<br />
rep set broken <pnfsid><br />
rep set cached <pnfsId> # DON'T USE, Potentially dangerous<br />
rep set precious <pnfsId><br />
rep set sticky [OPTIONS] [<pnfsid>] on|off # change sticky flags<br />
rep sticky ls <pnfsid></div>Mcaubethttps://pwiki.pic.es/index.php?title=DCache_Administration_Console_Commands&diff=195DCache Administration Console Commands2017-07-31T09:37:47Z<p>Mcaubet: /* Command help */</p>
<hr />
<div>= About this document =<br />
* This document shows several examples of useful commands that can be run in the dCache Administration Console.<br />
* Administration console can be accessed through '''[[https://www.dcache.org/downloads/gui/ PCells GUI]]''' or through '''SSHv2''' as follows (in the example, '''dccore.pic.es''' is the physical machine running the '''admin''' cell):<br />
ssh -2 -x -o StrictHostKeyChecking=no -l admin -c blowfish -p 22224 dccore.pic.es<br />
* In this document we show what we consider the most important commands for the everyday's administration tasks, and basically the ones that may be useful for the Manager On Duty. Maybe some important commands are missing, but will be added as soon as we detect that these may be important for the MoD.<br />
* This document is divided in two main blocks:<br />
# '''Pool commands''': shows useful commands that can be run on pools.<br />
# '''Service commands''': show useful commands that can be run on servers (this will include doors)<br />
= The basics =<br />
* If you access the Administration Console through '''SSHv2''' you should access to the different cells as follows:<br />
\? [command]... # display help for shell commands<br />
\c cell[@domain] [user] # connect to cell<br />
\exception [trace] # controls display of stack traces<br />
\h [command]... # display help for cell commands<br />
\l [cell[@domain]|pool/poolgroup]... # list cells<br />
\q # quit<br />
'''\s (cell[@domain]|pool/poolgroup)[,(cell[@domain]|pool/poolgroup)]... command... # send command'''<br />
\sl pnfsid|path command... # send to locations<br />
\sn command... # send pnfsmanager command<br />
\sp command... # send poolmanager command<br />
\timeout [seconds] # sets the command timeout<br />
* With '''\s''' you will be able to access to any '''cell[@domain]'''s, while branches of this command such like '''\sn''' or '''\sp''' are useful to (respectively) direct access to the ''namespace'' or ''poolmanager'' cells without the need of specifying the ''cell[@domain]''.<br />
<br />
= Pool commands =<br />
== Introduction ==<br />
* Any command run on a pool is a temporary change: on any pool service restart configuration will be restored back to the default value or to the value set in the '''setup''' file (locate in '''/dcpool*/vpool*/setup''').<br />
:* In example, whenever we set a pool to ''rdonly'', on restart, once the ''initialization'' process finishes, pool will switch back to ''enabled''.<br />
* If you want to make permanent changes, you should '''save''' changes. Running the '''save''' command will update the already mentioned '''setup''' file.<br />
:* In example, whenever we set a pool to ''rdonly'' '''and''' we ''save'' the configuration, on a restart, once the ''initialization'' process finishes, pool will remain ''rdonly''.<br />
<br />
== Disable / enable pools ==<br />
=== Command help ===<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? help hsm'''<br />
pool disable [options] [<errorCode> [<errorMessage>]] # suspend sending 'up messages'<br />
OPTIONS :<br />
-fetch # disallows fetch (transfer to client)<br />
-stage # disallows staging (from HSM)<br />
-store # disallows store (transfer from client)<br />
-p2p-client<br />
-rdonly # := store,stage,p2p-client<br />
-strict # := disallows everything<br />
pool enable # start sending 'up messages' again<br />
pool suppress hsmload on|off<br />
=== Disable / enable pool examples ===<br />
* Disallow store + stage + p2p-client (set pool in read only mode)<br />
'''''[dccore01] (local) admin >''''' '''pool disable -rdonly'''<br />
* Enable pool (set pool to RW)<br />
'''''[dccore01] (local) admin >''''' '''pool enable'''<br />
<br />
== HSM configuration ==<br />
=== Command help ===<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? help hsm'''<br />
hsm create <type> [<name> [<provider>]] [-<key>=<value>] ...<br />
hsm ls [<name>] ...<br />
hsm remove <name><br />
hsm set <name> [-<key>=<value>] ...<br />
hsm show providers # show available nearline storage providers<br />
hsm unset <name> [-<key>] ...<br />
=== HSM configuration examples ===<br />
* Create a ''enstore'' HSM with the ''real-encp.sh'' client script (this example can be found in the '''setup''' file):<br />
'''''[dccore01] (local) admin >''''' '''hsm create enstore enstore script -pnfs=/pnfs/fs/usr -command=/opt/enstore/bin/real-encp.sh -c:gets=200 -c:puts=20 -c:removes=2'''<br />
* Show ''enstore'' configuration:<br />
'''''[dccore01] (local) admin >''''' '''hsm ls enstore'''<br />
enstore(enstore):script<br />
pnfs /pnfs/fs/usr<br />
command /opt/enstore/bin/real-encp.sh<br />
c:gets 200<br />
c:puts 20<br />
c:removes 2<br />
* Set maximum restores to 400:<br />
'''''[dccore01] (local) admin >''''' '''hsm set enstore -c:gets=400'''<br />
<br />
== Mover management ==<br />
=== Command help ===<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? help hsm'''<br />
mover kill <jobid> # terminate a file transfer connection<br />
mover ls [-binary [jobId] ]<br />
mover queue ls [-l] # list all mover queues in this pool<br />
mover remove [<jobid>] # #OBSOLETE command<br />
mover set max active [OPTIONS] <maxactivemovers> # set the maximum number of active client transfers<br />
=== Mover management examples ===<br />
* Show current '''ftp''' transfers:<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? mover ls -l -queue=ftp'''<br />
33743014 : RUNNING : 0000D5B91029797545859181EB738C25E93C IoMode=READ h={SU=0;SA=0;S=None} bytes=12113543168 time/sec=3893 LM=0 si={vo-cms.cms}<br />
33743019 : RUNNING : 00008862CFF7A1C74513A617D364A4162DAA IoMode=READ h={SU=0;SA=0;S=None} bytes=10561388544 time/sec=3409 LM=0 si={vo-cms.cms}<br />
33743076 : RUNNING : 0000736CFC0DAF494816BFE7CC32D3192900 IoMode=READ h={SU=0;SA=0;S=None} bytes=2034630656 time/sec=2478 LM=0 si={vo-cms.cms}<br />
33743086 : RUNNING : 0000C5F1B9D6EEC04DEEA077CA6753E30DE0 IoMode=READ h={SU=0;SA=0;S=None} bytes=488898560 time/sec=1357 LM=1 si={vo-cms.cms}<br />
33743103 : RUNNING : 0000BBE684CB4BA44AB5ACB0D13C5DD25C8C IoMode=READ h={SU=0;SA=0;S=None} bytes=135921664 time/sec=239 LM=20 si={vo-cms.cms}<br />
* Show all mover queue properties:<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? mover queue ls -l'''<br />
regular 0 0 0<br />
p2p 0 40 0<br />
ftp 11 500 0<br />
dcap 0 10 0<br />
nfs4 0 100 0<br />
xrootd 406 1500 0<br />
xrootd-wan 0 700 0<br />
webdav 0 0 0<br />
* Set to 400 the maximum active client transfers for the '''ftp''' queue:<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? mover set max active -queue=ftp 400'''<br />
Max Active Io Movers set to 400<br />
<br />
== File / replica management ==<br />
=== Command help ===<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? help hsm'''<br />
rep ls [-l[=s,l,u,nc,p]] [-s[=kmgt]] [-storage=<glob>] | <pnfsId>...<br />
Format I : <pnfsId>...<br />
Format II : [-l[=<selectionOptions>]] [-storage=<glob>] [-s]<br />
Options :<br />
-l[=splunc] # selected list<br />
s : sticky files<br />
p : precious files<br />
l : locked files<br />
u : files in use<br />
nc : files which are not cached<br />
e : files which error condition<br />
-storage=<glob> # select only replicas<br />
of files with storage-<br />
info that matches <glob><br />
-s[=kmgt] [-sum] # statistics<br />
k : data amount in KBytes<br />
m : data amount in MBytes<br />
g : data amount in GBytes<br />
t : data amount in TBytes<br />
Output is a list of repository entries, one per line<br />
each line has the followin syntax:<br />
<pnfsid> <state> <size> <storageinfo><br />
state is a sequence of state bits inclosed in angular "<>" brackets <br />
bit 1 is "C" if entry is cached or "-" if not <br />
bit 2 is "P" if entry is precious or "-" if not <br />
bit 3 is "C" if entry is being transfered "from client" or "-" if not <br />
bit 4 is "S" if entry is being transfered "from store" or "-" if not <br />
bit 5 is "c" if entry is being transfered "to client" or "-" if not <br />
bit 6 is "s" if entry is being transfered "to store" or "-" if not <br />
bit 7 is "R" if entry is removed or "-" if not <br />
bit 8 is is always "-" <br />
bit 9 is "X" if entry is sticky or "-" if not <br />
bit 10 is "E" if entry is in error state or "-" if not <br />
bit 11 is "L(x)(y)" if entry is in locked or "-" if not <br />
x is epoch until which the entry is locked, 0 for non expiring lock <br />
y is the link count<br />
rep rm <pnfsid> [-force] # removes the pnfsfile from the cache<br />
rep rmclass <storageClass> # removes the from the cache<br />
rep set broken <pnfsid><br />
rep set cached <pnfsId> # DON'T USE, Potentially dangerous<br />
rep set precious <pnfsId><br />
rep set sticky [OPTIONS] [<pnfsid>] on|off # change sticky flags<br />
rep sticky ls <pnfsid></div>Mcaubethttps://pwiki.pic.es/index.php?title=DCache_Administration_Console_Commands&diff=194DCache Administration Console Commands2017-07-31T09:37:23Z<p>Mcaubet: /* Mover management */</p>
<hr />
<div>= About this document =<br />
* This document shows several examples of useful commands that can be run in the dCache Administration Console.<br />
* Administration console can be accessed through '''[[https://www.dcache.org/downloads/gui/ PCells GUI]]''' or through '''SSHv2''' as follows (in the example, '''dccore.pic.es''' is the physical machine running the '''admin''' cell):<br />
ssh -2 -x -o StrictHostKeyChecking=no -l admin -c blowfish -p 22224 dccore.pic.es<br />
* In this document we show what we consider the most important commands for the everyday's administration tasks, and basically the ones that may be useful for the Manager On Duty. Maybe some important commands are missing, but will be added as soon as we detect that these may be important for the MoD.<br />
* This document is divided in two main blocks:<br />
# '''Pool commands''': shows useful commands that can be run on pools.<br />
# '''Service commands''': show useful commands that can be run on servers (this will include doors)<br />
= The basics =<br />
* If you access the Administration Console through '''SSHv2''' you should access to the different cells as follows:<br />
\? [command]... # display help for shell commands<br />
\c cell[@domain] [user] # connect to cell<br />
\exception [trace] # controls display of stack traces<br />
\h [command]... # display help for cell commands<br />
\l [cell[@domain]|pool/poolgroup]... # list cells<br />
\q # quit<br />
'''\s (cell[@domain]|pool/poolgroup)[,(cell[@domain]|pool/poolgroup)]... command... # send command'''<br />
\sl pnfsid|path command... # send to locations<br />
\sn command... # send pnfsmanager command<br />
\sp command... # send poolmanager command<br />
\timeout [seconds] # sets the command timeout<br />
* With '''\s''' you will be able to access to any '''cell[@domain]'''s, while branches of this command such like '''\sn''' or '''\sp''' are useful to (respectively) direct access to the ''namespace'' or ''poolmanager'' cells without the need of specifying the ''cell[@domain]''.<br />
<br />
= Pool commands =<br />
== Introduction ==<br />
* Any command run on a pool is a temporary change: on any pool service restart configuration will be restored back to the default value or to the value set in the '''setup''' file (locate in '''/dcpool*/vpool*/setup''').<br />
:* In example, whenever we set a pool to ''rdonly'', on restart, once the ''initialization'' process finishes, pool will switch back to ''enabled''.<br />
* If you want to make permanent changes, you should '''save''' changes. Running the '''save''' command will update the already mentioned '''setup''' file.<br />
:* In example, whenever we set a pool to ''rdonly'' '''and''' we ''save'' the configuration, on a restart, once the ''initialization'' process finishes, pool will remain ''rdonly''.<br />
<br />
== Disable / enable pools ==<br />
=== Command help ===<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? help hsm'''<br />
pool disable [options] [<errorCode> [<errorMessage>]] # suspend sending 'up messages'<br />
OPTIONS :<br />
-fetch # disallows fetch (transfer to client)<br />
-stage # disallows staging (from HSM)<br />
-store # disallows store (transfer from client)<br />
-p2p-client<br />
-rdonly # := store,stage,p2p-client<br />
-strict # := disallows everything<br />
pool enable # start sending 'up messages' again<br />
pool suppress hsmload on|off<br />
=== Disable / enable pool examples ===<br />
* Disallow store + stage + p2p-client (set pool in read only mode)<br />
'''''[dccore01] (local) admin >''''' '''pool disable -rdonly'''<br />
* Enable pool (set pool to RW)<br />
'''''[dccore01] (local) admin >''''' '''pool enable'''<br />
<br />
== HSM configuration ==<br />
=== Command help ===<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? help hsm'''<br />
hsm create <type> [<name> [<provider>]] [-<key>=<value>] ...<br />
hsm ls [<name>] ...<br />
hsm remove <name><br />
hsm set <name> [-<key>=<value>] ...<br />
hsm show providers # show available nearline storage providers<br />
hsm unset <name> [-<key>] ...<br />
=== HSM configuration examples ===<br />
* Create a ''enstore'' HSM with the ''real-encp.sh'' client script (this example can be found in the '''setup''' file):<br />
'''''[dccore01] (local) admin >''''' '''hsm create enstore enstore script -pnfs=/pnfs/fs/usr -command=/opt/enstore/bin/real-encp.sh -c:gets=200 -c:puts=20 -c:removes=2'''<br />
* Show ''enstore'' configuration:<br />
'''''[dccore01] (local) admin >''''' '''hsm ls enstore'''<br />
enstore(enstore):script<br />
pnfs /pnfs/fs/usr<br />
command /opt/enstore/bin/real-encp.sh<br />
c:gets 200<br />
c:puts 20<br />
c:removes 2<br />
* Set maximum restores to 400:<br />
'''''[dccore01] (local) admin >''''' '''hsm set enstore -c:gets=400'''<br />
<br />
== Mover management ==<br />
=== Command help ===<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? help hsm'''<br />
mover kill <jobid> # terminate a file transfer connection<br />
mover ls [-binary [jobId] ]<br />
mover queue ls [-l] # list all mover queues in this pool<br />
mover remove [<jobid>] # #OBSOLETE command<br />
mover set max active [OPTIONS] <maxactivemovers> # set the maximum number of active client transfers<br />
=== Mover management examples ===<br />
* Show current '''ftp''' transfers:<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? mover ls -l -queue=ftp'''<br />
33743014 : RUNNING : 0000D5B91029797545859181EB738C25E93C IoMode=READ h={SU=0;SA=0;S=None} bytes=12113543168 time/sec=3893 LM=0 si={vo-cms.cms}<br />
33743019 : RUNNING : 00008862CFF7A1C74513A617D364A4162DAA IoMode=READ h={SU=0;SA=0;S=None} bytes=10561388544 time/sec=3409 LM=0 si={vo-cms.cms}<br />
33743076 : RUNNING : 0000736CFC0DAF494816BFE7CC32D3192900 IoMode=READ h={SU=0;SA=0;S=None} bytes=2034630656 time/sec=2478 LM=0 si={vo-cms.cms}<br />
33743086 : RUNNING : 0000C5F1B9D6EEC04DEEA077CA6753E30DE0 IoMode=READ h={SU=0;SA=0;S=None} bytes=488898560 time/sec=1357 LM=1 si={vo-cms.cms}<br />
33743103 : RUNNING : 0000BBE684CB4BA44AB5ACB0D13C5DD25C8C IoMode=READ h={SU=0;SA=0;S=None} bytes=135921664 time/sec=239 LM=20 si={vo-cms.cms}<br />
* Show all mover queue properties:<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? mover queue ls -l'''<br />
regular 0 0 0<br />
p2p 0 40 0<br />
ftp 11 500 0<br />
dcap 0 10 0<br />
nfs4 0 100 0<br />
xrootd 406 1500 0<br />
xrootd-wan 0 700 0<br />
webdav 0 0 0<br />
* Set to 400 the maximum active client transfers for the '''ftp''' queue:<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? mover set max active -queue=ftp 400'''<br />
Max Active Io Movers set to 400<br />
<br />
== File / replica management ==<br />
=== Command help ===<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? help hsm'''<br />
rep ls [-l[=s,l,u,nc,p]] [-s[=kmgt]] [-storage=<glob>] | <pnfsId>...<br />
Format I : <pnfsId>...<br />
Format II : [-l[=<selectionOptions>]] [-storage=<glob>] [-s]<br />
Options :<br />
-l[=splunc] # selected list<br />
s : sticky files<br />
p : precious files<br />
l : locked files<br />
u : files in use<br />
nc : files which are not cached<br />
e : files which error condition<br />
-storage=<glob> # select only replicas<br />
of files with storage-<br />
info that matches <glob><br />
-s[=kmgt] [-sum] # statistics<br />
k : data amount in KBytes<br />
m : data amount in MBytes<br />
g : data amount in GBytes<br />
t : data amount in TBytes<br />
Output is a list of repository entries, one per line<br />
each line has the followin syntax:<br />
<pnfsid> <state> <size> <storageinfo><br />
state is a sequence of state bits inclosed in angular "<>" brackets <br />
bit 1 is "C" if entry is cached or "-" if not <br />
bit 2 is "P" if entry is precious or "-" if not <br />
bit 3 is "C" if entry is being transfered "from client" or "-" if not <br />
bit 4 is "S" if entry is being transfered "from store" or "-" if not <br />
bit 5 is "c" if entry is being transfered "to client" or "-" if not <br />
bit 6 is "s" if entry is being transfered "to store" or "-" if not <br />
bit 7 is "R" if entry is removed or "-" if not <br />
bit 8 is is always "-" <br />
bit 9 is "X" if entry is sticky or "-" if not <br />
bit 10 is "E" if entry is in error state or "-" if not <br />
bit 11 is "L(x)(y)" if entry is in locked or "-" if not <br />
x is epoch until which the entry is locked, 0 for non expiring lock <br />
y is the link count<br />
rep rm <pnfsid> [-force] # removes the pnfsfile from the cache<br />
rep rmclass <storageClass> # removes the from the cache<br />
rep set broken <pnfsid><br />
rep set cached <pnfsId> # DON'T USE, Potentially dangerous<br />
rep set precious <pnfsId><br />
rep set sticky [OPTIONS] [<pnfsid>] on|off # change sticky flags<br />
rep sticky ls <pnfsid></div>Mcaubethttps://pwiki.pic.es/index.php?title=DCache_Administration_Console_Commands&diff=193DCache Administration Console Commands2017-07-31T09:35:16Z<p>Mcaubet: </p>
<hr />
<div>= About this document =<br />
* This document shows several examples of useful commands that can be run in the dCache Administration Console.<br />
* Administration console can be accessed through '''[[https://www.dcache.org/downloads/gui/ PCells GUI]]''' or through '''SSHv2''' as follows (in the example, '''dccore.pic.es''' is the physical machine running the '''admin''' cell):<br />
ssh -2 -x -o StrictHostKeyChecking=no -l admin -c blowfish -p 22224 dccore.pic.es<br />
* In this document we show what we consider the most important commands for the everyday's administration tasks, and basically the ones that may be useful for the Manager On Duty. Maybe some important commands are missing, but will be added as soon as we detect that these may be important for the MoD.<br />
* This document is divided in two main blocks:<br />
# '''Pool commands''': shows useful commands that can be run on pools.<br />
# '''Service commands''': show useful commands that can be run on servers (this will include doors)<br />
= The basics =<br />
* If you access the Administration Console through '''SSHv2''' you should access to the different cells as follows:<br />
\? [command]... # display help for shell commands<br />
\c cell[@domain] [user] # connect to cell<br />
\exception [trace] # controls display of stack traces<br />
\h [command]... # display help for cell commands<br />
\l [cell[@domain]|pool/poolgroup]... # list cells<br />
\q # quit<br />
'''\s (cell[@domain]|pool/poolgroup)[,(cell[@domain]|pool/poolgroup)]... command... # send command'''<br />
\sl pnfsid|path command... # send to locations<br />
\sn command... # send pnfsmanager command<br />
\sp command... # send poolmanager command<br />
\timeout [seconds] # sets the command timeout<br />
* With '''\s''' you will be able to access to any '''cell[@domain]'''s, while branches of this command such like '''\sn''' or '''\sp''' are useful to (respectively) direct access to the ''namespace'' or ''poolmanager'' cells without the need of specifying the ''cell[@domain]''.<br />
<br />
= Pool commands =<br />
== Introduction ==<br />
* Any command run on a pool is a temporary change: on any pool service restart configuration will be restored back to the default value or to the value set in the '''setup''' file (locate in '''/dcpool*/vpool*/setup''').<br />
:* In example, whenever we set a pool to ''rdonly'', on restart, once the ''initialization'' process finishes, pool will switch back to ''enabled''.<br />
* If you want to make permanent changes, you should '''save''' changes. Running the '''save''' command will update the already mentioned '''setup''' file.<br />
:* In example, whenever we set a pool to ''rdonly'' '''and''' we ''save'' the configuration, on a restart, once the ''initialization'' process finishes, pool will remain ''rdonly''.<br />
<br />
== Disable / enable pools ==<br />
=== Command help ===<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? help hsm'''<br />
pool disable [options] [<errorCode> [<errorMessage>]] # suspend sending 'up messages'<br />
OPTIONS :<br />
-fetch # disallows fetch (transfer to client)<br />
-stage # disallows staging (from HSM)<br />
-store # disallows store (transfer from client)<br />
-p2p-client<br />
-rdonly # := store,stage,p2p-client<br />
-strict # := disallows everything<br />
pool enable # start sending 'up messages' again<br />
pool suppress hsmload on|off<br />
=== Disable / enable pool examples ===<br />
* Disallow store + stage + p2p-client (set pool in read only mode)<br />
'''''[dccore01] (local) admin >''''' '''pool disable -rdonly'''<br />
* Enable pool (set pool to RW)<br />
'''''[dccore01] (local) admin >''''' '''pool enable'''<br />
<br />
== HSM configuration ==<br />
=== Command help ===<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? help hsm'''<br />
hsm create <type> [<name> [<provider>]] [-<key>=<value>] ...<br />
hsm ls [<name>] ...<br />
hsm remove <name><br />
hsm set <name> [-<key>=<value>] ...<br />
hsm show providers # show available nearline storage providers<br />
hsm unset <name> [-<key>] ...<br />
=== HSM configuration examples ===<br />
* Create a ''enstore'' HSM with the ''real-encp.sh'' client script (this example can be found in the '''setup''' file):<br />
'''''[dccore01] (local) admin >''''' '''hsm create enstore enstore script -pnfs=/pnfs/fs/usr -command=/opt/enstore/bin/real-encp.sh -c:gets=200 -c:puts=20 -c:removes=2'''<br />
* Show ''enstore'' configuration:<br />
'''''[dccore01] (local) admin >''''' '''hsm ls enstore'''<br />
enstore(enstore):script<br />
pnfs /pnfs/fs/usr<br />
command /opt/enstore/bin/real-encp.sh<br />
c:gets 200<br />
c:puts 20<br />
c:removes 2<br />
* Set maximum restores to 400:<br />
'''''[dccore01] (local) admin >''''' '''hsm set enstore -c:gets=400'''<br />
<br />
== Mover management ==<br />
=== Command help ===<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? help hsm'''<br />
mover kill <jobid> # terminate a file transfer connection<br />
mover ls [-binary [jobId] ]<br />
mover queue ls [-l] # list all mover queues in this pool<br />
mover remove [<jobid>] # #OBSOLETE command<br />
mover set max active [OPTIONS] <maxactivemovers> # set the maximum number of active client transfers<br />
=== Mover management examples ===<br />
* Show current '''ftp''' transfers:<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? mover ls -l -queue=ftp'''<br />
33743014 : RUNNING : 0000D5B91029797545859181EB738C25E93C IoMode=READ h={SU=0;SA=0;S=None} bytes=12113543168 time/sec=3893 LM=0 si={vo-cms.cms}<br />
33743019 : RUNNING : 00008862CFF7A1C74513A617D364A4162DAA IoMode=READ h={SU=0;SA=0;S=None} bytes=10561388544 time/sec=3409 LM=0 si={vo-cms.cms}<br />
33743076 : RUNNING : 0000736CFC0DAF494816BFE7CC32D3192900 IoMode=READ h={SU=0;SA=0;S=None} bytes=2034630656 time/sec=2478 LM=0 si={vo-cms.cms}<br />
33743086 : RUNNING : 0000C5F1B9D6EEC04DEEA077CA6753E30DE0 IoMode=READ h={SU=0;SA=0;S=None} bytes=488898560 time/sec=1357 LM=1 si={vo-cms.cms}<br />
33743103 : RUNNING : 0000BBE684CB4BA44AB5ACB0D13C5DD25C8C IoMode=READ h={SU=0;SA=0;S=None} bytes=135921664 time/sec=239 LM=20 si={vo-cms.cms}<br />
* Show all mover queue properties:<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? mover queue ls -l'''<br />
regular 0 0 0<br />
p2p 0 40 0<br />
ftp 11 500 0<br />
dcap 0 10 0<br />
nfs4 0 100 0<br />
xrootd 406 1500 0<br />
xrootd-wan 0 700 0<br />
webdav 0 0 0<br />
* Set to 400 the maximum active client transfers for the '''ftp''' queue:<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? mover set max active -queue=ftp 400'''<br />
Max Active Io Movers set to 400</div>Mcaubethttps://pwiki.pic.es/index.php?title=DCache_Administration_Console_Commands&diff=192DCache Administration Console Commands2017-07-31T09:28:17Z<p>Mcaubet: /* Disable / enable pool examples */</p>
<hr />
<div>= About this document =<br />
* This document shows several examples of useful commands that can be run in the dCache Administration Console.<br />
* Administration console can be accessed through '''[[https://www.dcache.org/downloads/gui/ PCells GUI]]''' or through '''SSHv2''' as follows (in the example, '''dccore.pic.es''' is the physical machine running the '''admin''' cell):<br />
ssh -2 -x -o StrictHostKeyChecking=no -l admin -c blowfish -p 22224 dccore.pic.es<br />
* In this document we show what we consider the most important commands for the everyday's administration tasks, and basically the ones that may be useful for the Manager On Duty. Maybe some important commands are missing, but will be added as soon as we detect that these may be important for the MoD.<br />
* This document is divided in two main blocks:<br />
# '''Pool commands''': shows useful commands that can be run on pools.<br />
# '''Service commands''': show useful commands that can be run on servers (this will include doors)<br />
= The basics =<br />
* If you access the Administration Console through '''SSHv2''' you should access to the different cells as follows:<br />
\? [command]... # display help for shell commands<br />
\c cell[@domain] [user] # connect to cell<br />
\exception [trace] # controls display of stack traces<br />
\h [command]... # display help for cell commands<br />
\l [cell[@domain]|pool/poolgroup]... # list cells<br />
\q # quit<br />
'''\s (cell[@domain]|pool/poolgroup)[,(cell[@domain]|pool/poolgroup)]... command... # send command'''<br />
\sl pnfsid|path command... # send to locations<br />
\sn command... # send pnfsmanager command<br />
\sp command... # send poolmanager command<br />
\timeout [seconds] # sets the command timeout<br />
* With '''\s''' you will be able to access to any '''cell[@domain]'''s, while branches of this command such like '''\sn''' or '''\sp''' are useful to (respectively) direct access to the ''namespace'' or ''poolmanager'' cells without the need of specifying the ''cell[@domain]''.<br />
<br />
= Pool commands =<br />
== Introduction ==<br />
* Any command run on a pool is a temporary change: on any pool service restart configuration will be restored back to the default value or to the value set in the '''setup''' file (locate in '''/dcpool*/vpool*/setup''').<br />
:* In example, whenever we set a pool to ''rdonly'', on restart, once the ''initialization'' process finishes, pool will switch back to ''enabled''.<br />
* If you want to make permanent changes, you should '''save''' changes. Running the '''save''' command will update the already mentioned '''setup''' file.<br />
:* In example, whenever we set a pool to ''rdonly'' '''and''' we ''save'' the configuration, on a restart, once the ''initialization'' process finishes, pool will remain ''rdonly''.<br />
<br />
== Disable / enable pools ==<br />
=== Command help ===<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? help hsm'''<br />
pool disable [options] [<errorCode> [<errorMessage>]] # suspend sending 'up messages'<br />
OPTIONS :<br />
-fetch # disallows fetch (transfer to client)<br />
-stage # disallows staging (from HSM)<br />
-store # disallows store (transfer from client)<br />
-p2p-client<br />
-rdonly # := store,stage,p2p-client<br />
-strict # := disallows everything<br />
pool enable # start sending 'up messages' again<br />
pool suppress hsmload on|off<br />
=== Disable / enable pool examples ===<br />
* Disallow store + stage + p2p-client (set pool in read only mode)<br />
'''''[dccore01] (local) admin >''''' '''pool disable -rdonly'''<br />
* Enable pool (set pool to RW)<br />
'''''[dccore01] (local) admin >''''' '''pool enable'''<br />
<br />
== HSM configuration ==<br />
=== Command help ===<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? help hsm'''<br />
hsm create <type> [<name> [<provider>]] [-<key>=<value>] ...<br />
hsm ls [<name>] ...<br />
hsm remove <name><br />
hsm set <name> [-<key>=<value>] ...<br />
hsm show providers # show available nearline storage providers<br />
hsm unset <name> [-<key>] ...<br />
=== HSM configuration examples ===<br />
:* Create a ''enstore'' HSM with the ''real-encp.sh'' client script (this example can be found in the '''setup''' file):<br />
'''''[dccore01] (local) admin >''''' '''hsm create enstore enstore script -pnfs=/pnfs/fs/usr -command=/opt/enstore/bin/real-encp.sh -c:gets=200 -c:puts=20 -c:removes=2'''<br />
:* Show ''enstore'' configuration:<br />
'''''[dccore01] (local) admin >''''' '''hsm ls enstore'''<br />
enstore(enstore):script<br />
pnfs /pnfs/fs/usr<br />
command /opt/enstore/bin/real-encp.sh<br />
c:gets 200<br />
c:puts 20<br />
c:removes 2<br />
:* Set maximum restores to 400:<br />
'''''[dccore01] (local) admin >''''' '''hsm set enstore -c:gets=400'''<br />
<br />
== Mover management ==<br />
=== Command help ===<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? help hsm'''<br />
mover kill <jobid> # terminate a file transfer connection<br />
mover ls [-binary [jobId] ]<br />
mover queue ls [-l] # list all mover queues in this pool<br />
mover remove [<jobid>] # #OBSOLETE command<br />
mover set max active [OPTIONS] <maxactivemovers> # set the maximum number of active client transfers<br />
=== Mover management examples ===</div>Mcaubethttps://pwiki.pic.es/index.php?title=DCache_Administration_Console_Commands&diff=191DCache Administration Console Commands2017-07-31T09:27:56Z<p>Mcaubet: /* Disable / enable pool examples */</p>
<hr />
<div>= About this document =<br />
* This document shows several examples of useful commands that can be run in the dCache Administration Console.<br />
* Administration console can be accessed through '''[[https://www.dcache.org/downloads/gui/ PCells GUI]]''' or through '''SSHv2''' as follows (in the example, '''dccore.pic.es''' is the physical machine running the '''admin''' cell):<br />
ssh -2 -x -o StrictHostKeyChecking=no -l admin -c blowfish -p 22224 dccore.pic.es<br />
* In this document we show what we consider the most important commands for the everyday's administration tasks, and basically the ones that may be useful for the Manager On Duty. Maybe some important commands are missing, but will be added as soon as we detect that these may be important for the MoD.<br />
* This document is divided in two main blocks:<br />
# '''Pool commands''': shows useful commands that can be run on pools.<br />
# '''Service commands''': show useful commands that can be run on servers (this will include doors)<br />
= The basics =<br />
* If you access the Administration Console through '''SSHv2''' you should access to the different cells as follows:<br />
\? [command]... # display help for shell commands<br />
\c cell[@domain] [user] # connect to cell<br />
\exception [trace] # controls display of stack traces<br />
\h [command]... # display help for cell commands<br />
\l [cell[@domain]|pool/poolgroup]... # list cells<br />
\q # quit<br />
'''\s (cell[@domain]|pool/poolgroup)[,(cell[@domain]|pool/poolgroup)]... command... # send command'''<br />
\sl pnfsid|path command... # send to locations<br />
\sn command... # send pnfsmanager command<br />
\sp command... # send poolmanager command<br />
\timeout [seconds] # sets the command timeout<br />
* With '''\s''' you will be able to access to any '''cell[@domain]'''s, while branches of this command such like '''\sn''' or '''\sp''' are useful to (respectively) direct access to the ''namespace'' or ''poolmanager'' cells without the need of specifying the ''cell[@domain]''.<br />
<br />
= Pool commands =<br />
== Introduction ==<br />
* Any command run on a pool is a temporary change: on any pool service restart configuration will be restored back to the default value or to the value set in the '''setup''' file (locate in '''/dcpool*/vpool*/setup''').<br />
:* In example, whenever we set a pool to ''rdonly'', on restart, once the ''initialization'' process finishes, pool will switch back to ''enabled''.<br />
* If you want to make permanent changes, you should '''save''' changes. Running the '''save''' command will update the already mentioned '''setup''' file.<br />
:* In example, whenever we set a pool to ''rdonly'' '''and''' we ''save'' the configuration, on a restart, once the ''initialization'' process finishes, pool will remain ''rdonly''.<br />
<br />
== Disable / enable pools ==<br />
=== Command help ===<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? help hsm'''<br />
pool disable [options] [<errorCode> [<errorMessage>]] # suspend sending 'up messages'<br />
OPTIONS :<br />
-fetch # disallows fetch (transfer to client)<br />
-stage # disallows staging (from HSM)<br />
-store # disallows store (transfer from client)<br />
-p2p-client<br />
-rdonly # := store,stage,p2p-client<br />
-strict # := disallows everything<br />
pool enable # start sending 'up messages' again<br />
pool suppress hsmload on|off<br />
=== Disable / enable pool examples ===<br />
====== Disallow store + stage + p2p-client (set pool in read only mode) ======<br />
'''''[dccore01] (local) admin >''''' '''pool disable -rdonly'''<br />
===== Enable pool (set pool to RW) =====<br />
'''''[dccore01] (local) admin >''''' '''pool enable'''<br />
<br />
== HSM configuration ==<br />
=== Command help ===<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? help hsm'''<br />
hsm create <type> [<name> [<provider>]] [-<key>=<value>] ...<br />
hsm ls [<name>] ...<br />
hsm remove <name><br />
hsm set <name> [-<key>=<value>] ...<br />
hsm show providers # show available nearline storage providers<br />
hsm unset <name> [-<key>] ...<br />
=== HSM configuration examples ===<br />
:* Create a ''enstore'' HSM with the ''real-encp.sh'' client script (this example can be found in the '''setup''' file):<br />
'''''[dccore01] (local) admin >''''' '''hsm create enstore enstore script -pnfs=/pnfs/fs/usr -command=/opt/enstore/bin/real-encp.sh -c:gets=200 -c:puts=20 -c:removes=2'''<br />
:* Show ''enstore'' configuration:<br />
'''''[dccore01] (local) admin >''''' '''hsm ls enstore'''<br />
enstore(enstore):script<br />
pnfs /pnfs/fs/usr<br />
command /opt/enstore/bin/real-encp.sh<br />
c:gets 200<br />
c:puts 20<br />
c:removes 2<br />
:* Set maximum restores to 400:<br />
'''''[dccore01] (local) admin >''''' '''hsm set enstore -c:gets=400'''<br />
<br />
== Mover management ==<br />
=== Command help ===<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? help hsm'''<br />
mover kill <jobid> # terminate a file transfer connection<br />
mover ls [-binary [jobId] ]<br />
mover queue ls [-l] # list all mover queues in this pool<br />
mover remove [<jobid>] # #OBSOLETE command<br />
mover set max active [OPTIONS] <maxactivemovers> # set the maximum number of active client transfers<br />
=== Mover management examples ===</div>Mcaubethttps://pwiki.pic.es/index.php?title=DCache_Administration_Console_Commands&diff=190DCache Administration Console Commands2017-07-31T09:27:20Z<p>Mcaubet: /* Disable / enable pool examples */</p>
<hr />
<div>= About this document =<br />
* This document shows several examples of useful commands that can be run in the dCache Administration Console.<br />
* Administration console can be accessed through '''[[https://www.dcache.org/downloads/gui/ PCells GUI]]''' or through '''SSHv2''' as follows (in the example, '''dccore.pic.es''' is the physical machine running the '''admin''' cell):<br />
ssh -2 -x -o StrictHostKeyChecking=no -l admin -c blowfish -p 22224 dccore.pic.es<br />
* In this document we show what we consider the most important commands for the everyday's administration tasks, and basically the ones that may be useful for the Manager On Duty. Maybe some important commands are missing, but will be added as soon as we detect that these may be important for the MoD.<br />
* This document is divided in two main blocks:<br />
# '''Pool commands''': shows useful commands that can be run on pools.<br />
# '''Service commands''': show useful commands that can be run on servers (this will include doors)<br />
= The basics =<br />
* If you access the Administration Console through '''SSHv2''' you should access to the different cells as follows:<br />
\? [command]... # display help for shell commands<br />
\c cell[@domain] [user] # connect to cell<br />
\exception [trace] # controls display of stack traces<br />
\h [command]... # display help for cell commands<br />
\l [cell[@domain]|pool/poolgroup]... # list cells<br />
\q # quit<br />
'''\s (cell[@domain]|pool/poolgroup)[,(cell[@domain]|pool/poolgroup)]... command... # send command'''<br />
\sl pnfsid|path command... # send to locations<br />
\sn command... # send pnfsmanager command<br />
\sp command... # send poolmanager command<br />
\timeout [seconds] # sets the command timeout<br />
* With '''\s''' you will be able to access to any '''cell[@domain]'''s, while branches of this command such like '''\sn''' or '''\sp''' are useful to (respectively) direct access to the ''namespace'' or ''poolmanager'' cells without the need of specifying the ''cell[@domain]''.<br />
<br />
= Pool commands =<br />
== Introduction ==<br />
* Any command run on a pool is a temporary change: on any pool service restart configuration will be restored back to the default value or to the value set in the '''setup''' file (locate in '''/dcpool*/vpool*/setup''').<br />
:* In example, whenever we set a pool to ''rdonly'', on restart, once the ''initialization'' process finishes, pool will switch back to ''enabled''.<br />
* If you want to make permanent changes, you should '''save''' changes. Running the '''save''' command will update the already mentioned '''setup''' file.<br />
:* In example, whenever we set a pool to ''rdonly'' '''and''' we ''save'' the configuration, on a restart, once the ''initialization'' process finishes, pool will remain ''rdonly''.<br />
<br />
== Disable / enable pools ==<br />
=== Command help ===<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? help hsm'''<br />
pool disable [options] [<errorCode> [<errorMessage>]] # suspend sending 'up messages'<br />
OPTIONS :<br />
-fetch # disallows fetch (transfer to client)<br />
-stage # disallows staging (from HSM)<br />
-store # disallows store (transfer from client)<br />
-p2p-client<br />
-rdonly # := store,stage,p2p-client<br />
-strict # := disallows everything<br />
pool enable # start sending 'up messages' again<br />
pool suppress hsmload on|off<br />
=== Disable / enable pool examples ===<br />
===== Disallow store + stage + p2p-client (set pool in read only mode) =====<br />
'''''[dccore01] (local) admin >''''' '''pool disable -rdonly'''<br />
===== Enable pool (set pool to RW) =====<br />
'''''[dccore01] (local) admin >''''' '''pool enable'''<br />
<br />
== HSM configuration ==<br />
=== Command help ===<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? help hsm'''<br />
hsm create <type> [<name> [<provider>]] [-<key>=<value>] ...<br />
hsm ls [<name>] ...<br />
hsm remove <name><br />
hsm set <name> [-<key>=<value>] ...<br />
hsm show providers # show available nearline storage providers<br />
hsm unset <name> [-<key>] ...<br />
=== HSM configuration examples ===<br />
:* Create a ''enstore'' HSM with the ''real-encp.sh'' client script (this example can be found in the '''setup''' file):<br />
'''''[dccore01] (local) admin >''''' '''hsm create enstore enstore script -pnfs=/pnfs/fs/usr -command=/opt/enstore/bin/real-encp.sh -c:gets=200 -c:puts=20 -c:removes=2'''<br />
:* Show ''enstore'' configuration:<br />
'''''[dccore01] (local) admin >''''' '''hsm ls enstore'''<br />
enstore(enstore):script<br />
pnfs /pnfs/fs/usr<br />
command /opt/enstore/bin/real-encp.sh<br />
c:gets 200<br />
c:puts 20<br />
c:removes 2<br />
:* Set maximum restores to 400:<br />
'''''[dccore01] (local) admin >''''' '''hsm set enstore -c:gets=400'''<br />
<br />
== Mover management ==<br />
=== Command help ===<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? help hsm'''<br />
mover kill <jobid> # terminate a file transfer connection<br />
mover ls [-binary [jobId] ]<br />
mover queue ls [-l] # list all mover queues in this pool<br />
mover remove [<jobid>] # #OBSOLETE command<br />
mover set max active [OPTIONS] <maxactivemovers> # set the maximum number of active client transfers<br />
=== Mover management examples ===</div>Mcaubethttps://pwiki.pic.es/index.php?title=DCache_Administration_Console_Commands&diff=189DCache Administration Console Commands2017-07-31T09:26:51Z<p>Mcaubet: /* Mover management */</p>
<hr />
<div>= About this document =<br />
* This document shows several examples of useful commands that can be run in the dCache Administration Console.<br />
* Administration console can be accessed through '''[[https://www.dcache.org/downloads/gui/ PCells GUI]]''' or through '''SSHv2''' as follows (in the example, '''dccore.pic.es''' is the physical machine running the '''admin''' cell):<br />
ssh -2 -x -o StrictHostKeyChecking=no -l admin -c blowfish -p 22224 dccore.pic.es<br />
* In this document we show what we consider the most important commands for the everyday's administration tasks, and basically the ones that may be useful for the Manager On Duty. Maybe some important commands are missing, but will be added as soon as we detect that these may be important for the MoD.<br />
* This document is divided in two main blocks:<br />
# '''Pool commands''': shows useful commands that can be run on pools.<br />
# '''Service commands''': show useful commands that can be run on servers (this will include doors)<br />
= The basics =<br />
* If you access the Administration Console through '''SSHv2''' you should access to the different cells as follows:<br />
\? [command]... # display help for shell commands<br />
\c cell[@domain] [user] # connect to cell<br />
\exception [trace] # controls display of stack traces<br />
\h [command]... # display help for cell commands<br />
\l [cell[@domain]|pool/poolgroup]... # list cells<br />
\q # quit<br />
'''\s (cell[@domain]|pool/poolgroup)[,(cell[@domain]|pool/poolgroup)]... command... # send command'''<br />
\sl pnfsid|path command... # send to locations<br />
\sn command... # send pnfsmanager command<br />
\sp command... # send poolmanager command<br />
\timeout [seconds] # sets the command timeout<br />
* With '''\s''' you will be able to access to any '''cell[@domain]'''s, while branches of this command such like '''\sn''' or '''\sp''' are useful to (respectively) direct access to the ''namespace'' or ''poolmanager'' cells without the need of specifying the ''cell[@domain]''.<br />
<br />
= Pool commands =<br />
== Introduction ==<br />
* Any command run on a pool is a temporary change: on any pool service restart configuration will be restored back to the default value or to the value set in the '''setup''' file (locate in '''/dcpool*/vpool*/setup''').<br />
:* In example, whenever we set a pool to ''rdonly'', on restart, once the ''initialization'' process finishes, pool will switch back to ''enabled''.<br />
* If you want to make permanent changes, you should '''save''' changes. Running the '''save''' command will update the already mentioned '''setup''' file.<br />
:* In example, whenever we set a pool to ''rdonly'' '''and''' we ''save'' the configuration, on a restart, once the ''initialization'' process finishes, pool will remain ''rdonly''.<br />
<br />
== Disable / enable pools ==<br />
=== Command help ===<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? help hsm'''<br />
pool disable [options] [<errorCode> [<errorMessage>]] # suspend sending 'up messages'<br />
OPTIONS :<br />
-fetch # disallows fetch (transfer to client)<br />
-stage # disallows staging (from HSM)<br />
-store # disallows store (transfer from client)<br />
-p2p-client<br />
-rdonly # := store,stage,p2p-client<br />
-strict # := disallows everything<br />
pool enable # start sending 'up messages' again<br />
pool suppress hsmload on|off<br />
=== Disable / enable pool examples ===<br />
:* Disallow store + stage + p2p-client (set pool in read only mode)<br />
'''''[dccore01] (local) admin >''''' '''pool disable -rdonly'''<br />
:* Enable pool (set pool to RW)<br />
'''''[dccore01] (local) admin >''''' '''pool enable'''<br />
<br />
== HSM configuration ==<br />
=== Command help ===<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? help hsm'''<br />
hsm create <type> [<name> [<provider>]] [-<key>=<value>] ...<br />
hsm ls [<name>] ...<br />
hsm remove <name><br />
hsm set <name> [-<key>=<value>] ...<br />
hsm show providers # show available nearline storage providers<br />
hsm unset <name> [-<key>] ...<br />
=== HSM configuration examples ===<br />
:* Create a ''enstore'' HSM with the ''real-encp.sh'' client script (this example can be found in the '''setup''' file):<br />
'''''[dccore01] (local) admin >''''' '''hsm create enstore enstore script -pnfs=/pnfs/fs/usr -command=/opt/enstore/bin/real-encp.sh -c:gets=200 -c:puts=20 -c:removes=2'''<br />
:* Show ''enstore'' configuration:<br />
'''''[dccore01] (local) admin >''''' '''hsm ls enstore'''<br />
enstore(enstore):script<br />
pnfs /pnfs/fs/usr<br />
command /opt/enstore/bin/real-encp.sh<br />
c:gets 200<br />
c:puts 20<br />
c:removes 2<br />
:* Set maximum restores to 400:<br />
'''''[dccore01] (local) admin >''''' '''hsm set enstore -c:gets=400'''<br />
<br />
== Mover management ==<br />
=== Command help ===<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? help hsm'''<br />
mover kill <jobid> # terminate a file transfer connection<br />
mover ls [-binary [jobId] ]<br />
mover queue ls [-l] # list all mover queues in this pool<br />
mover remove [<jobid>] # #OBSOLETE command<br />
mover set max active [OPTIONS] <maxactivemovers> # set the maximum number of active client transfers<br />
=== Mover management examples ===</div>Mcaubethttps://pwiki.pic.es/index.php?title=DCache_Administration_Console_Commands&diff=188DCache Administration Console Commands2017-07-31T09:21:03Z<p>Mcaubet: /* Disable / enable pools */</p>
<hr />
<div>= About this document =<br />
* This document shows several examples of useful commands that can be run in the dCache Administration Console.<br />
* Administration console can be accessed through '''[[https://www.dcache.org/downloads/gui/ PCells GUI]]''' or through '''SSHv2''' as follows (in the example, '''dccore.pic.es''' is the physical machine running the '''admin''' cell):<br />
ssh -2 -x -o StrictHostKeyChecking=no -l admin -c blowfish -p 22224 dccore.pic.es<br />
* In this document we show what we consider the most important commands for the everyday's administration tasks, and basically the ones that may be useful for the Manager On Duty. Maybe some important commands are missing, but will be added as soon as we detect that these may be important for the MoD.<br />
* This document is divided in two main blocks:<br />
# '''Pool commands''': shows useful commands that can be run on pools.<br />
# '''Service commands''': show useful commands that can be run on servers (this will include doors)<br />
= The basics =<br />
* If you access the Administration Console through '''SSHv2''' you should access to the different cells as follows:<br />
\? [command]... # display help for shell commands<br />
\c cell[@domain] [user] # connect to cell<br />
\exception [trace] # controls display of stack traces<br />
\h [command]... # display help for cell commands<br />
\l [cell[@domain]|pool/poolgroup]... # list cells<br />
\q # quit<br />
'''\s (cell[@domain]|pool/poolgroup)[,(cell[@domain]|pool/poolgroup)]... command... # send command'''<br />
\sl pnfsid|path command... # send to locations<br />
\sn command... # send pnfsmanager command<br />
\sp command... # send poolmanager command<br />
\timeout [seconds] # sets the command timeout<br />
* With '''\s''' you will be able to access to any '''cell[@domain]'''s, while branches of this command such like '''\sn''' or '''\sp''' are useful to (respectively) direct access to the ''namespace'' or ''poolmanager'' cells without the need of specifying the ''cell[@domain]''.<br />
<br />
= Pool commands =<br />
== Introduction ==<br />
* Any command run on a pool is a temporary change: on any pool service restart configuration will be restored back to the default value or to the value set in the '''setup''' file (locate in '''/dcpool*/vpool*/setup''').<br />
:* In example, whenever we set a pool to ''rdonly'', on restart, once the ''initialization'' process finishes, pool will switch back to ''enabled''.<br />
* If you want to make permanent changes, you should '''save''' changes. Running the '''save''' command will update the already mentioned '''setup''' file.<br />
:* In example, whenever we set a pool to ''rdonly'' '''and''' we ''save'' the configuration, on a restart, once the ''initialization'' process finishes, pool will remain ''rdonly''.<br />
<br />
== Disable / enable pools ==<br />
=== Command help ===<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? help hsm'''<br />
pool disable [options] [<errorCode> [<errorMessage>]] # suspend sending 'up messages'<br />
OPTIONS :<br />
-fetch # disallows fetch (transfer to client)<br />
-stage # disallows staging (from HSM)<br />
-store # disallows store (transfer from client)<br />
-p2p-client<br />
-rdonly # := store,stage,p2p-client<br />
-strict # := disallows everything<br />
pool enable # start sending 'up messages' again<br />
pool suppress hsmload on|off<br />
=== Disable / enable pool examples ===<br />
:* Disallow store + stage + p2p-client (set pool in read only mode)<br />
'''''[dccore01] (local) admin >''''' '''pool disable -rdonly'''<br />
:* Enable pool (set pool to RW)<br />
'''''[dccore01] (local) admin >''''' '''pool enable'''<br />
<br />
== HSM configuration ==<br />
=== Command help ===<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? help hsm'''<br />
hsm create <type> [<name> [<provider>]] [-<key>=<value>] ...<br />
hsm ls [<name>] ...<br />
hsm remove <name><br />
hsm set <name> [-<key>=<value>] ...<br />
hsm show providers # show available nearline storage providers<br />
hsm unset <name> [-<key>] ...<br />
=== HSM configuration examples ===<br />
:* Create a ''enstore'' HSM with the ''real-encp.sh'' client script (this example can be found in the '''setup''' file):<br />
'''''[dccore01] (local) admin >''''' '''hsm create enstore enstore script -pnfs=/pnfs/fs/usr -command=/opt/enstore/bin/real-encp.sh -c:gets=200 -c:puts=20 -c:removes=2'''<br />
:* Show ''enstore'' configuration:<br />
'''''[dccore01] (local) admin >''''' '''hsm ls enstore'''<br />
enstore(enstore):script<br />
pnfs /pnfs/fs/usr<br />
command /opt/enstore/bin/real-encp.sh<br />
c:gets 200<br />
c:puts 20<br />
c:removes 2<br />
:* Set maximum restores to 400:<br />
'''''[dccore01] (local) admin >''''' '''hsm set enstore -c:gets=400'''<br />
<br />
== Mover management ==<br />
=== Command help ===<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? help hsm'''<br />
mover kill <jobid> # terminate a file transfer connection<br />
mover ls [-binary [jobId] ]<br />
mover queue ls [-l] # list all mover queues in this pool<br />
mover remove [<jobid>] # #OBSOLETE command<br />
mover set max active [OPTIONS] <maxactivemovers> # set the maximum number of active client transfers</div>Mcaubethttps://pwiki.pic.es/index.php?title=DCache_Administration_Console_Commands&diff=187DCache Administration Console Commands2017-07-31T09:20:51Z<p>Mcaubet: /* HSM configuration */</p>
<hr />
<div>= About this document =<br />
* This document shows several examples of useful commands that can be run in the dCache Administration Console.<br />
* Administration console can be accessed through '''[[https://www.dcache.org/downloads/gui/ PCells GUI]]''' or through '''SSHv2''' as follows (in the example, '''dccore.pic.es''' is the physical machine running the '''admin''' cell):<br />
ssh -2 -x -o StrictHostKeyChecking=no -l admin -c blowfish -p 22224 dccore.pic.es<br />
* In this document we show what we consider the most important commands for the everyday's administration tasks, and basically the ones that may be useful for the Manager On Duty. Maybe some important commands are missing, but will be added as soon as we detect that these may be important for the MoD.<br />
* This document is divided in two main blocks:<br />
# '''Pool commands''': shows useful commands that can be run on pools.<br />
# '''Service commands''': show useful commands that can be run on servers (this will include doors)<br />
= The basics =<br />
* If you access the Administration Console through '''SSHv2''' you should access to the different cells as follows:<br />
\? [command]... # display help for shell commands<br />
\c cell[@domain] [user] # connect to cell<br />
\exception [trace] # controls display of stack traces<br />
\h [command]... # display help for cell commands<br />
\l [cell[@domain]|pool/poolgroup]... # list cells<br />
\q # quit<br />
'''\s (cell[@domain]|pool/poolgroup)[,(cell[@domain]|pool/poolgroup)]... command... # send command'''<br />
\sl pnfsid|path command... # send to locations<br />
\sn command... # send pnfsmanager command<br />
\sp command... # send poolmanager command<br />
\timeout [seconds] # sets the command timeout<br />
* With '''\s''' you will be able to access to any '''cell[@domain]'''s, while branches of this command such like '''\sn''' or '''\sp''' are useful to (respectively) direct access to the ''namespace'' or ''poolmanager'' cells without the need of specifying the ''cell[@domain]''.<br />
<br />
= Pool commands =<br />
== Introduction ==<br />
* Any command run on a pool is a temporary change: on any pool service restart configuration will be restored back to the default value or to the value set in the '''setup''' file (locate in '''/dcpool*/vpool*/setup''').<br />
:* In example, whenever we set a pool to ''rdonly'', on restart, once the ''initialization'' process finishes, pool will switch back to ''enabled''.<br />
* If you want to make permanent changes, you should '''save''' changes. Running the '''save''' command will update the already mentioned '''setup''' file.<br />
:* In example, whenever we set a pool to ''rdonly'' '''and''' we ''save'' the configuration, on a restart, once the ''initialization'' process finishes, pool will remain ''rdonly''.<br />
<br />
== Disable / enable pools ==<br />
* Command help ('''\s dc???_? help hsm'''):<br />
pool disable [options] [<errorCode> [<errorMessage>]] # suspend sending 'up messages'<br />
OPTIONS :<br />
-fetch # disallows fetch (transfer to client)<br />
-stage # disallows staging (from HSM)<br />
-store # disallows store (transfer from client)<br />
-p2p-client<br />
-rdonly # := store,stage,p2p-client<br />
-strict # := disallows everything<br />
pool enable # start sending 'up messages' again<br />
pool suppress hsmload on|off<br />
=== Disable / enable pool examples ===<br />
:* Disallow store + stage + p2p-client (set pool in read only mode)<br />
'''''[dccore01] (local) admin >''''' '''pool disable -rdonly'''<br />
:* Enable pool (set pool to RW)<br />
'''''[dccore01] (local) admin >''''' '''pool enable'''<br />
<br />
== HSM configuration ==<br />
=== Command help ===<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? help hsm'''<br />
hsm create <type> [<name> [<provider>]] [-<key>=<value>] ...<br />
hsm ls [<name>] ...<br />
hsm remove <name><br />
hsm set <name> [-<key>=<value>] ...<br />
hsm show providers # show available nearline storage providers<br />
hsm unset <name> [-<key>] ...<br />
=== HSM configuration examples ===<br />
:* Create a ''enstore'' HSM with the ''real-encp.sh'' client script (this example can be found in the '''setup''' file):<br />
'''''[dccore01] (local) admin >''''' '''hsm create enstore enstore script -pnfs=/pnfs/fs/usr -command=/opt/enstore/bin/real-encp.sh -c:gets=200 -c:puts=20 -c:removes=2'''<br />
:* Show ''enstore'' configuration:<br />
'''''[dccore01] (local) admin >''''' '''hsm ls enstore'''<br />
enstore(enstore):script<br />
pnfs /pnfs/fs/usr<br />
command /opt/enstore/bin/real-encp.sh<br />
c:gets 200<br />
c:puts 20<br />
c:removes 2<br />
:* Set maximum restores to 400:<br />
'''''[dccore01] (local) admin >''''' '''hsm set enstore -c:gets=400'''<br />
<br />
== Mover management ==<br />
=== Command help ===<br />
'''''[dccore01] (local) admin >''''' '''\s dc???_? help hsm'''<br />
mover kill <jobid> # terminate a file transfer connection<br />
mover ls [-binary [jobId] ]<br />
mover queue ls [-l] # list all mover queues in this pool<br />
mover remove [<jobid>] # #OBSOLETE command<br />
mover set max active [OPTIONS] <maxactivemovers> # set the maximum number of active client transfers</div>Mcaubethttps://pwiki.pic.es/index.php?title=DCache_Administration_Console_Commands&diff=186DCache Administration Console Commands2017-07-31T09:16:24Z<p>Mcaubet: /* The basics */</p>
<hr />
<div>= About this document =<br />
* This document shows several examples of useful commands that can be run in the dCache Administration Console.<br />
* Administration console can be accessed through '''[[https://www.dcache.org/downloads/gui/ PCells GUI]]''' or through '''SSHv2''' as follows (in the example, '''dccore.pic.es''' is the physical machine running the '''admin''' cell):<br />
ssh -2 -x -o StrictHostKeyChecking=no -l admin -c blowfish -p 22224 dccore.pic.es<br />
* In this document we show what we consider the most important commands for the everyday's administration tasks, and basically the ones that may be useful for the Manager On Duty. Maybe some important commands are missing, but will be added as soon as we detect that these may be important for the MoD.<br />
* This document is divided in two main blocks:<br />
# '''Pool commands''': shows useful commands that can be run on pools.<br />
# '''Service commands''': show useful commands that can be run on servers (this will include doors)<br />
= The basics =<br />
* If you access the Administration Console through '''SSHv2''' you should access to the different cells as follows:<br />
\? [command]... # display help for shell commands<br />
\c cell[@domain] [user] # connect to cell<br />
\exception [trace] # controls display of stack traces<br />
\h [command]... # display help for cell commands<br />
\l [cell[@domain]|pool/poolgroup]... # list cells<br />
\q # quit<br />
'''\s (cell[@domain]|pool/poolgroup)[,(cell[@domain]|pool/poolgroup)]... command... # send command'''<br />
\sl pnfsid|path command... # send to locations<br />
\sn command... # send pnfsmanager command<br />
\sp command... # send poolmanager command<br />
\timeout [seconds] # sets the command timeout<br />
* With '''\s''' you will be able to access to any '''cell[@domain]'''s, while branches of this command such like '''\sn''' or '''\sp''' are useful to (respectively) direct access to the ''namespace'' or ''poolmanager'' cells without the need of specifying the ''cell[@domain]''.<br />
<br />
= Pool commands =<br />
== Introduction ==<br />
* Any command run on a pool is a temporary change: on any pool service restart configuration will be restored back to the default value or to the value set in the '''setup''' file (locate in '''/dcpool*/vpool*/setup''').<br />
:* In example, whenever we set a pool to ''rdonly'', on restart, once the ''initialization'' process finishes, pool will switch back to ''enabled''.<br />
* If you want to make permanent changes, you should '''save''' changes. Running the '''save''' command will update the already mentioned '''setup''' file.<br />
:* In example, whenever we set a pool to ''rdonly'' '''and''' we ''save'' the configuration, on a restart, once the ''initialization'' process finishes, pool will remain ''rdonly''.<br />
<br />
== Disable / enable pools ==<br />
* Command help ('''\s dc???_? help hsm'''):<br />
pool disable [options] [<errorCode> [<errorMessage>]] # suspend sending 'up messages'<br />
OPTIONS :<br />
-fetch # disallows fetch (transfer to client)<br />
-stage # disallows staging (from HSM)<br />
-store # disallows store (transfer from client)<br />
-p2p-client<br />
-rdonly # := store,stage,p2p-client<br />
-strict # := disallows everything<br />
pool enable # start sending 'up messages' again<br />
pool suppress hsmload on|off<br />
=== Disable / enable pool examples ===<br />
:* Disallow store + stage + p2p-client (set pool in read only mode)<br />
'''''[dccore01] (local) admin >''''' '''pool disable -rdonly'''<br />
:* Enable pool (set pool to RW)<br />
'''''[dccore01] (local) admin >''''' '''pool enable'''<br />
<br />
== HSM configuration ==<br />
* Command help ('''\s dc???_? help hsm'''):<br />
hsm create <type> [<name> [<provider>]] [-<key>=<value>] ...<br />
hsm ls [<name>] ...<br />
hsm remove <name><br />
hsm set <name> [-<key>=<value>] ...<br />
hsm show providers # show available nearline storage providers<br />
hsm unset <name> [-<key>] ...<br />
=== HSM configuration examples ===<br />
:* Create a ''enstore'' HSM with the ''real-encp.sh'' client script (this example can be found in the '''setup''' file):<br />
'''''[dccore01] (local) admin >''''' '''hsm create enstore enstore script -pnfs=/pnfs/fs/usr -command=/opt/enstore/bin/real-encp.sh -c:gets=200 -c:puts=20 -c:removes=2'''<br />
:* Show ''enstore'' configuration:<br />
'''''[dccore01] (local) admin >''''' '''hsm ls enstore'''<br />
enstore(enstore):script<br />
pnfs /pnfs/fs/usr<br />
command /opt/enstore/bin/real-encp.sh<br />
c:gets 200<br />
c:puts 20<br />
c:removes 2<br />
:* Set maximum restores to 400:<br />
'''''[dccore01] (local) admin >''''' '''hsm set enstore -c:gets=400'''</div>Mcaubethttps://pwiki.pic.es/index.php?title=DCache_Administration_Console_Commands&diff=185DCache Administration Console Commands2017-07-31T09:15:40Z<p>Mcaubet: /* The basics */</p>
<hr />
<div>= About this document =<br />
* This document shows several examples of useful commands that can be run in the dCache Administration Console.<br />
* Administration console can be accessed through '''[[https://www.dcache.org/downloads/gui/ PCells GUI]]''' or through '''SSHv2''' as follows (in the example, '''dccore.pic.es''' is the physical machine running the '''admin''' cell):<br />
ssh -2 -x -o StrictHostKeyChecking=no -l admin -c blowfish -p 22224 dccore.pic.es<br />
* In this document we show what we consider the most important commands for the everyday's administration tasks, and basically the ones that may be useful for the Manager On Duty. Maybe some important commands are missing, but will be added as soon as we detect that these may be important for the MoD.<br />
* This document is divided in two main blocks:<br />
# '''Pool commands''': shows useful commands that can be run on pools.<br />
# '''Service commands''': show useful commands that can be run on servers (this will include doors)<br />
= The basics =<br />
* If you access the Administration Console through '''SSHv2''' you should access to the different cells as follows:<br />
\? [command]... # display help for shell commands<br />
\c cell[@domain] [user] # connect to cell<br />
\exception [trace] # controls display of stack traces<br />
\h [command]... # display help for cell commands<br />
\l [cell[@domain]|pool/poolgroup]... # list cells<br />
\q # quit<br />
'''\s (cell[@domain]|pool/poolgroup)[,(cell[@domain]|pool/poolgroup)]... command... # send command'''<br />
\sl pnfsid|path command... # send to locations<br />
\sn command... # send pnfsmanager command<br />
\sp command... # send poolmanager command<br />
\timeout [seconds] # sets the command timeout<br />
* With '''\s''' you will be able to access to any '''cell[@domain]'''s, while branches of this command such like '''\sn''' or '''\sp''' are useful to (respectively) direct access to the ''namespace'' or ''poolmanager'' cells without the need of specifying the ''cell[@domain]''.<br />
<br />
== Pool commands ==<br />
=== Introduction ===<br />
* Any command run on a pool is a temporary change: on any pool service restart configuration will be restored back to the default value or to the value set in the '''setup''' file (locate in '''/dcpool*/vpool*/setup''').<br />
:* In example, whenever we set a pool to ''rdonly'', on restart, once the ''initialization'' process finishes, pool will switch back to ''enabled''.<br />
* If you want to make permanent changes, you should '''save''' changes. Running the '''save''' command will update the already mentioned '''setup''' file.<br />
:* In example, whenever we set a pool to ''rdonly'' '''and''' we ''save'' the configuration, on a restart, once the ''initialization'' process finishes, pool will remain ''rdonly''.<br />
<br />
=== Disable / enable pools ===<br />
* Command help ('''\s dc???_? help hsm'''):<br />
pool disable [options] [<errorCode> [<errorMessage>]] # suspend sending 'up messages'<br />
OPTIONS :<br />
-fetch # disallows fetch (transfer to client)<br />
-stage # disallows staging (from HSM)<br />
-store # disallows store (transfer from client)<br />
-p2p-client<br />
-rdonly # := store,stage,p2p-client<br />
-strict # := disallows everything<br />
pool enable # start sending 'up messages' again<br />
pool suppress hsmload on|off<br />
==== Disable / enable pool examples ====<br />
:* Disallow store + stage + p2p-client (set pool in read only mode)<br />
'''''[dccore01] (local) admin >''''' '''pool disable -rdonly'''<br />
:* Enable pool (set pool to RW)<br />
'''''[dccore01] (local) admin >''''' '''pool enable'''<br />
<br />
=== HSM configuration ===<br />
* Command help ('''\s dc???_? help hsm'''):<br />
hsm create <type> [<name> [<provider>]] [-<key>=<value>] ...<br />
hsm ls [<name>] ...<br />
hsm remove <name><br />
hsm set <name> [-<key>=<value>] ...<br />
hsm show providers # show available nearline storage providers<br />
hsm unset <name> [-<key>] ...<br />
==== HSM configuration examples ====<br />
:* Create a ''enstore'' HSM with the ''real-encp.sh'' client script (this example can be found in the '''setup''' file):<br />
'''''[dccore01] (local) admin >''''' '''hsm create enstore enstore script -pnfs=/pnfs/fs/usr -command=/opt/enstore/bin/real-encp.sh -c:gets=200 -c:puts=20 -c:removes=2'''<br />
:* Show ''enstore'' configuration:<br />
'''''[dccore01] (local) admin >''''' '''hsm ls enstore'''<br />
enstore(enstore):script<br />
pnfs /pnfs/fs/usr<br />
command /opt/enstore/bin/real-encp.sh<br />
c:gets 200<br />
c:puts 20<br />
c:removes 2<br />
:* Set maximum restores to 400:<br />
'''''[dccore01] (local) admin >''''' '''hsm set enstore -c:gets=400'''</div>Mcaubethttps://pwiki.pic.es/index.php?title=DCache_Administration_Console_Commands&diff=184DCache Administration Console Commands2017-07-31T09:14:35Z<p>Mcaubet: /* The basics */</p>
<hr />
<div>= About this document =<br />
* This document shows several examples of useful commands that can be run in the dCache Administration Console.<br />
* Administration console can be accessed through '''[[https://www.dcache.org/downloads/gui/ PCells GUI]]''' or through '''SSHv2''' as follows (in the example, '''dccore.pic.es''' is the physical machine running the '''admin''' cell):<br />
ssh -2 -x -o StrictHostKeyChecking=no -l admin -c blowfish -p 22224 dccore.pic.es<br />
* In this document we show what we consider the most important commands for the everyday's administration tasks, and basically the ones that may be useful for the Manager On Duty. Maybe some important commands are missing, but will be added as soon as we detect that these may be important for the MoD.<br />
* This document is divided in two main blocks:<br />
# '''Pool commands''': shows useful commands that can be run on pools.<br />
# '''Service commands''': show useful commands that can be run on servers (this will include doors)<br />
== The basics ==<br />
* If you access the Administration Console through '''SSHv2''' you should access to the different cells as follows:<br />
\? [command]... # display help for shell commands<br />
\c cell[@domain] [user] # connect to cell<br />
\exception [trace] # controls display of stack traces<br />
\h [command]... # display help for cell commands<br />
\l [cell[@domain]|pool/poolgroup]... # list cells<br />
\q # quit<br />
'''\s (cell[@domain]|pool/poolgroup)[,(cell[@domain]|pool/poolgroup)]... command... # send command'''<br />
\sl pnfsid|path command... # send to locations<br />
\sn command... # send pnfsmanager command<br />
\sp command... # send poolmanager command<br />
\timeout [seconds] # sets the command timeout<br />
* With '''\s''' you will be able to access to any '''cell[@domain]'''s, while branches of this command such like '''\sn''' or '''\sp''' are useful to (respectively) direct access to the ''namespace'' or ''poolmanager'' cells without the need of specifying the ''cell[@domain]''.<br />
<br />
== Pool commands ==<br />
=== Introduction ===<br />
* Any command run on a pool is a temporary change: on any pool service restart configuration will be restored back to the default value or to the value set in the '''setup''' file (locate in '''/dcpool*/vpool*/setup''').<br />
:* In example, whenever we set a pool to ''rdonly'', on restart, once the ''initialization'' process finishes, pool will switch back to ''enabled''.<br />
* If you want to make permanent changes, you should '''save''' changes. Running the '''save''' command will update the already mentioned '''setup''' file.<br />
:* In example, whenever we set a pool to ''rdonly'' '''and''' we ''save'' the configuration, on a restart, once the ''initialization'' process finishes, pool will remain ''rdonly''.<br />
<br />
=== Disable / enable pools ===<br />
* Command help ('''\s dc???_? help hsm'''):<br />
pool disable [options] [<errorCode> [<errorMessage>]] # suspend sending 'up messages'<br />
OPTIONS :<br />
-fetch # disallows fetch (transfer to client)<br />
-stage # disallows staging (from HSM)<br />
-store # disallows store (transfer from client)<br />
-p2p-client<br />
-rdonly # := store,stage,p2p-client<br />
-strict # := disallows everything<br />
pool enable # start sending 'up messages' again<br />
pool suppress hsmload on|off<br />
==== Disable / enable pool examples ====<br />
:* Disallow store + stage + p2p-client (set pool in read only mode)<br />
'''''[dccore01] (local) admin >''''' '''pool disable -rdonly'''<br />
:* Enable pool (set pool to RW)<br />
'''''[dccore01] (local) admin >''''' '''pool enable'''<br />
<br />
=== HSM configuration ===<br />
* Command help ('''\s dc???_? help hsm'''):<br />
hsm create <type> [<name> [<provider>]] [-<key>=<value>] ...<br />
hsm ls [<name>] ...<br />
hsm remove <name><br />
hsm set <name> [-<key>=<value>] ...<br />
hsm show providers # show available nearline storage providers<br />
hsm unset <name> [-<key>] ...<br />
==== HSM configuration examples ====<br />
:* Create a ''enstore'' HSM with the ''real-encp.sh'' client script (this example can be found in the '''setup''' file):<br />
'''''[dccore01] (local) admin >''''' '''hsm create enstore enstore script -pnfs=/pnfs/fs/usr -command=/opt/enstore/bin/real-encp.sh -c:gets=200 -c:puts=20 -c:removes=2'''<br />
:* Show ''enstore'' configuration:<br />
'''''[dccore01] (local) admin >''''' '''hsm ls enstore'''<br />
enstore(enstore):script<br />
pnfs /pnfs/fs/usr<br />
command /opt/enstore/bin/real-encp.sh<br />
c:gets 200<br />
c:puts 20<br />
c:removes 2<br />
:* Set maximum restores to 400:<br />
'''''[dccore01] (local) admin >''''' '''hsm set enstore -c:gets=400'''</div>Mcaubethttps://pwiki.pic.es/index.php?title=DCache_Administration_Console_Commands&diff=183DCache Administration Console Commands2017-07-31T09:13:19Z<p>Mcaubet: /* The basics = */</p>
<hr />
<div>= About this document =<br />
* This document shows several examples of useful commands that can be run in the dCache Administration Console.<br />
* Administration console can be accessed through '''[[https://www.dcache.org/downloads/gui/ PCells GUI]]''' or through '''SSHv2''' as follows (in the example, '''dccore.pic.es''' is the physical machine running the '''admin''' cell):<br />
ssh -2 -x -o StrictHostKeyChecking=no -l admin -c blowfish -p 22224 dccore.pic.es<br />
* In this document we show what we consider the most important commands for the everyday's administration tasks, and basically the ones that may be useful for the Manager On Duty. Maybe some important commands are missing, but will be added as soon as we detect that these may be important for the MoD.<br />
* This document is divided in two main blocks:<br />
# '''Pool commands''': shows useful commands that can be run on pools.<br />
# '''Service commands''': show useful commands that can be run on servers (this will include doors)<br />
== The basics ==<br />
* If you access the Administration Console through '''SSHv2''' you should access to the different cells as follows:<br />
\? [command]... # display help for shell commands<br />
\c cell[@domain] [user] # connect to cell<br />
\exception [trace] # controls display of stack traces<br />
\h [command]... # display help for cell commands<br />
\l [cell[@domain]|pool/poolgroup]... # list cells<br />
\q # quit<br />
'''\s (cell[@domain]|pool/poolgroup)[,(cell[@domain]|pool/poolgroup)]... command... # send command'''<br />
\sl pnfsid|path command... # send to locations<br />
\sn command... # send pnfsmanager command<br />
\sp command... # send poolmanager command<br />
\timeout [seconds] # sets the command timeout<br />
* With '''\s''' you will be able to access to all '''cell[@domain]'''s, while branches of this command such like '''\sn''' or '''\sp''' are useful to directly access to the ''namespace'' or ''poolmanager'' cells without the needing of specifying the ''cell[@domain]''.<br />
<br />
== Pool commands ==<br />
=== Introduction ===<br />
* Any command run on a pool is a temporary change: on any pool service restart configuration will be restored back to the default value or to the value set in the '''setup''' file (locate in '''/dcpool*/vpool*/setup''').<br />
:* In example, whenever we set a pool to ''rdonly'', on restart, once the ''initialization'' process finishes, pool will switch back to ''enabled''.<br />
* If you want to make permanent changes, you should '''save''' changes. Running the '''save''' command will update the already mentioned '''setup''' file.<br />
:* In example, whenever we set a pool to ''rdonly'' '''and''' we ''save'' the configuration, on a restart, once the ''initialization'' process finishes, pool will remain ''rdonly''.<br />
<br />
=== Disable / enable pools ===<br />
* Command help ('''\s dc???_? help hsm'''):<br />
pool disable [options] [<errorCode> [<errorMessage>]] # suspend sending 'up messages'<br />
OPTIONS :<br />
-fetch # disallows fetch (transfer to client)<br />
-stage # disallows staging (from HSM)<br />
-store # disallows store (transfer from client)<br />
-p2p-client<br />
-rdonly # := store,stage,p2p-client<br />
-strict # := disallows everything<br />
pool enable # start sending 'up messages' again<br />
pool suppress hsmload on|off<br />
==== Disable / enable pool examples ====<br />
:* Disallow store + stage + p2p-client (set pool in read only mode)<br />
'''''[dccore01] (local) admin >''''' '''pool disable -rdonly'''<br />
:* Enable pool (set pool to RW)<br />
'''''[dccore01] (local) admin >''''' '''pool enable'''<br />
<br />
=== HSM configuration ===<br />
* Command help ('''\s dc???_? help hsm'''):<br />
hsm create <type> [<name> [<provider>]] [-<key>=<value>] ...<br />
hsm ls [<name>] ...<br />
hsm remove <name><br />
hsm set <name> [-<key>=<value>] ...<br />
hsm show providers # show available nearline storage providers<br />
hsm unset <name> [-<key>] ...<br />
==== HSM configuration examples ====<br />
:* Create a ''enstore'' HSM with the ''real-encp.sh'' client script (this example can be found in the '''setup''' file):<br />
'''''[dccore01] (local) admin >''''' '''hsm create enstore enstore script -pnfs=/pnfs/fs/usr -command=/opt/enstore/bin/real-encp.sh -c:gets=200 -c:puts=20 -c:removes=2'''<br />
:* Show ''enstore'' configuration:<br />
'''''[dccore01] (local) admin >''''' '''hsm ls enstore'''<br />
enstore(enstore):script<br />
pnfs /pnfs/fs/usr<br />
command /opt/enstore/bin/real-encp.sh<br />
c:gets 200<br />
c:puts 20<br />
c:removes 2<br />
:* Set maximum restores to 400:<br />
'''''[dccore01] (local) admin >''''' '''hsm set enstore -c:gets=400'''</div>Mcaubethttps://pwiki.pic.es/index.php?title=DCache_Administration_Console_Commands&diff=182DCache Administration Console Commands2017-07-31T09:12:26Z<p>Mcaubet: /* HSM configuration examples */</p>
<hr />
<div>= About this document =<br />
* This document shows several examples of useful commands that can be run in the dCache Administration Console.<br />
* Administration console can be accessed through '''[[https://www.dcache.org/downloads/gui/ PCells GUI]]''' or through '''SSHv2''' as follows (in the example, '''dccore.pic.es''' is the physical machine running the '''admin''' cell):<br />
ssh -2 -x -o StrictHostKeyChecking=no -l admin -c blowfish -p 22224 dccore.pic.es<br />
* In this document we show what we consider the most important commands for the everyday's administration tasks, and basically the ones that may be useful for the Manager On Duty. Maybe some important commands are missing, but will be added as soon as we detect that these may be important for the MoD.<br />
* This document is divided in two main blocks:<br />
# '''Pool commands''': shows useful commands that can be run on pools.<br />
# '''Service commands''': show useful commands that can be run on servers (this will include doors)<br />
== The basics ===<br />
* If you access the Administration Console through '''SSHv2''' you should access to the different cells as follows:<br />
\? [command]... # display help for shell commands<br />
\c cell[@domain] [user] # connect to cell<br />
\exception [trace] # controls display of stack traces<br />
\h [command]... # display help for cell commands<br />
\l [cell[@domain]|pool/poolgroup]... # list cells<br />
\q # quit<br />
'''\s (cell[@domain]|pool/poolgroup)[,(cell[@domain]|pool/poolgroup)]... command... # send command'''<br />
\sl pnfsid|path command... # send to locations<br />
\sn command... # send pnfsmanager command<br />
\sp command... # send poolmanager command<br />
\timeout [seconds] # sets the command timeout<br />
* With '''\s''' you will be able to access to all '''cell[@domain]'''s, while branches of this command such like '''\sn''' or '''\sp''' are useful to directly access to the ''namespace'' or ''poolmanager'' cells without the needing of specifying the ''cell[@domain]''.<br />
== Pool commands ==<br />
=== Introduction ===<br />
* Any command run on a pool is a temporary change: on any pool service restart configuration will be restored back to the default value or to the value set in the '''setup''' file (locate in '''/dcpool*/vpool*/setup''').<br />
:* In example, whenever we set a pool to ''rdonly'', on restart, once the ''initialization'' process finishes, pool will switch back to ''enabled''.<br />
* If you want to make permanent changes, you should '''save''' changes. Running the '''save''' command will update the already mentioned '''setup''' file.<br />
:* In example, whenever we set a pool to ''rdonly'' '''and''' we ''save'' the configuration, on a restart, once the ''initialization'' process finishes, pool will remain ''rdonly''.<br />
<br />
=== Disable / enable pools ===<br />
* Command help ('''\s dc???_? help hsm'''):<br />
pool disable [options] [<errorCode> [<errorMessage>]] # suspend sending 'up messages'<br />
OPTIONS :<br />
-fetch # disallows fetch (transfer to client)<br />
-stage # disallows staging (from HSM)<br />
-store # disallows store (transfer from client)<br />
-p2p-client<br />
-rdonly # := store,stage,p2p-client<br />
-strict # := disallows everything<br />
pool enable # start sending 'up messages' again<br />
pool suppress hsmload on|off<br />
==== Disable / enable pool examples ====<br />
:* Disallow store + stage + p2p-client (set pool in read only mode)<br />
'''''[dccore01] (local) admin >''''' '''pool disable -rdonly'''<br />
:* Enable pool (set pool to RW)<br />
'''''[dccore01] (local) admin >''''' '''pool enable'''<br />
<br />
=== HSM configuration ===<br />
* Command help ('''\s dc???_? help hsm'''):<br />
hsm create <type> [<name> [<provider>]] [-<key>=<value>] ...<br />
hsm ls [<name>] ...<br />
hsm remove <name><br />
hsm set <name> [-<key>=<value>] ...<br />
hsm show providers # show available nearline storage providers<br />
hsm unset <name> [-<key>] ...<br />
==== HSM configuration examples ====<br />
:* Create a ''enstore'' HSM with the ''real-encp.sh'' client script (this example can be found in the '''setup''' file):<br />
'''''[dccore01] (local) admin >''''' '''hsm create enstore enstore script -pnfs=/pnfs/fs/usr -command=/opt/enstore/bin/real-encp.sh -c:gets=200 -c:puts=20 -c:removes=2'''<br />
:* Show ''enstore'' configuration:<br />
'''''[dccore01] (local) admin >''''' '''hsm ls enstore'''<br />
enstore(enstore):script<br />
pnfs /pnfs/fs/usr<br />
command /opt/enstore/bin/real-encp.sh<br />
c:gets 200<br />
c:puts 20<br />
c:removes 2<br />
:* Set maximum restores to 400:<br />
'''''[dccore01] (local) admin >''''' '''hsm set enstore -c:gets=400'''</div>Mcaubet