LUN cache dirty

[jetsenwang]

Xserver mdc1 ： Xsan 2.2.1 controller
Xserver mdc2 ： Xsan 2.2.1 failover

client：windows server 2003 ，stornext fx。

problem：recently，Client have the problem log ，such as：

(Critical) scan_disk_states: I/O path [ hostid ： 4 lun ：1 edev ：0x0 path (PhysicalDriver20)] restarted afer 500 seconds.


I check the raid ，that‘s well，no disk is down. So I think this problem about LUN cache dirty .

how to do?

eg： the Volumes cvlog：

[0523 18:41:17] 0x113c16000 (Info) Node [51] [172.168.1.20:2883] Client Logout (active 20).
[0523 18:41:25.039556] 0x7fff70fd4c20 (Debug) Active service - NSS ping from 172.168.1.20:2904.
[0523 18:41:25.043581] 0x7fff70fd4c20 (Debug) Node [53] [172.168.1.20:2905] connected.
[0523 18:41:25.046123] 0x10a8dd000 (Debug) FSM received client capabilities: capsClient = 0x5680effef
[0523 18:41:25.046131] 0x10a8dd000 (Debug) CvRootDir is /: CvRootCookie is 0x2
[0523 18:41:25] 0x10a8dd000 (Warning) [Node 53] Disk Stripe Group 1 is DOWN for this client. # disks 4 unitmap[1] 0xfffff partaccess 0x1
[0523 18:41:25] 0x10a8dd000 (Info) Node [53] [172.168.1.20:2905] Client Login (active 21).
[0523 18:41:25.050404] 0x113c16000 (Debug) Cache SUMMARY [172.168.1.20] attrs now/0 min/0 max/0.
[0523 18:41:25.050417] 0x113c16000 (Debug) RSVD SUMMARY [172.168.1.20] reserved space max requested/32 MB accounted now/0 MB
[0523 18:41:25.050428] 0x113c16000 (Debug) VOP SUMMARY [172.168.1.20] VopCapNegotiate cnt/1 avg/14+9 min/14+9 max/14+9.
[0523 18:41:25.050434] 0x113c16000 (Debug) VOP SUMMARY [172.168.1.20] VopClientId cnt/1 avg/11+11 min/11+11 max/11+11.
[0523 18:41:25.050441] 0x113c16000 (Debug) TKN SUMMARY [172.168.1.20] TokenRequestV2 cnt/2 avg/10+11 min/10+11 max/11+12.
[0523 18:41:25.050449] 0x113c16000 (Debug) NETWORK SUMMARY [172.168.1.20:2905]: QueuedInputMsgs max/1 QueuedOutputBytes max/0.
[0523 18:41:25] 0x113c16000 (Info) Node [53] [172.168.1.20:2905] Client Logout (active 20).
[0523 18:41:25.628114] 0x7fff70fd4c20 (Debug) Node [56] [mdc1.thm.com:51237] connected.
[0523 18:41:25.628655] 0x7fff70fd4c20 (Debug) Active service - NSS ping from mdc1.thm.com:51236.
[0523 18:41:25] 0x113c16000 (Info) Node [56] [mdc1.thm.com:51237] Administrator Logout.
[0523 18:41:26.854197] 0x7fff70fd4c20 (Debug) Node [57] [mdc2.thm.com:52969] connected.
[0523 18:41:26.854730] 0x7fff70fd4c20 (Debug) Active service - NSS ping from mdc2.thm.com:52968.
[0523 18:41:26] 0x113c16000 (Info) Node [57] [mdc2.thm.com:52969] Administrator Logout.
[0523 18:41:33.179989] 0x7fff70fd4c20 (Debug) Active service - NSS ping from 172.168.1.20:2926.
[0523 18:41:33.184014] 0x7fff70fd4c20 (Debug) Node [58] [172.168.1.20:2927] connected.
[0523 18:41:33.186168] 0x10cdb5000 (Debug) FSM received client capabilities: capsClient = 0x5680effef
[0523 18:41:33.186176] 0x10cdb5000 (Debug) CvRootDir is /: CvRootCookie is 0x2
[0523 18:41:33] 0x10cdb5000 (Warning) [Node 58] Disk Stripe Group 1 is DOWN for this client. # disks 4 unitmap[1] 0xfffff partaccess 0x1
[0523 18:41:33] 0x10cdb5000 (Info) Node [58] [172.168.1.20:2927] Client Login (active 21).
[0523 18:41:33.190528] 0x113c16000 (Debug) Cache SUMMARY [172.168.1.20] attrs now/0 min/0 max/0.
[0523 18:41:33.190541] 0x113c16000 (Debug) RSVD SUMMARY [172.168.1.20] reserved space max requested/32 MB accounted now/0 MB
[0523 18:41:33.190552] 0x113c16000 (Debug) VOP SUMMARY [172.168.1.20] VopCapNegotiate cnt/1 avg/18+11 min/18+11 max/18+11.
[0523 18:41:33.190558] 0x113c16000 (Debug) VOP SUMMARY [172.168.1.20] VopClientId cnt/1 avg/15+13 min/15+13 max/15+13.
[0523 18:41:33.190565] 0x113c16000 (Debug) TKN SUMMARY [172.168.1.20] TokenRequestV2 cnt/2 avg/13+15 min/13+14 max/13+16.
[0523 18:41:33.190572] 0x113c16000 (Debug) NETWORK SUMMARY [172.168.1.20:2927]: QueuedInputMsgs max/1 QueuedOutputBytes max/0.
[0523 18:41:33] 0x113c16000 (Info) Node [58] [172.168.1.20:2927] Client Logout (active 20).
[0523 18:41:41.211027] 0x7fff70fd4c20 (Debug) Active service - NSS ping from 172.168.1.20:2948.
[0523 18:41:41.214958] 0x7fff70fd4c20 (Debug) Node [60] [172.168.1.20:2949] connected.
[0523 18:41:41.217128] 0x10b39c000 (Debug) FSM received client capabilities: capsClient = 0x5680effef
[0523 18:41:41.217136] 0x10b39c000 (Debug) CvRootDir is /: CvRootCookie is 0x2
[0523 18:41:41] 0x10b39c000 (Warning) [Node 60] Disk Stripe Group 1 is DOWN for this client. # disks 4 unitmap[1] 0xfffff partaccess 0x1
[0523 18:41:41] 0x10b39c000 (Info) Node [60] [172.168.1.20:2949] Client Login (active 21).
[0523 18:41:41.221427] 0x113e22000 (Debug) Cache SUMMARY [172.168.1.20] attrs now/0 min/0 max/0.
[0523 18:41:41.221440] 0x113e22000 (Debug) RSVD SUMMARY [172.168.1.20] reserved space max requested/32 MB accounted now/0 MB
[0523 18:41:41.221450] 0x113e22000 (Debug) VOP SUMMARY [172.168.1.20] VopCapNegotiate cnt/1 avg/18+11 min/18+11 max/18+11.
[0523 18:41:41.221456] 0x113e22000 (Debug) VOP SUMMARY [172.168.1.20] VopClientId cnt/1 avg/16+12 min/16+12 max/16+12.
[0523 18:41:41.221463] 0x113e22000 (Debug) TKN SUMMARY [172.168.1.20] TokenRequestV2 cnt/2 avg/14+14 min/14+13 max/14+15.
[0523 18:41:41.221471] 0x113e22000 (Debug) NETWORK SUMMARY [172.168.1.20:2949]: QueuedInputMsgs max/1 QueuedOutputBytes max/0.
[0523 18:41:41] 0x113e22000 (Info) Node [60] [172.168.1.20:2949] Client Logout (active 20).
[0523 18:41:49.241603] 0x7fff70fd4c20 (Debug) Active service - NSS ping from 172.168.1.20:2970.
[0523 18:41:49.245976] 0x7fff70fd4c20 (Debug) Node [61] [172.168.1.20:2971] connected.
[0523 18:41:49.248145] 0x10b296000 (Debug) FSM received client capabilities: capsClient = 0x5680effef
[0523 18:41:49.248153] 0x10b296000 (Debug) CvRootDir is /: CvRootCookie is 0x2
[0523 18:41:49] 0x10b296000 (Warning) [Node 61] Disk Stripe Group 1 is DOWN for this client. # disks 4 unitmap[1] 0xfffff partaccess 0x1
[0523 18:41:49] 0x10b296000 (Info) Node [61] [172.168.1.20:2971] Client Login (active 21).
[0523 18:41:49.252368] 0x113c16000 (Debug) Cache SUMMARY [172.168.1.20] attrs now/0 min/0 max/0.
[0523 18:41:49.252380] 0x113c16000 (Debug) RSVD SUMMARY [172.168.1.20] reserved space max requested/32 MB accounted now/0 MB
[0523 18:41:49.252390] 0x113c16000 (Debug) VOP SUMMARY [172.168.1.20] VopCapNegotiate cnt/1 avg/18+11 min/18+11 max/18+11.
[0523 18:41:49.252396] 0x113c16000 (Debug) VOP SUMMARY [172.168.1.20] VopClientId cnt/1 avg/16+12 min/16+12 max/16+12.
[0523 18:41:49.252403] 0x113c16000 (Debug) TKN SUMMARY [172.168.1.20] TokenRequestV2 cnt/2 avg/14+14 min/14+14 max/14+15.
[0523 18:41:49.252411] 0x113c16000 (Debug) NETWORK SUMMARY [172.168.1.20:2971]: QueuedInputMsgs max/1 QueuedOutputBytes max/0.
[0523 18:41:49] 0x113c16000 (Info) Node [61] [172.168.1.20:2971] Client Logout (active 20).
[0523 18:41:57.272456] 0x7fff70fd4c20 (Debug) Active service - NSS ping from 172.168.1.20:2992.
[0523 18:41:57.276790] 0x7fff70fd4c20 (Debug) Node [62] [172.168.1.20:2993] connected.
[0523 18:41:57.279406] 0x10b525000 (Debug) FSM received client capabilities: capsClient = 0x5680effef
[0523 18:41:57.279413] 0x10b525000 (Debug) CvRootDir is /: CvRootCookie is 0x2
[0523 18:41:57] 0x10b525000 (Warning) [Node 62] Disk Stripe Group 1 is DOWN for this client. # disks 4 unitmap[1] 0xfffff partaccess 0x1
[0523 18:41:57] 0x10b525000 (Info) Node [62] [172.168.1.20:2993] Client Login (active 21).
[0523 18:41:57.283874] 0x113c16000 (Debug) Cache SUMMARY [172.168.1.20] attrs now/0 min/0 max/0.
[0523 18:41:57.283886] 0x113c16000 (Debug) RSVD SUMMARY [172.168.1.20] reserved space max requested/32 MB accounted now/0 MB
[0523 18:41:57.283897] 0x113c16000 (Debug) VOP SUMMARY [172.168.1.20] VopCapNegotiate cnt/1 avg/18+12 min/18+12 max/18+12.
[0523 18:41:57.283902] 0x113c16000 (Debug) VOP SUMMARY [172.168.1.20] VopClientId cnt/1 avg/18+13 min/18+13 max/18+13.
[0523 18:41:57.283910] 0x113c16000 (Debug) TKN SUMMARY [172.168.1.20] TokenRequestV2 cnt/2 avg/15+15 min/15+15 max/15+16.
[0523 18:41:57.283917] 0x113c16000 (Debug) NETWORK SUMMARY [172.168.1.20:2993]: QueuedInputMsgs max/1 QueuedOutputBytes max/0.
[0523 18:41:57] 0x113c16000 (Info) Node [62] [172.168.1.20:2993] Client Logout (active 20).
[0523 18:42:05.319198] 0x7fff70fd4c20 (Debug) Active service - NSS ping from 172.168.1.20:3016.
[0523 18:42:05.323310] 0x7fff70fd4c20 (Debug) Node [64] [172.168.1.20:3017] connected.
[0523 18:42:05.325490] 0x10cba9000 (Debug) FSM received client capabilities: capsClient = 0x5680effef
[0523 18:42:05.325497] 0x10cba9000 (Debug) CvRootDir is /: CvRootCookie is 0x2
[0523 18:42:05] 0x10cba9000 (Warning) [Node 64] Disk Stripe Group 1 is DOWN for this client. # disks 4 unitmap[1] 0xfffff partaccess 0x1
[0523 18:42:05] 0x10cba9000 (Info) Node [64] [172.168.1.20:3017] Client Login (active 21).
[0523 18:42:05.329826] 0x113c16000 (Debug) Cache SUMMARY [172.168.1.20] attrs now/0 min/0 max/0.
[0523 18:42:05.329838] 0x113c16000 (Debug) RSVD SUMMARY [172.168.1.20] reserved space max requested/32 MB accounted now/0 MB
[0523 18:42:05.329849] 0x113c16000 (Debug) VOP SUMMARY [172.168.1.20] VopCapNegotiate cnt/1 avg/17+13 min/17+13 max/17+13.
[0523 18:42:05.329855] 0x113c16000 (Debug) VOP SUMMARY [172.168.1.20] VopClientId cnt/1 avg/15+13 min/15+13 max/15+13.
[0523 18:42:05.329862] 0x113c16000 (Debug) TKN SUMMARY [172.168.1.20] TokenRequestV2 cnt/2 avg/14+16 min/13+15 max/15+17.
[0523 18:42:05.329869] 0x113c16000 (Debug) NETWORK SUMMARY [172.168.1.20:3017]: QueuedInputMsgs max/1 QueuedOutputBytes max/0.
[0523 18:42:05] 0x113c16000 (Info) Node [64] [172.168.1.20:3017] Client Logout (active 20).
[0523 18:42:13.350337] 0x7fff70fd4c20 (Debug) Active service - NSS ping from 172.168.1.20:3038.
[0523 18:42:13.354412] 0x7fff70fd4c20 (Debug) Node [65] [172.168.1.20:3039] connected.
[0523 18:42:13.356947] 0x10d97a000 (Debug) FSM received client capabilities: capsClient = 0x5680effef
[0523 18:42:13.356955] 0x10d97a000 (Debug) CvRootDir is /: CvRootCookie is 0x2
[0523 18:42:13] 0x10d97a000 (Warning) [Node 65] Disk Stripe Group 1 is DOWN for this client. # disks 4 unitmap[1] 0xfffff partaccess 0x1
[0523 18:42:13] 0x10d97a000 (Info) Node [65] [172.168.1.20:3039] Client Login (active 21).
[0523 18:42:13.361362] 0x113c16000 (Debug) Cache SUMMARY [172.168.1.20] attrs now/0 min/0 max/0.
[0523 18:42:13.361374] 0x113c16000 (Debug) RSVD SUMMARY [172.168.1.20] reserved space max requested/32 MB accounted now/0 MB
[0523 18:42:13.361384] 0x113c16000 (Debug) VOP SUMMARY [172.168.1.20] VopCapNegotiate cnt/1 avg/17+13 min/17+13 max/17+13.
[0523 18:42:13.361390] 0x113c16000 (Debug) VOP SUMMARY [172.168.1.20] VopClientId cnt/1 avg/15+13 min/15+13 max/15+13.
[0523 18:42:13.361397] 0x113c16000 (Debug) TKN SUMMARY [172.168.1.20] TokenRequestV2 cnt/2 avg/13+15 min/13+15 max/13+15.
[0523 18:42:13.361405] 0x113c16000 (Debug) NETWORK SUMMARY [172.168.1.20:3039]: QueuedInputMsgs max/1 QueuedOutputBytes max/0.
[0523 18:42:13] 0x113c16000 (Info) Node [65] [172.168.1.20:3039] Client Logout (active 20).
[0523 18:42:21.410580] 0x7fff70fd4c20 (Debug) Active service - NSS ping from 172.168.1.20:3060.
[0523 18:42:21.415835] 0x7fff70fd4c20 (Debug) Node [67] [172.168.1.20:3061] connected.
[0523 18:42:21.418053] 0x10e4bc000 (Debug) FSM received client capabilities: capsClient = 0x5680effef
[0523 18:42:21.418061] 0x10e4bc000 (Debug) CvRootDir is /: CvRootCookie is 0x2
[0523 18:42:21] 0x10e4bc000 (Warning) [Node 67] Disk Stripe Group 1 is DOWN for this client. # disks 4 unitmap[1] 0xfffff partaccess 0x1
[0523 18:42:21] 0x10e4bc000 (Info) Node [67] [172.168.1.20:3061] Client Login (active 21).
[0523 18:42:21.422194] 0x113c16000 (Debug) Cache SUMMARY [172.168.1.20] attrs now/0 min/0 max/0.
[0523 18:42:21.422206] 0x113c16000 (Debug) RSVD SUMMARY [172.168.1.20] reserved space max requested/32 MB accounted now/0 MB
[0523 18:42:21.422217] 0x113c16000 (Debug) VOP SUMMARY [172.168.1.20] VopCapNegotiate cnt/1 avg/13+9 min/13+9 max/13+9.
[0523 18:42:21.422223] 0x113c16000 (Debug) VOP SUMMARY [172.168.1.20] VopClientId cnt/1 avg/13+10 min/13+10 max/13+10.
[0523 18:42:21.422231] 0x113c16000 (Debug) TKN SUMMARY [172.168.1.20] TokenRequestV2 cnt/2 avg/11+12 min/11+11 max/12+13.
[0523 18:42:21.422238] 0x113c16000 (Debug) NETWORK SUMMARY [172.168.1.20:3061]: QueuedInputMsgs max/1 QueuedOutputBytes max/0.
[0523 18:42:21] 0x113c16000 (Info) Node [67] [172.168.1.20:3061] Client Logout (active 20).
[0523 18:42:25.215666] 0x7fff70fd4c20 (Debug) Node [69] [mdc1.thm.com:51296] connected.
[0523 18:42:25.216179] 0x7fff70fd4c20 (Debug) Active service - NSS ping from mdc1.thm.com:51295.
[0523 18:42:25] 0x113c16000 (Info) Node [69] [mdc1.thm.com:51296] Administrator Logout.
[0523 18:42:28.024018] 0x7fff70fd4c20 (Debug) Node [70] [mdc2.thm.com:52998] connected.
[0523 18:42:28.024578] 0x7fff70fd4c20 (Debug) Active service - NSS ping from mdc2.thm.com:52997.
[0523 18:42:28] 0x113c16000 (Info) Node [70] [mdc2.thm.com:52998] Administrator Logout.
[0523 18:42:29.456980] 0x7fff70fd4c20 (Debug) Active service - NSS ping from 172.168.1.20:3082.
[0523 18:42:29.462550] 0x7fff70fd4c20 (Debug) Node [71] [172.168.1.20:3083] connected.
[0523 18:42:29.464877] 0x10db86000 (Debug) FSM received client capabilities: capsClient = 0x5680effef
[0523 18:42:29.464884] 0x10db86000 (Debug) CvRootDir is /: CvRootCookie is 0x2
[0523 18:42:29] 0x10db86000 (Warning) [Node 71] Disk Stripe Group 1 is DOWN for this client. # disks 4 unitmap[1] 0xfffff partaccess 0x1
[0523 18:42:29] 0x10db86000 (Info) Node [71] [172.168.1.20:3083] Client Login (active 21).
[0523 18:42:29.468837] 0x113c16000 (Debug) Cache SUMMARY [172.168.1.20] attrs now/0 min/0 max/0.
[0523 18:42:29.468849] 0x113c16000 (Debug) RSVD SUMMARY [172.168.1.20] reserved space max requested/32 MB accounted now/0 MB
[0523 18:42:29.468860] 0x113c16000 (Debug) VOP SUMMARY [172.168.1.20] VopCapNegotiate cnt/1 avg/17+11 min/17+11 max/17+11.
[0523 18:42:29.468866] 0x113c16000 (Debug) VOP SUMMARY [172.168.1.20] VopClientId cnt/1 avg/15+12 min/15+12 max/15+12.
[0523 18:42:29.468873] 0x113c16000 (Debug) TKN SUMMARY [172.168.1.20] TokenRequestV2 cnt/2 avg/13+13 min/13+13 max/14+14.
[0523 18:42:29.468881] 0x113c16000 (Debug) NETWORK SUMMARY [172.168.1.20:3083]: QueuedInputMsgs max/1 QueuedOutputBytes max/0.
[0523 18:42:29] 0x113c16000 (Info) Node [71] [172.168.1.20:3083] Client Logout (active 20).
[0523 18:42:37.503450] 0x7fff70fd4c20 (Debug) Active service - NSS ping from 172.168.1.20:3104.
[0523 18:42:37.508525] 0x7fff70fd4c20 (Debug) Node [72] [172.168.1.20:3105] connected.
[0523 18:42:37.510815] 0x10bcd2000 (Debug) FSM received client capabilities: capsClient = 0x5680effef
[0523 18:42:37.510823] 0x10bcd2000 (Debug) CvRootDir is /: CvRootCookie is 0x2
[0523 18:42:37] 0x10bcd2000 (Warning) [Node 72] Disk Stripe Group 1 is DOWN for this client. # disks 4 unitmap[1] 0xfffff partaccess 0x1
[0523 18:42:37] 0x10bcd2000 (Info) Node [72] [172.168.1.20:3105] Client Login (active 21).
[0523 18:42:37.515025] 0x113c16000 (Debug) Cache SUMMARY [172.168.1.20] attrs now/0 min/0 max/0.
[0523 18:42:37.515038] 0x113c16000 (Debug) RSVD SUMMARY [172.168.1.20] reserved space max requested/32 MB accounted now/0 MB
[0523 18:42:37.515049] 0x113c16000 (Debug) VOP SUMMARY [172.168.1.20] VopCapNegotiate cnt/1 avg/14+9 min/14+9 max/14+9.
[0523 18:42:37.515055] 0x113c16000 (Debug) VOP SUMMARY [172.168.1.20] VopClientId cnt/1 avg/12+11 min/12+11 max/12+11.
[0523 18:42:37.515062] 0x113c16000 (Debug) TKN SUMMARY [172.168.1.20] TokenRequestV2 cnt/2 avg/10+11 min/10+10 max/10+12.
[0523 18:42:37.515070] 0x113c16000 (Debug) NETWORK SUMMARY [172.168.1.20:3105]: QueuedInputMsgs max/1 QueuedOutputBytes max/0.
[0523 18:42:37] 0x113c16000 (Info) Node [72] [172.168.1.20:3105] Client Logout (active 20).