SHIFT

--- Sjoerd Hooft's InFormation Technology ---

User Tools

Site Tools


Sidebar

Recently Changed Pages:

View All Pages


View All Tags


LinkedIn




WIKI Disclaimer: As with most other things on the Internet, the content on this wiki is not supported. It was contributed by me and is published “as is”. It has worked for me, and might work for you.
Also note that any view or statement expressed anywhere on this site are strictly mine and not the opinions or views of my employer.


Pages with comments

View All Comments

fcppartnerpathmisconfigured

Fix: FCP Partner Path Misconfigured

This is a small manual on fixing the error

FCP Partner Path Misconfigured

which could show in the AutoSupport report created in NetApp/N-series filers. It means that a particular LUN is being accessed over the partner path through the interconnect link rather than through the preferred primary (optimized) path. For more information see the resources.
Finding and resolving the issue is a matter of a few steps:

  1. Identify the LUN which is being accessed over the partner path
  2. Identify the host initiator for the LUN
  3. Identify the correct target WWPN of the filer
  4. Solve the path selection error

Identify the LUNs

The only way to see which LUN is the source of the error is by checking the data traffic over the partner path. It's possible to see the LUNs statistics but you should zero them first so events in the past don't interfere with the troubleshooting process:
This is the command to zero the LUN statistics:

filer01> lun stats -z

After issuing the command you should wait for a couple of minutes.
Than check the statistics so far. Note that a few partner ops as well as a few kbytes are normal for multipathing operations but not too much:

filer01> lun stats -o
...<cut>...
    /vol/DATA_Volume_0/F1A13_DATA  (0 hours, 1 minute, 18 seconds)
        Read (kbytes)   Write (kbytes)  Read Ops  Write Ops  Other Ops  QFulls  Partner Ops Partner KBytes
        66672           105168          2480      23267      44         0       12957       42990
...<cut>...

The affected LUN is “/vol/DATA_Volume_0/F1A13_DATA”

Identify the host Initiators

Identify the Initiator Group

Now you have to identify the host which is accessing the LUN. To do so you first check the initiator group which has access, followed by checking the members for the identified initiator group:
First find the initiator group which accesses the LUNs:

filer01> lun show -v /vol/DATA_Volume_0/F1A13_DATA
        /vol/DATA_Volume_0/F1A13_DATA  800.1g (859047985152)  (r/w, online, mapped)
                Serial#: W-CqCJRe/kaJ
                Share: none
                Space Reservation: disabled
                Multiprotocol Type: vmware
                Maps: ESXServers_datacenter=21

The LUN is accessed by hosts in the initiator group “ESXServers_datacenter”

Identify Member Servers for Initiator Group

Now check which hosts are in the initiator group:

filer01> igroup show -v ESXServers_datacenter
    ESXServers_datacenter (FCP):
        OS Type: vmware
        Member: 21:01:00:1b:32:b0:33:00 (logged in on: 4b, 4a, vtic)
        Member: 21:00:00:1b:32:90:33:00 (logged in on: 4d, 4c, vtic)
        Member: 21:00:00:1b:32:90:35:00 (logged in on: 4d, 4c, vtic)
        Member: 21:01:00:1b:32:b0:35:00 (logged in on: 4b, 4a, vtic)
        Member: 21:00:00:1b:32:90:35:01 (logged in on: 4d, 4c, vtic)
        Member: 21:01:00:1b:32:b0:35:01 (logged in on: 4b, 4a, vtic)
        Member: 21:00:00:1b:32:90:35:01 (logged in on: 4d, 4c, vtic)
        Member: 21:01:00:1b:32:b0:35:01 (logged in on: 4b, 4a, vtic)
        Member: 21:01:00:1b:32:b0:35:0f (logged in on: 4b, 4a, vtic)
        Member: 21:00:00:1b:32:90:35:0f (logged in on: 4d, 4c, vtic)
Note: Only members that are also clogged in on “vtic” (interconnect link) can access the LUN over the partnerpath. The order in which they are listed is not relevant when the host utilities are installed.

Reduce host initiators list

Since the member list of the initiator group is quite extensive you can reduce the number of hosts by checking which ones are sending Read/Write IO over the partner path:

filer01> lun config_check -A

The following FCP Initiators are sending Read/Write i/o over the
FCP Partner Paths during the last 10 seconds
WWPN                      Partner's Port       ops         bytes      ostype
21:00:00:1b:32:90:35:0f               4c      1453       5047296      vmware

Proxy ops exceeded 4 out of 10 times
Proxy bytes exceed 6 out of 10 times

The following values have been changed from
their recommended values
lun.use_partner.cc.warn_limit    Recommended:300         Changed:10
lun.use_partner.cc.bytes         Recommended:2457600     Changed:512000
No Problems Found

So the server accessing LUN F1A13_DATA over it's partner path has a WWN “21:00:00:1b:32:90:35:0f”. I manually found the responsible host in vCenter but there are scripts available if needed. The corresponding host is: esx03

Identify the primary storage controller node

Now you need to know what are the primary paths. First we need the cfmode. The cfmode determines if LUNs can be accessed over their partner path:

filer01> fcp show cfmode
fcp show cfmode: single_image

In the active-active storage controller configuration, primary versus secondary ports will depend on the cluster failover mode (cfmode) employed. The single_image cfmode allows each FC target port to be used for primary path access to LUNs hosted on the local node and partner path access for LUNs hosted on the partner node.
Now find the WWPN for the local FC adapters:

filer01> fcp show adapters
Slot:                    4a
Description:             Fibre Channel Target Adapter 4a (Dual-channel, QLogic 2432 (2464) rev. 3)
Adapter Type:            Local
Status:                  ONLINE
FC Nodename:             50:0a:09:80:89:7a:cc:db (500a0980897accdb)
FC Portname:             50:0a:09:81:89:7a:cc:db (500a0981897accdb)
Standby:                 No

Slot:                    4b
Description:             Fibre Channel Target Adapter 4b (Dual-channel, QLogic 2432 (2464) rev. 3)
Adapter Type:            Local
Status:                  ONLINE
FC Nodename:             50:0a:09:80:89:7a:cc:db (500a0980897accdb)
FC Portname:             50:0a:09:82:89:7a:cc:db (500a0982897accdb)
Standby:                 No

Slot:                    4c
Description:             Fibre Channel Target Adapter 4c (Dual-channel, QLogic 2432 (2464) rev. 3)
Adapter Type:            Local
Status:                  ONLINE
FC Nodename:             50:0a:09:80:89:7a:cc:db (500a0980897accdb)
FC Portname:             50:0a:09:83:89:7a:cc:db (500a0983897accdb)
Standby:                 No

Slot:                    4d
Description:             Fibre Channel Target Adapter 4d (Dual-channel, QLogic 2432 (2464) rev. 3)
Adapter Type:            Local
Status:                  ONLINE
FC Nodename:             50:0a:09:80:89:7a:cc:db (500a0980897accdb)
FC Portname:             50:0a:09:84:89:7a:cc:db (500a0984897accdb)
Standby:                 No

As these are local FC ports, these ports are in potential all primary path to the LUNs.
Conclusion, if the host is not using these ports for its active paths the traffic is routed over the partner path.

ESX solving

As you can see in the screenshot the path was indeed configured to go over the partner path:

fcppartnerpathmisconfigured01.jpg

After running this command on the host the issue was resolved, first query, solve and than query again:

[root@esx03 ~]# /opt/ontap/santools/config_mpath --query --name naa.60a98000572d4371434a52652f6b614a
Verifying firewall port is open...................................DONE

Determining Primary/Proxy paths for LUN=naa.60a98000572d4371434a52652f6b614a
   PATH=vmhba3:C0:T2:L21 is a PROXY path
   PATH=vmhba3:C0:T3:L21 is a PROXY path
   PATH=vmhba4:C0:T0:L21 is a PRIMARY path
   PATH=vmhba4:C0:T1:L21 is a PRIMARY path
   PATH=vmhba3:C0:T0:L21 is a PRIMARY path
   PATH=vmhba4:C0:T2:L21 is a PROXY path * Preferred
   PATH=vmhba3:C0:T1:L21 is a PRIMARY path
   PATH=vmhba4:C0:T3:L21 is a PROXY path

[root@esx03 ~]# /opt/ontap/santools/config_mpath --primary --loadbalance --policy fixed --name naa.60a98000572d4371434a52652f6b614a
Verifying firewall port is open...................................DONE

Determining Primary/Proxy paths for LUN=naa.60a98000572d4371434a52652f6b614a
   PATH=vmhba3:C0:T2:L21 is a PROXY path active
   PATH=vmhba3:C0:T3:L21 is a PROXY path active
   PATH=vmhba4:C0:T0:L21 is a PRIMARY path active
   PATH=vmhba4:C0:T1:L21 is a PRIMARY path active
   PATH=vmhba3:C0:T0:L21 is a PRIMARY path active* Preferred
   PATH=vmhba4:C0:T2:L21 is a PROXY path active
   PATH=vmhba3:C0:T1:L21 is a PRIMARY path active
   PATH=vmhba4:C0:T3:L21 is a PROXY path active


PSP Policy for LUN naa.60a98000572d4371434a52652f6b614a is now set to VMW_PSP_FIXED
Preferred path has been set for vmhba4:C0:T0:L21 for LUN naa.60a98000572d4371434a52652f6b614a

The path policy has been set for your configuration.

[root@esx03 ~]# /opt/ontap/santools/config_mpath --query --name naa.60a98000572d4371434a52652f6b614a
Verifying firewall port is open...................................DONE

Determining Primary/Proxy paths for LUN=naa.60a98000572d4371434a52652f6b614a
   PATH=vmhba3:C0:T2:L21 is a PROXY path
   PATH=vmhba3:C0:T3:L21 is a PROXY path
   PATH=vmhba4:C0:T0:L21 is a PRIMARY path * Preferred
   PATH=vmhba4:C0:T1:L21 is a PRIMARY path
   PATH=vmhba3:C0:T0:L21 is a PRIMARY path
   PATH=vmhba4:C0:T2:L21 is a PROXY path
   PATH=vmhba3:C0:T1:L21 is a PRIMARY path
   PATH=vmhba4:C0:T3:L21 is a PROXY path
Note: Not using the –name parameter will set or query all LUNs

The result also shows in the vSphere client:

fcppartnerpathmisconfigured02.jpg

Resources

You could leave a comment if you were logged in.
fcppartnerpathmisconfigured.txt · Last modified: 2021/09/24 00:24 (external edit)