Friday, September 21, 2012

SAN Port Channel with UCS

Hi Guys!

I don't see many people doing this particular configuration, I am not sure why i tested it for failover etc and it works great!

This config is for uplinks between your UCS 62XX series Fabric Interconnect and your Nexus (or MDS with some slight modification) Fibre channel switch, for those of you with devices not doing full FCoE yet.

Let's say you have four FC 8 Gig uplinks between your Nexus-5548-A and your UCS-6248-A, and the same again for Nexus-5548_B and UCS-6248-B.

Now normally if you configure these as uplink ports and just plug them into your Nexus 5548UP and assign the appropriate VLAN's you will be fine: UCS is smart enough that it will "pin" service profiles to particular uplinks and try and get a bit of load balancing going on, and ofcourse it offers re-pinning if anything goes wrong.

But chances are you probably etherchannel your Ethernet uplinks from your UCS 6248, why not your fibre channel?

You can do it and I am going to show you how. :)

Nexus 5548UP Configuration
Below we have the relevant Nexus 5548UP Configuration:


I have assumed you already have basic fibre channel connectivity going so you already have things like feature npiv etc and you have your VSAN setup etc


First, enable the fibre-port-channel feature:

nexus#
nexus#conf t
nexus(config)#feature fport-channel-trunk

Next we need to configure the SAN port channel first BEFORE we add the ports to it (opposite way to how we do it with ethernet but whatever :))




interface san-port-channel 100
  channel mode active
  switchport trunk mode off



We set the channel mode to auto, SAN port channel only supports on or active, active negotiates an FC port channel but on forces it to be on. Next we set the  trunking mode to be off, this might be diffirent for you if your using both npiv and trunking multiple VSAN's but for most deployments this is fine.


Next, configure the actual ports to be members of the port channel:

interface fc1/29
  switchport trunk mode off
  channel-group 100 force


interface fc1/30
  switchport trunk mode off
  channel-group 100 force



Once you have done this if your VSAN was not vsan 1 you would need to bind this interface:


vsan database
 vsan XYZ interface san-port-channel-100

You can now no shut this:

interface fc1/29
no shut
interface fc1/30
no shut

interface san-port-channel-100
no shut

Once this is done you need to configure the UCS to support this connectivity.

First, make sure you have already defined your FC uplink ports and be CAREFUL THAT YOU HAVE PLACED THEM IN THE CORRECT VSAN







next, click on the SAN tab in UCS, expand out SAN Cloud, Fabric A, and FC Port channel.


Click the add button and give your san-port-channel a name and ID, i like to make the ID match the
 san-port-channel number on my Nexus switch.

Now once your port channel is created, you need to make sure that the VSAN is assigned correctly (it should match the VSAN's that you have assigned to the FC uplink ports) and then click enable portchannel. Walla! Done!




Check the status in UCS, and use "show flogi database" to make sure your Fabric Interconnect is logging into the the SAN, it should show as coming over your new SAN port channel. Do the same for Fabric B and your done!



Interestingly here is the actual UCS config that all of this you are doing is actually generating:




interface san-port-channel 100
  channel mode active
  switchport mode NP


Here you can see it has defined a port  channel and said that the port mode is NP


interface fc1/30
  channel-group 100 force
  no shutdown


Next you can see that UCS has placed the interface into this port channel, funny that this config is very similar to the guide on configuring SAN-port-channel between a fabric switch and an NPIV switch available here ;)
http://www.cisco.com/en/US/docs/switches/datacenter/mds9000/sw/4_1/configuration/guides/cli_4_1/pc.html

If you would like to know more about NX-OS specifically, I recommend this book (i have it myself through the Safari Online subscription, which I also recommend :)

If you want to know more about UCS, I quite enjoy the following book:
Finally if your MDS skills are not up to scratch, check out:


7 comments:

  1. Hello Peter, thanks for this post. My question is, with UCS operating in end-host mode, is this still feasible?

    ReplyDelete
    Replies
    1. This example is actually for FC EHM. If the UCS was changed to FC switching mode, then the uplink FC ports at the N5K would have to be configured as E ports and you would not need to enable the fport-channel feature.

      Delete
  2. Thanks Peter. I got it working with MDS9222i. This step really counts:
    "we need to configure the SAN port channel first BEFORE we add the ports to it (opposite way to how we do it with ethernet)"

    ReplyDelete
  3. Are the links for the books you recommend at the end deleted?

    ReplyDelete
  4. Trung tâm đào tạo kế toán thực tế Tại cầu giấy
    Trung tâm đào tạo kế toán thực tế Tại từ liêm
    Trung tâm đào tạo kế toán thực tế Tại thanh xuân
    Trung tâm đào tạo kế toán thực tế Tại hà đông
    Trung tâm đào tạo kế toán thực tế Tại long biên
    Trung tâm đào tạo kế toán thực tế Tại nguyễn chính thanh đống đa
    Trung tâm đào tạo kế toán thực tế Tại minh khai hai bà trưng
    Trung tâm đào tạo kế toán thực tế Tại bắc ninh
    Trung tâm đào tạo kế toán thực tế Tại hải phòng
    Trung tâm đào tạo kế toán thực tế Tại tphcm
    Trung tâm đào tạo kế toán thực tế Tại quận 3
    Trung tâm đào tạo kế toán thực tế Tại thủ đức
    Trung tâm đào tạo kế toán thực tế Tại đà nẵng
    Trung tâm đào tạo kế toán thực tế Tại biên hòa
    Trung tâm đào tạo kế toán thực tế Tại đồng nai
    Trung tâm đào tạo kế toán thực tế Tại nam định
    Trung tâm đào tạo kế toán thực tế Tại thái bình
    Trung tâm đào tạo kế toán thực tế Tại bắc giang
    Trung tâm đào tạo kế toán thực tế Tại vĩnh phúc
    Trung tâm đào tạo kế toán thực tế Tại thái nguyên
    Trung tâm đào tạo kế toán thực tế Tại quảng ninh
    Trung tâm đào tạo kế toán thực tế Tại hải dương
    Trung tâm đào tạo kế toán thực tế Tại hưng yên
    Trung tâm đào tạo kế toán thực tế Tại hà nam
    Trung tâm đào tạo kế toán thực tế Tại ninh bình
    Trung tâm đào tạo kế toán thực tế Tại nghệ an
    Trung tâm đào tạo kế toán thực tế Tại vũng tàu

    ReplyDelete
  5. Cisco UCS tutorial

    http://www.anyconsulting.com.br/blog/


    ReplyDelete