Sunday, April 21, 2013

CCIE DC: FCoE NPV

Hi Guys!

Final blog post for tonight I have spent lots of time this weekend working super hard on my study!

So let's check out FCoE NPV, first a quick word of advice: Make sure when switching fibre channel modes (NPV vs FC Switching) make sure you completely write erase before you switch modes.

Without further adue, let's look at the config!

First thing's first, let's turn on the FCoE NPV Feature on the switch that is going to be on the actual switch performing NPV:

feature fcoe-npv

On the FC switch that will act as our core and WILL participate in the FC as a FC Forwarder, we need to turn on NPIV

feature npiv


As usual with FCoE, let's create our VSAN's and our VLAN's

vsan database
  vsan 10

!
vlan 10
  fcoe vsan 10

!

Next, we configure our ethernet interface that faces up towards the core, we can do this on both our core switch and our NPV Switch



interface Ethernet1/10
  switchport mode trunk
  switchport trunk allowed vlan 10,20
!


Next, let's configure our VFC Interface:


interface vfc1
  bind interface Ethernet1/10
  switchport mode NP
  switchport trunk mode on
  no shutdown



This is now ready on our NPV side, on our Core switch the config is only slightly diffirent:


interface vfc1
  bind interface Ethernet1/10
  switchport trunk allowed vsan 10
  no shutdown

As you can see on the upstream switch our core switch just has this configured as an F Port.


Now finally, we need to make each of them realise that the port is a member of the VSAN:

vsan database
  vsan 10 interface vfc1


Now we can check to see if the vfc interface is up on our NPV Switch:

switch# show int vfc1
vfc1 is trunking (Not all VSANs UP on the trunk)
    Bound interface is Ethernet1/10
    Hardware is Ethernet
    Port WWN is 20:00:54:7f:ee:af:1c:bf
    Admin port mode is NP, trunk mode is on
    snmp link state traps are enabled
    Port mode is TNP
    Port vsan is 10
    Trunk vsans (admin allowed and active) (1,10)
    Trunk vsans (up)                       (10)



Success! the next step is to get our server facing interface going, for the sake of brevity i will show all the configuration for this at once:

switch# show run int vfc2

interface vfc2
  bind interface Ethernet1/1
  switchport trunk mode on
  no shutdown

!

interface Ethernet1/1
  switchport mode trunk
  switchport trunk allowed vlan 10,20
  spanning-tree port type edge trunk

!

vsan database
  vsan 10 interface vfc2

Let's check the server facing interface:

switch# show int vfc2
vfc2 is trunking (Not all VSANs UP on the trunk)
    Bound interface is Ethernet1/1
    Hardware is Ethernet
    Port WWN is 20:01:54:7f:ee:af:1c:bf
    Admin port mode is F, trunk mode is on
    snmp link state traps are enabled
    Port mode is TF
    Port vsan is 10
    Trunk vsans (admin allowed and active) (1,10)
    Trunk vsans (up)                       (10)

!

Success! Let's check the NPV flogi table which is very useful.


switch# show npv flogi-table
--------------------------------------------------------------------------------
SERVER                                                                  EXTERNAL INTERFACE VSAN FCID             PORT NAME               NODE NAME       INTERFACE
--------------------------------------------------------------------------------
vfc2      10   0xd80002 20:00:a4:4c:11:13:8c:d1 10:00:a4:4c:11:13:8c:d1 vfc1


Very Useful output showing us that the port vfc2 is logged in and is using vfc1 for its external interface.


The show npv command is actually quite powerful:


switch# show npv ?
  external-interface-usage  Show external interface usage by server interfaces
  flogi-table               Show information about FLOGI sessions
  internal                  Show internal NPV information
  status                    Show NPV status
  traffic-map               Show information about Traffic Map



In particular, i found the following command VERY helpful:

in parti


switch# show npv internal events

1) Event:E_DEBUG, length:94, at 270505 usecs after Sun Apr 21 10:39:22 2013
    [538976288] E(10,vfc1) Received GMAL Response from core switch with Core Switch Inet Addr: 10.1.1.15



2) Event:E_DEBUG, length:252, at 270497 usecs after Sun Apr 21 10:39:22 2013
    [538976288] E(10,vfc1) npivp_get_ext_intf_fsm(1466): Is Core VF Capable: TRU
E
, Is Phy Login Done: TRUE, Is Port Channel: FALSE RID: { Type: Ext-If-VSAN(8),
IfIndex: vfc1(0x1e000000), UCD: 0, VSAN: 10, PWWN: 00:00:00:00:00:00:00:00 }, St
ate NPIVP_EXT_IF_ST_UP
This, along with show npv internal errors is really helpful in troubleshooting upstream issues.


If for example the upstream switch was not enabled for NPIV, I would see the following output in this command:




11) Event:E_DEBUG, length:98, at 464269 usecs after Sun Apr 21 10:41:22 2013
    [112] E(10,vfc1) FC Upstream switch is not NPIV enabled. Bringing down the external interface: vfc1



Saturday, April 20, 2013

CCIE DC: Multihop FCoE on Nexus 5k

Hi Guys.

There is a lot of confusion in my own mind for FCoE Multihop, I remember back in the day that the argument was "well, you can do FCoE with the Nexus 2k's which is not true multihop but it kind of is!" I also remember that this blog post: http://brasstacksblog.typepad.com/brass-tacks/2011/06/fcfcoe-connectivity-options-as-of-6272011.html by Erik was one of the best posts on the topic. I have asked Erik Via Twitter to update this diagram as a lot of it has changed now in regards to Cisco UCS.

In this blog post I will show one of the examples given by Erik, which is multihop FCOE Configuration on the Nexus 5k's in the hope that it will help someone out there.

Authors note: I think that the order of operation is very important when configuring multihop FCoE as I had major issues getting this going the first time I attempted it. My advice is to get the absolute basic FCoE Connectivity going first, then add complexity once your comfortable that it's all configured correctly.

Steps:
After enabling FCoE Ofcourse with feature FCOE on your nexus 5k,The first thing to do on both your Nexus 5k's is configure the appropriate VLAN's and VSAN's.

vsan database
 vsan 10
!
 
vlan 10
  fcoe vsan 10

vlan 20
  name DataVLAN


In this example I have configured VSAN 10, and bound it to VLAN 10.

Next, I need to configure the ethernet interface between my two switches appropriately:

interface Ethernet1/10
  switchport mode trunk
  switchport trunk allowed vlan 10

!

At this point I personally still don't "no shut" the interfaces and I wait until I have configured everything.

The next step is to configure the VFC Interface:


interface vfc1
  bind interface Ethernet1/10
  switchport mode E
  switchport trunk allowed vsan 10
  no shutdown

!

Now this is done, I no shut the ethernet interface on each end:

int eth1/10
no shut
!
 
Now let's look at our VFC1 interface:


switch# show int vfc1
vfc1 is trunking
    Bound interface is Ethernet1/10
    Hardware is Ethernet
    Port WWN is 20:00:54:7f:ee:af:1c:bf
    Admin port mode is E, trunk mode is on
    snmp link state traps are enabled
    Port mode is TE
    Port vsan is 1
    Trunk vsans (admin allowed and active) (10)
    Trunk vsans (up)                       (10)
    Trunk vsans (isolated)                 ()
    Trunk vsans (initializing)             ()
    1 minute input rate 216 bits/sec, 27 bytes/sec, 0 frames/sec
    1 minute output rate 200 bits/sec, 25 bytes/sec, 0 frames/sec
      3667 frames input, 440724 bytes
        0 discards, 0 errors
      3790 frames output, 521116 bytes
        0 discards, 0 errors
    last clearing of "show interface" counters Sun Apr 21 04:54:33 2013

    Interface last changed at Sun Apr 21 05:34:21 2013



Success!  Our VSAN is trunking across the link, a show fcns database helps us verify this:


switch# show fcns database

VSAN 10:
--------------------------------------------------------------------------
FCID        TYPE  PWWN                    (VENDOR)        FC4-TYPE:FEATURE
--------------------------------------------------------------------------
0x0f08d1    NL    21:00:00:0a:60:55:17:69 (Seagate)       scsi-fcp:target
                  [disk]
0x0f08d2    NL    21:00:00:11:d6:3e:ee:2e                 scsi-fcp:target
                  [disk1]




Finally an FCPING helps us verify 100 percent.


switch# fcping pwwn  22:00:00:04:cf:21:a5:2e vsan 10
28 bytes from 22:00:00:04:cf:21:a5:2e time = 1759 usec
28 bytes from 22:00:00:04:cf:21:a5:2e time = 287 usec
28 bytes from 22:00:00:04:cf:21:a5:2e time = 222 usec
28 bytes from 22:00:00:04:cf:21:a5:2e time = 304 usec
28 bytes from 22:00:00:04:cf:21:a5:2e time = 270 usec


At this point we have an FCoE Multihop topology.


If all you came for is how to configure an FCoE Trunk between the two 5k's you can stop reading now as now we are getting into a bit more detail


So the first thing I asked myself is, does this VLAN run spanning tree?


ToSanSWITCH# show spanning-tree vlan 10
Spanning tree instance(s) for vlan does not exist.


 Alright, that answers that question, but what happens if i assign a normal ethernet port to this FCoE VLAN?:

ToSanSWITCH(config)# int eth1/1
ToSanSWITCH(config-if)# switchport access vlan 10
ToSanSWITCH(config-if)# end


What does my show spanning-tree say now?

ToSanSWITCH# show spanning-tree vlan 10

VLAN0010
  Spanning tree enabled protocol rstp
  Root ID    Priority    32778
             Address     547f.eeaf.3a3c
             This bridge is the root
             Hello Time  2  sec  Max Age 20 sec  Forward Delay 15 sec

  Bridge ID  Priority    32778  (priority 32768 sys-id-ext 10)
             Address     547f.eeaf.3a3c
             Hello Time  2  sec  Max Age 20 sec  Forward Delay 15 sec

Interface        Role Sts Cost      Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
Eth1/1           Desg FWD 2         128.129  P2p



So according to this, the VLAN is forwarding on Eth1/1, at first I was quite confused and wondered to myself what kind of effect this would have on the FCoE Traffic, but it occured to me that the ethertype for FCoE Traffic is diffirent, what I imagine would happen is that Eth1/1 would receive copies of the FCoE Frames, but if the device plugged in at Eth1/1 has no idea what those frames are (i.e. doesn't recognise or want to use the ethertype) chances are it will just ignore them.


Regardless, this configuration had no ill effect on my VFC Interface as it was still showing as up:

ToSanSWITCH# show int vfc1
vfc1 is trunking
    Bound interface is Ethernet1/10
    Hardware is Ethernet
    Port WWN is 20:00:54:7f:ee:af:3a:3f
    Admin port mode is E, trunk mode is on
    snmp link state traps are enabled
    Port mode is TE
    Port vsan is 1
    Trunk vsans (admin allowed and active) (10)
    Trunk vsans (up)                       (10)
    Trunk vsans (isolated)                 ()
    Trunk vsans (initializing)             ()



I decided to see if I could convince it to "break", the first thing I tried was adding a VSAN to this interface that was not configured with an appropriate FCoE VLAN, which in this case was VSAN 1

I configured this on both ends:

 
switch(config)# int vfc1
switch(config-if)# switchport trunk allowed vsan add 1


The worst that happens is that the VSAN 1 stays in isolation mode:


switch(config)# int vfc1
switch(config-if)# switchport trunk allowed vsan add 1


 Next, I configured on one of the switches a port facing a server:

interface Ethernet1/1
  switchport mode trunk

  switchport trunk native vlan 20
  spanning-tree port type edge trunk


!

interface vfc10
  bind interface Ethernet1/1
  switchport trunk allowed vsan 10
  no shutdown

!

vsan database
  vsan 10 interface vfc10

!


The server port comes up fine:

switch# show int vfc10
vfc10 is trunking
    Bound interface is Ethernet1/1
    Hardware is Ethernet
    Port WWN is 20:09:54:7f:ee:af:1c:bf
    Admin port mode is F, trunk mode is on
    snmp link state traps are enabled
    Port mode is TF
    Port vsan is 10
    Trunk vsans (admin allowed and active) (10)
    Trunk vsans (up)                       (10)



At this point I am determined to make it my mission to break this FCoE Link as when I first tried to set this up, I had major problems, I want to see what configuration or what order of configuration you have to do for it NOT to work.

Some of you may have noticed that despite what the cisco configuration guide mentions I did NOT have to add these trunk interfaces to my vsan database, i.e. i did NOT need the following configuration:

vsan database
  vsan 10 interface vfc1




I decide to add this config to see if this kills the VFC, it does not:



switch# show int vfc1
vfc1 is trunking (Not all VSANs UP on the trunk)
    Bound interface is Ethernet1/10
    Hardware is Ethernet
    Port WWN is 20:00:54:7f:ee:af:1c:bf
    Admin port mode is E, trunk mode is on
    snmp link state traps are enabled
    Port mode is TE
    Port vsan is 10
    Trunk vsans (admin allowed and active) (1,10)
    Trunk vsans (up)                       (10)



So at this point I have not been able to get the VFC not to work for VSAN 10, it does not work for VSAN 1 but that is because I do not have an equivilant FCoE VLAN for that VSAN, which I can totally understand.

 Next I try making the Ethernet interface carry both LAN and SAN Traffic:


interface Ethernet1/10
  switchport mode trunk
  switchport trunk allowed vlan 10,20

switch# show spanning-tree int eth1/10

Vlan             Role Sts Cost      Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
VLAN0020         Desg FWD 2         128.138  P2p


This does not faze the device and the VFC interface remains up:


switch# show int vfc1
vfc1 is trunking (Not all VSANs UP on the trunk)
    Bound interface is Ethernet1/10
    Hardware is Ethernet
    Port WWN is 20:00:54:7f:ee:af:1c:bf
    Admin port mode is E, trunk mode is on
    snmp link state traps are enabled
    Port mode is TE
    Port vsan is 10
    Trunk vsans (admin allowed and active) (1,10)
    Trunk vsans (up)                       (10)


At this point I decide to up the ante by adding another interface:


interface Ethernet1/20
  switchport mode trunk
  switchport trunk allowed vlan 10,20

!
 interface vfc2
  bind interface Ethernet1/20
  switchport mode E
  no shutdown

!


This new interface comes up just fine:

switch# show int vfc2
vfc2 is trunking (Not all VSANs UP on the trunk)
    Bound interface is Ethernet1/20
    Hardware is Ethernet
    Port WWN is 20:01:54:7f:ee:af:1c:bf
    Admin port mode is E, trunk mode is on
    snmp link state traps are enabled
    Port mode is TE
    Port vsan is 1
    Trunk vsans (admin allowed and active) (1,10)
    Trunk vsans (up)                       (10)
    Trunk vsans (isolated)                 ()
    Trunk vsans (initializing)             (1)



I even get nice load balancing across the links:

switch# show fspf database vsan 10

FSPF Link State Database for VSAN 10 Domain 0xd8(216)
LSR Type                = 1
Advertising domain ID   = 0xd8(216)
LSR Age                 = 341
LSR Incarnation number  = 0x80000013
LSR Checksum            = 0x79e0
Number of links         = 3
 NbrDomainId      IfIndex   NbrIfIndex    Link Type         Cost
-----------------------------------------------------------------------------
    0x0f(15) 0x00040000     0x00040000               1          125
   0x9a(154) 0x001e0000     0x001e0000               1          100
   0x9a(154) 0x001e0001     0x001e0001               1          100

So at this point, everything is working pretty well. I decide to try making the link between the two switches a port-channel:





feature lacp

 
interface Ethernet1/10
  switchport mode trunk
  switchport trunk allowed vlan 10,20
  channel-group 20 mode active

 !
interface Ethernet1/20
  switchport mode trunk
  switchport trunk allowed vlan 10,20
  channel-group 20 mode active

!
interface vfc1
  bind interface port-channel20
  switchport mode E

  switchport trunk allowed vsan  10
  no shutdown

!

The VFC Int is still up and carrying traffic:


switch# show int vfc1
vfc1 is trunking (Not all VSANs UP on the trunk)
    Bound interface is port-channel10
    Hardware is Ethernet
    Port WWN is 20:00:54:7f:ee:af:1c:bf
    Admin port mode is E, trunk mode is on
    snmp link state traps are enabled
    Port mode is TE
    Port vsan is 10
    Trunk vsans (admin allowed and active) (1,10)
    Trunk vsans (up)                       (10)
    Trunk vsans (isolated)                 ()


At this point I don't understand why I had trouble getting the FCoE VE Int to come up the first time but am willing to accept that the feature works perfectly and it must have been some sort of order of operation issue.










CCIE DC: Roll your own OTV

Hi Guys!

So today I played with OTV and managed to get it going on the new Cisco Cloud Services Router (CSR 1000V)

In this blog post I am going to show you the receipe I used to get this going :).

You will need:

  • An ESX Server to run virtual images on
  • Two guests on that VM running Windows or Linux (doesn't really matter)
  • Two Cisco Cloud Services Router VM Guests
  • Nexus 1000V and a VMWare DvSwitch, if no Nexus 1000V is available you can get away with just two DV-Switches :)

Basically, your going to have a topology that looks a little something like this:

Now if you haven't got the Cisco Cloud Services Router or you haven't set that up, which is really the first step you should check out the blog post from INE that explains how to configure it in VMWARE here.

OK, now that's set up. Let's quickly look at our topology.


So basically as you can see from the topology diagram our goal is to get the server 172.28.0.11 talking to 172.28.0.20 even though they are not in the same Layer 2 domain by using the CSR Routers to perform OTV for us :).

Let's quickly look at the initial configuration on the CSR Routers:

CSR 1:
interface GigabitEthernet1
 ip address 172.27.0.1 255.255.255.252
end


CSR 2:

interface GigabitEthernet1
 ip address 172.27.0.2 255.255.255.252
end



Can we ping across these hosts?

CSR1#ping 172.27.0.2
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 172.27.0.2, timeout is 2 seconds:
!!!!!


Yes we can, so now we get onto the next stage.


The next stage involves configuring our dvSwitch and our Nexus 1000V, if we are using the Nexus 1000V basically I create a port profile that trunks up the VLAN to my CSR:

port-profile type vethernet EXTEND_SITE_TRUNK
  vmware port-group
  switchport mode trunk
  switchport trunk allowed vlan 499-501
  no shutdown
  state enabled



VLAN 499 is my VLAN that the server will sit on, so I have created a port profile for that:


port-profile type vethernet EXTEND_VLAN_SERVER_1
  vmware port-group
  switchport access vlan 499
  switchport mode access
  no shutdown
  state enabled


Now my server and my CSR #1 Router can see each other once I assign the port profiles to the appropriate VM's.


Ok now we have to duplicate the same sort of thing on the dvSwitch, because in this example we are trying to make the two hosts be on totally seperate switches that are not connected in any way. To do this I created a DVSwitch with No Uplinks and assigned from there:












As you can see I set the server to use the untagged port group and the Router to use the Trunk Port Group, the reasoning for this will become clear later.


Ok now I have all the basic infrastructure setup, let's get onto the OTV Config.

First thing I need to do on both CSR's is specify a site identifier and a site VLAN (Or in the case of CSR, which is not like configuring OTV on a nexus 7000, I specify a Bridge-domain)

CSR1(config)#otv site bridge-domain 1
CSR1(config-otv-site)#exit

CSR1(config)#otv site-identifier 0x2

And then the same on CSR2 but with a diffirent site Identifier


CSR2(config)#otv site bridge-domain 1
CSR2(config-otv-site)#exit

CSR2(config)#otv site-identifier 0x1


Now we need to configure the actual OTV Interfaces, the first thing to do is go to our join interfaces (which are the 172.27.0.0/30 links we setup previously) and specify IGMP Version 3 to support SSM multicast

CSR1(config)#interface GigabitEthernet1
CSR1(config-if)# ip igmp version 3

!

Now we repeat the above on CSR2 and then we are ready to configure the OTV Interface:

CSR 1:
interface Overlay1
 no ip address
 otv join-interface GigabitEthernet1
 otv use-adjacency-server 172.27.0.1 unicast-only
 otv adjacency-server unicast-only

!

CSR 2:


interface Overlay1
 no ip address
 otv join-interface GigabitEthernet1
 otv use-adjacency-server 172.27.0.1 unicast-only
!


In the above example I have used an adjacency server that lives on CSR, by default OTV wants to use multicast but I had trouble getting this going with multicast, I believe the issue is related to the way I have the CSR's join interfaces connected to each other, so I did this to make life easier for myself for now.

As you can see one router (CSR1) is the adjacency server and I have configured CSR2 to point to this, you can have multiple adjacency servers specified for redundancy if you like.

OK now we need to configure the "extend" interface on our inside of our CSR, which is the interface on VLAN 499 on each switch respectively that heads down towards our end host servers.

Here is the configuration there:

interface GigabitEthernet2
 no ip address
 negotiation auto

 service instance 2 ethernet
  encapsulation dot1q 499
  bridge-domain 200
 !
end



I believe that the CSR has the same restriction that a Nexus 7000 has in terms of you can't have IP addresses assigned to interfaces that your trying to extend over the OTV. If anyone knows for sure if this is possible/not possible please post in the comments section :).

So the most important part of this configuration is where we specify a service instance, this will be how we map our OTV VLAN, as you can see the VLAN number is set here as well as a bridge domain.

If we now go back to our OTV interface:

interface Overlay1
 no ip address
 otv join-interface GigabitEthernet1
 otv use-adjacency-server 172.27.0.1 unicast-only
 service instance 2 ethernet
  encapsulation dot1q 499
  bridge-domain 200
  no shutdown

!
 
end


 Here we are actually mapping that service instance we created, now when we perform a no shut we should see the OTV tunnel come up, here are some useful troubleshooting commands to verify:


CSR1#show otv detail
Overlay Interface Overlay1
 VPN name                 : None
 VPN ID                   : 1
 State                    : UP
 AED Capable              : Yes
 Join interface(s)        : GigabitEthernet1
 Join IPv4 address        : 172.27.0.1
 Tunnel interface(s)      : Tunnel0
 Encapsulation format     : GRE/IPv4
 Site Bridge-Domain       : 1
 Capability               : Unicast-only
 Is Adjacency Server      : Yes
 Adj Server Configured    : Yes
 Prim/Sec Adj Svr(s)      : 172.27.0.1
 OTV instance(s)          : 0
 FHRP Filtering Enabled   : Yes
 ARP Suppression Enabled  : Yes
 ARP Cache Timeout        : 600 seconds



This first command shows us that the AED has been elected (it's us), that our join interface is Gi1 and that our bridge ID is 1, also that we are using an adjacency server and that we ourselves are configured as an adjacency server. If you have errors in here you should revisit your config and make sure your not missing any of the critical commands such as the site bridge domain.

The next useful troubleshooting command is to establish if an ISIS adjacency exists between the hosts:


CSR1#show otv isis neighbors

Tag Overlay1:
System Id      Type Interface   IP Address      State Holdtime Circuit Id
CSR2           L1   Ov1         172.27.0.2      UP    25       CSR1.01 


If the OTV is successful the only state for this should be UP, you will also notice that you can see the hostname of your peer.

CSR1#show otv adjacency
Overlay 1 Adjacency Database
Hostname                       System-ID      Dest Addr       Up Time   State
CSR2                           001e.4925.fc00 172.27.0.2      00:20:11  UP 

!

Here is another helpful command that just helps you verify the connectivity.

So far, it looks like everything is up and we should be good to go, let's do a ping from our host!








Success! Our OTV Tunnel is carrying the traffic, we can do some verification here too (this will also be helpful to you when troubleshooting if it does not work)




CSR1#show otv route

Codes: BD - Bridge-Domain, AD - Admin-Distance,
       SI - Service Instance, * - Backup Route

OTV Unicast MAC Routing Table for Overlay1

 Inst VLAN BD     MAC Address    AD    Owner  Next Hops(s)
----------------------------------------------------------
 0    499  200    0050.5601.face 50    ISIS   CSR2
 0    499  200    0050.56a0.1aad 40    BD Eng Gi2:SI2


As you can see this shows some very useful information, you can see that the above routes are in the ISIS routing table, and what bridge-domain and VLAN they are attached to, also the owner helps you know if its local to you or over the OTV tunnel.

You can also see that it knows that one of the Mac's is actually pointed out interface Gi2:SI2 (Gigabit 2, Service Instance 2)


So this is quite a helpful command. Another useful command is:

CSR1#show otv arp-nd-cache
Overlay1 ARP/ND L3->L2 Address Mapping Cache
BD     MAC            Layer-3 Address  Age (HH:MM:SS) Local/Remote
200    0050.5601.face 172.28.0.11      00:02:46       Remote


This will show you the ARP cache that the router will respond with whenever it see's an ARP request, this is part of the proxy-arp function that OTV performs to ensure that ARP flooding is not occuring too often, this would be a good place to look if you had recently changed IP address or recently changed a host on the OTV and where wondering why you couldn't see it, it could be that the ARP cache is out of date.


Finally, if your using the Nexus 1000V you can also use the following helpful troubleshooting command:

DCNexus1000V# show mac address-table vlan 499
VLAN      MAC Address       Type    Age       Port                           Mod
---------+-----------------+-------+---------+------------------------------+---
499       0050.56a0.115e    static  0         Veth11                         3 
499       0050.56a0.1aad    static  0         Veth2                          3 
499       001e.bd93.82bc    dynamic 15        Veth11                         3 
499       0050.5601.face    dynamic 248       Veth11                         3




So Veth11 is the interface on my Nexus 1000V that goes towards my CSR router, so the fact that I have learnt the mac 0050.5601.face via this interface shows that the traffic is coming over the OTV.


I hope this helps someone out there, especially my fellow CCIE DC candidates.