Tag: network

  • How to Add and Delete Persistant Routes in AIX

    Chatter-phone-300x234Wow, AIX does not like to make anything easy.

    Nor do they like to make things intuitive. Need to remove a route from AIX, well get ready to have one command to temporarily remove a route and another command to remove the route for good.

    Same goes with adding a route. “Quit your bitching, and use Smitty“, you say? Well smitty does not make anything any easier, especially since the UI likes to show you fields that you does not necessarily need you to use. Oh, plus they let you type in them. Asking me for a netmask when adding a static route does not seem like a crazy request to me? But jokes on you, you were not supposed to type anything there.

    Anyway, I was tasked with cleaning up a few bad routes that were added to a handful of servers. Note that I was not interested in adding the routes temporarily before I added them as persistent routes. These were simple one liners that I felt very comfortable adding to the ODM right out the gate.


    So first we need to check the ODM for the routes that we need to remove. In this example we want to delete the route fro 10.11.1.56, so lets find just that route. Note all of these are host routes.

    # lsattr -El inet0 | grep 10.11.1.56
    route host,-hopcount,0,,-if,en1,,,,-static,10.11.1.56,10.22.13.1 Route True

     Now lets delete the route above

    # chdev -l inet0 -a delroute=10.11.1.56,10.22.13.1

     We have a second route to delete so lets kill that one too

    # chdev -l inet0 -a delroute=10.11.1.57,10.22.13.1

     Now lets add the correct routes – the format is as shown below

    chdev -l inet0 -a route=host,-hopcount,1,-netmask,netmask,network,gateway

     Now on a few of the servers I was working on I had to remove network routes as well as host routes. You need to know that the syntax is anoyingly different adding network routes. In the example below I first need to remove the network route.

    chdev -l inet0 -a delroute="net,-hopcount,0,,,,,,-static,10.11.1.56,10.11.1.254"

     Now lets add our new and correct route

    chdev -l inet0 -a route="host,-hopcount,0,,,,,,-static,10.11.1.56,172.30.150.190"

  • Cisco: Principles of Application Centric Infrastructure

    Question-mark Attending Cisco Live 2014 in San Francisco this week?

    Well, get ready to hear a lot about ACI (Application Centric Infrastructure). It's almost here and its everywhere. And the acronyms, holy crap, get ready to get hit upside the head with of ton of those that you have never heard before. Cisco is only a close second to VMware when it comes to their love of acronyms.

     

    Here are a couple of them that I have heard for the first time today.

    • APIC
    • VTEP
    • EPG

    Anyway – here is Cisco's primer on ACI. Its a good read and might help you wrap your head around a few subjects before you head full bore into your sessions.
    Related articles

    Announcing Cisco UCS Director 5.0 – with Support for Application Centric Infrastructure
    Software Defined Network Services – A Sneak Preview at Cisco Live
    HomeLab: Upgrading Cisco IOS Via tftp on RHEL
  • HomeLab: Simple Cisco EIGRP Setup

    Sugar_skull_by_nickgo79EIGRP (Enhanced Interior Gateway Routing Protocol), is a Cisco proprietary routing protocol (until recently). When a router runs EIGRP, it keeps a copy of its neighbors routing table. If I router cannot find a route it its, or its neighbor's routing table, it will query its neighbors who in turn query their neighbors.

    Exactly how routing protocols work is serious business, but dont worry we are not going to go into that here. Below is the process that I used to setup EIGRP on a Cisco 2811.

    Before we do anything, lets get into Configuration mode (conf t).

    r-2811-1#conf t

     

    Your first step is going to be to enable IP Routing on your device. But before you do so, make sure that you have configured a Gateway of Last Restort. I did not and had to hook up the old console cable.

    The IP of this router is 10.1.0.2, and its directly connected to 10.1.0.1, which is its last resort first hop, so lets configure that .

    r-2811-1(config)#ip route 0.0.0.0 0.0.0.0 10.1.0.1

     

    Ok so now lets enable IP Routing

    r-2811-1(config)#ip routing

     

    Now lets start EIGRP and chose an AS number. Note that I used 10 on the other three routers in my setup so thats what we are going to use here.

    r-2811-1(config)#router eigrp 10

     

    Now we need to tell the router what networks are connected to it (or in this case, will be connected to it). This is the information that the router will share with its neighbors.

    r-2811-1(config-router)#network 10.3.0.0

     

    In this instance my ourside interface is on 10.1.0.0/16, and its inside interface will serve up 10.3.0.0./16.

    Dont forget to save your work

    r-2811-1#copy run start

     
    Related articles

    HomeLab: Simple SSH Setup on a Cisco Router
    HomeLab: Cisco 2621 Router Password Recovery/Factory Reset
    Hour 40: OSPF the new advanced link-state protocol
    Route Redistribution: Protocol Rules + EIGRP Redistribution
    Eigrp
    Hour 31: IGP Review EIGRP
  • HomeLab: The Cisco 3560G

    WS-C3560G-24TSThe Cisco Catalyst 3506G is a layer 3 switch which went end-of-life in 2009. For the home lab its a pretty nice switch to have due to its layer 3 support and gigabit speed. Mine is the model seen to the left, 24 gigabit ports and 4x1gb SFP uplinks.

     

    The Cisco Catalyst 3560 is available with one of two software images:

    • IP Base software includes advanced quality of service (QoS),
      rate limiting, access control lists (ACLs), Open Shortest Path First
      (OSPF) for routed access, and IPv6 functionality.
    • IP Services software provides a broader set of
      enterprise-class features, including advanced hardware-based IP Unicast
      and IP Multicast routing, as well as policy-based routing (PBR).

    Anyway, gathered here are a few manadatory resources for the Cisco Catalyst 3560G.

    Related articles

    Vlan Configuration between Cisco & sonicwall help
    Cisco WS C3560X 24T L 3560X Catalyst Switch
  • Broadcom (bnx2) Network Adapters Dropping Recieved Packets Under Linux

    VampSo a few weeks ago some of our Centos 5.4 and OEL 5.5 servers started exibiting strange connectivity problems. Monitoring started alerting that hosts were down when they weren't; some boxes could ping target hosts and some couldn't; some boxes became unresponsive when interfaces were failed over, and the strangest of all is that some of the boxes would magically "repair" themselves. Like I said, strange.

    Over the next week or so we ran into the issue a few more times and were able to see a pattern emerge. All the affected servers were running Centos 5.4 or Oracle Linux 5.5 and had broadcom (bnx2) adapters that were on the recieving end of some pretty decent traffic. Most importantly, all had a good number of dropped recieved packets that was continuously, albeit slowly, increasing.

    A bit of google research led us to this bugzilla, which suggested changing the adapter's coalescense settings…. so a bit on coalescense.

    Coalescense

    In your network adapter, coalescence is all about interupts. Traditionally interupt coalescense (or IC) is used to reduce the number of interupts generated by the system by delaying the generating of an interrupt by a very short period of time…think less then a milisecond. In turn more traffic will be recieved by the host and the next interupt generated will be larger in size. You can find out more than you would ever want to know about coalesence here

    The Fix

    So apparently the Broadcom IC settings were not aggressive enough. Packets would come in, fill up the receive queue, and get dropped before they could be sent off for processing via an interrupt. This takes us back to the bugzilla above and the suggested settings below which you set with the ethtool command

     ethtool -C ethX rx-usecs 8 rx-usecs-irq 8 rx-frames 0 rx-frames-irq 0

    Note that this was not an issue on any in Centos 5.6, any server with Intel adapters, or any server with 10g adapters. As a matter of fact, those servers had IC settings even more agressive then those above. See the Intel 82599EB 10-gigabit settings below

    rx-usecs: 1
    rx-frames: 0
    rx-usecs-irq: 0
    rx-frames-irq: 0

    Final Configuration

    So now that we know the fix we need to make it permanent, which is not as easy as editing a config file for the device as the coalescence config is set at boot and it part of the installed driver for the device. Rather than muck around with trying to modify the driver itself, we decided to set and configure our devices at boottime with a rc script that checks the checks the each network interface on the box and modifys their IC settings if they are using the bnx2 (Broadcom) driver.  We dropped the script below into /etc/rc.d and created a symbolic link to it in /etc/rc3.d.

    #!/bin/bash

    case "$1" in
    start)

    IFACE=$(ls /etc/sysconfig/network-scripts/ifcfg-eth*| grep -v bak | cut -d – -f 3)

    for ETH in $IFACE
            do
                    if ( ethtool -i $ETH | grep -qw bnx2 )
                    then
                            echo "$Changing Settings for $ETH"
                            ethtool -C $ETH rx-usecs 8 rx-usecs-irq 8 rx-frames 0 rx-frames-irq 0
                    else
                            echo "$ETH is not a broadcom"

                    fi
            done
    exit 1
    ;;

    stop)

    echo " hammer time"
    ;;

    *)
        echo "usage: $0 (start|stop)"
    ;;
    esac

  • XenServer Switch Ports Configuration Best Practices

    Old_switch
    Finally I have found it! Citrix's XenServer switch configuration best practices document.

    While everyone in the world has blog posts and documentation regarding how to set up and configure bridged networks in Xen, they hardly ever go into the physical switch configuration required.

    This is the document that you will need to pass along to your friendly Network Administrator, as they will more than likely not be familar with networking for Xen as its much different from networking for Vmware ESX.

    http://support.citrix.com/article/CTX123158

    The contents of the document above are outlined below.

    Change the following options on the switches for XenServer ports:

    1. Enable PortFast on XenServer connected ports.
      PortFast allows a switch port running Spanning Tree Protocol (STP) to go directly from blocking to forwarding mode by skipping the learning and listening modes. PortFast should only be enabled on ports connected to a single host. Port must be an 802.1q trunk port if you are using VLANS and the port must be in access mode.
      Ports used for storage should have PortFast enabled.
      Note
      : It is important that you enable PortFast with caution, and only on ports that do not connect to multi-homed devices such as hubs or switches.
    2. Disable Port Security on XenServer connected ports.
      Port security prevents multiple MAC addresses from being presented to the same port. In a virtual environment, you see multiple MAC addresses presented from Virtual Machines to the same port. If you have enabled Port Security, it shuts down the port.
    3. Disable Spanning Tree Protocol on XenServer connected ports.
      Spanning Tree Protocol must be disabled if you are using Bonded or teamed NICs in a virtual environment. Spanning Tree Protocol should be disabled because of the nature of Bonds and NIC teaming, to avoid failover delay issues when using bonding.
    4. Disable BPDU guard on XenServer connected ports.
      BPDU is a protection setting part of the STP that prevents you from attaching a network device to a switch port. When you attach a network device, the port shuts down and has to be enabled by an administrator.
      A PortFast port should never receive configuration BPDUs.
      Note
      : When BPDUs are received by a PortFast port, it indicates another bridge is connected to the port, and it indicates that there is a possibility of a bridging loop formation during the Listening and Learning phases. In a valid PortFast configuration, configuration BPDUs should never be received, so Cisco switches support a feature called PortFast BPDU Guard, which is a feature that shuts down a PortFast-enabled port in the event a BPDU is received. This feature ensures that a bridging loop is not formed, because the switch's shutting down the port removes the possibility of a loop forming.
  • How to Create A Dedicated Storage NIC in XenServer

    Wickenburg-networkIn Vmware ESX, when using NFS storage, you are required to create a separate and additional vmkernel portgroup to access your NFS storage. This way ESX Management traffic travels over one port group and NFS traffic travels over another.

    In Xenserver the concept is similar; however executed much differently.

    First and foremost you need to be aware of a few limitations in XenServer. While XenServer does allow you to create a dedicated nic (or bond) for NFS traffic, this nic must remain unmanaged by Xenserver. In contrast, a XenServer's management traffic travels over its "Management interface", which obviously has to be an interface that is managed via XenServer.

    Allow me sum this up. You cannot share a nic or nics between management traffic and NFS traffic. This means that you are going want to create a bond for management traffic and a bond for NFS traffic you are going to need 4 free interfaces on your Xenserver box.

    Now it is possible to use VLAN tagging and route your Management traffic and virtual machine traffic over the same physical interfaces. However its important to know that XenServer does not support VLAN tagging on the Management interface, so whatever VLAN you use for management, it must be the native vlan on the ports configuration.

    The information below is from the Admin Guide for XenServer 5.5

    Switch ports configured to perform 802.1Q VLAN tagging/untagging, commonly referred to as ports with a native VLAN or as access mode ports, can be used with XenServer management interfaces to place managementtraffic on a desired VLAN. In this case the XenServer host is unaware of any VLAN configuration.

    XenServer management interfaces cannot be assigned to a XenServer VLAN via a trunk port.

    Bottom line; its probably best to have seperate physical connections for your management traffic and NFS traffic.

    Also, according to the Admin Guide, your NFS network should not be routed. See the words below and read them thusly.

    "Before dedicating a network interface as a storage interface for use with iSCSI or NFS SRs, you must ensure that the dedicated interface uses a separate IP subnet which is not routable from the main management interface. If this is not enforced, then storage traffic may be directed via the main management interface after a host reboot, due to the order in which network interfaces are initialized."

    Ok now that we got all that out of the way, lets actually create our dedicated storage nic.

    First we need to get the uuid of the pif (physical interface) that we want to use. Note that this is just an example using a standalone interface.

    #xe pif-list host-name-label=<"your_hostname>

    Next we reconfigure our pif

    #xe pif-reconfigure-ip mode=statc IP=<your-ip-on-nfs-vlan> netmask=<your-mask> uuid=<pif-uuid>

    #xe pif-param-set disallow-unplug=true uuid=<pif-uuid>

    #xe pif-param-set other-config:ManagementPurpose="Storage" uuid=<pif-uuid>

    Alternatively you can use xe pif-forget to remote the interface from Xencenter database and configure it manually via the XenServer Kernel like you would any other interface in Linux, however this could be more confusing in the long run.