h1

Troubleshooting a Cisco 6500 crash

March 10, 2010


I was asked recently to share some knowledge about the support of the Cisco 6500 switches as the information available on the DOC-CD could be fairly overwhelming.

As it happens a clients Cisco-6509 switch fell over yesterday. I was called out to address the issue of the Cisco-6509 that decided it was tired of life by rebooting itself.  I’ll go through some of the steps I did to find the root cause. Obviously note the steps listed here will not find the cause of every possible issue with a 6500 switch, but can be used as a guideline.

Usually the first thing I would do is to see the reason for the reboot with a “sh version”. Look at the highlighted lines.

ndcbbnpendc0103#sh ver
Cisco Internetwork Operating System Software
IOS (tm) s72033_rp Software (s72033_rp-ADVENTERPRISEK9_WAN-M), Version 12.2(18)SXF6, RELEASE SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2006 by cisco Systems, Inc.
Compiled Mon 18-Sep-06 23:32 by tinhuang
Image text-base: 0x40101040, data-base: 0x42D90000

ROM: System Bootstrap, Version 12.2(17r)SX5, RELEASE SOFTWARE (fc1)
BOOTLDR: s72033_rp Software (s72033_rp-ADVENTERPRISEK9_WAN-M), Version 12.2(18)SXF6, RELEASE SOFTWARE (fc1)

ndcbbnpendc0103 uptime is 3 hours, 23 minutes
Time since ndcbbnpendc0103 switched to active is 3 hours, 22 minutes
System returned to ROM by s/w reset at 00:14:27 PDT Wed Sep 20 2006 (SP by bus error at PC 0x402DC89C, address 0x0)
System restarted at 09:13:44 ZA Wed Mar 10 2010
System image file is "disk0:s72033-adventerprisek9_wan-mz.122-18.SXF6.bin"

Obviously it is clear that the switch did a software reset caused by ‘bus error at PC 0×402DC89C‘.

Read the rest of this entry »

Posted in Switching | Tagged , | 1 Comment »

h1

OTV Part II

February 26, 2010


The second part about OVT in a previous post.

As promised, here are the slides from the presentation about OLV.
Feel free to download the slides Data-Center_Interconnect_Architecture_and_Solutions. (5mb)

  • Shivlu Jain did a post on OTV on his blog here.
  • Mr Lapukhov did a really great post here.

Posted in General info | Tagged | 1 Comment »

h1

Troubleshooting BGP

February 25, 2010


The new focus is of the R&S exam is troubleshooting. And for some reason this is seen as a new topic to study and as a result feared. It is vital to understand why troubleshooting was added to the lab, and why it will possibly be added to other tracks.  Anybody can apply vanilla configs, provided it is done without error, in the correct order, and by avoiding the question pitfalls.

Troubleshooting was introduced by Cisco to give the CCIE certification that edge it needs to seperate the guys that really understand the technologies and those that just learned to configure labs. Troubleshooting is thus NOT a new section! If you understand know each technology, understand its it building blocks, processes and states, troubleshooting should be nothing ‘new’.

That said once you understand the work, drafting a troubleshooting methodology per technology should be fairly straight forward. I am busy been drafting up a troubleshooting questionaire for each section that I will include in R&S Short-Notes v4.2. (Note ‘{ }’ curl-brackets indicate replaceble values, the rest is regex)

Troubleshooting BGP session start-up problems

1- Are you seeing the expected neighbors in a NON ‘idle’ or ‘active’ state?
#sh ip bgp summary

2- Is a sourced telnet to the neighbor address working?
#telnet {peer-ip} 179 /source {src-int-ip}

3- Confirm if the configuration is correct and matching to neighbors configuration?
#sh run | b router bgp

4- If eBGP, is the neighbor directly connected? (Should be 1 hop in the trace)
 4.1- If not directly connected is multihop configured?
#trace {peer-ip} source {src-int-ip}
#sh run | i {peer-ip}.*ebgp-multihop

Read the rest of this entry »

Posted in BGP, CCIE R&S | Tagged | 3 Comments »

h1

Overlay Transport Virtualization (OTV)

February 22, 2010


I am currently attending a 2-day Cisco PVT seminar for Service Providers, where the Cisco boys are sharing some of the new and upcoming technologies and hardware on the roadmap from 2010 onwards.

Cisco Systems are where they are today because of their groundbreaking innovation.
After seeing some of the specifics and configuration, OTV really seems impressive.

What is OTV (Overlay Transport Virtualization)?


Read the rest of this entry »

Posted in General info, Resources | 2 Comments »

h1

MPLS Quick Reference guide

February 15, 2010


I have updated the MPLS quick reference guide. Some of the guys enquired how detailed the new version4 topics are covered.

This is a great preview of the full MPLS chapter included in R&S Short-Notes.
This MPLS quick reference guide includes all the show and configuration command with descriptions. I have also included the newly added ‘Config-Sets’ to show how easy MPLS is configured in varies scenarios.  Please note this demo does not include any theory, or pitfalls to watch out for.

Feel free to download it :) But please let me know you views and comments

Download

Click to Download

Posted in CCIE R&S, MPLS | Tagged , , | 2 Comments »

h1

Welcome the 3rd Sextupple CCIE

February 10, 2010


It is absolutely incredible that Matthew White has successfully completed his wireless CCIE lab on the 27th of January 2010.  Along with the previous 5 CCIEs in less than 5 years, Matthew has just with joined to super élite Kings of the CCIE Mountain.

There are few words to describe the admiration for this huge accomplishment.Well Done Matthew!!!

See my TOP-CCIE post I did a while back.

Looks like space on top of the mountain are getting a little crowded. Michael could always do a seventh. :)

Posted in Non Tech stuff | Tagged | Leave a Comment »

h1

Working out Bc values quickly

February 9, 2010


I was asked today how to calculate the Bc values. The known formulas always add confusion. So the aim of this article is not to add more confusion, but offer an easy alternate way to calculate the Bc values used with shaping.

First lets review some basic shaping definitions.

CIR (Committed Information Rate)

  • Dictates the output rate one aims to average per second on the circuit/interface.
  • Book formula : CIR = Bc / (Tc/1000)

Tc (Time-Interval)

  • It is the time in milliseconds into which a second is divided for transmission intervals.
  • The Tc can’t be adjusted directly, but it can be changed by setting the Bc to a specific value..
  • The maximum value of Tc is 125ms (8 intervals per second) and the minimum value is 10ms (100 intervals per second).
  • Actually 8ms (125 intervals per second) on distributed platforms. On distributed platforms, the Tc must be defined in 4-ms increments. The nearest multiple of 4 ms within the 10-ms target is 8 ms.
  • Book formula :  Tc = (Bc / CIR) x 1000

Bc (Committed Burst Rate)

  • Bc is the number of committed bits allowed to be sent per interval (Tc) to conform with the target-rate (CIR) per second.
  • If Bc worth of bits are sent every interval in a second, the output rate is the CIR.
  • Book formula : Bc = CIR x (Tc/1000)

Read the rest of this entry »

Posted in General info, QOS | Tagged | Leave a Comment »

h1

Configuration Lock

February 5, 2010


Ever busy with a scheduled change, and the configuration all of a sudden differs from what you configured five minutes ago?

Normal IOS (not XR) behaviour allows multiple users to make instant changes to the running configuration. Occasionally two users make changes to the same config portion at the same time. One overwriting the others. ONLY the last commands entered will take effect.

The Configuration Lock  feature allows a one to have exclusive change access to the Cisco IOS running configuration, preventing multiple users from making concurrent configuration changes.

There are two modes:

  • Auto
  • Manual

Read the rest of this entry »

Posted in General info | Tagged | 3 Comments »

h1

What is Short-Notes all about?

January 31, 2010


CCIE Short-Notes is the result of an unplanned book I wrote during my studies for the CCIE Routing & Switching lab.

It is a good source for theory, and a great reference guide for the practical commands.  It also presents full configuration examples, in a completely new way to easily see how the technologies are implemented. Most topics include the DOC-CD locations, if more info is needed. Ultimately this is the only CCIE study guide, (I found) which can be used to review all the work start to finish the day before the lab.

I have always had a passion for training and development, so this is my first step to entering the training market. After 500+ hours, I have written and developed all my writings into a fully fletched book covering the content comprehensive in a proven study friendly format that still reads like study notes.

A deeper look at the origins…

Read the rest of this entry »

Posted in CCIE R&S | Tagged , | Leave a Comment »

h1

Getting your router to Tweet

January 20, 2010


Ok, so an earlier post sharing a really neat geek trick is awesome, but how the hell does one go about configuring a router to tweet something? (if you not a programmer)

To do it, you would need the following:

  • IOS image that supports EEM.
  • A twitter account.
  • A base64 encoded representation of you twitter account’s
  • Bruno’s twitter script. Download tweet-policy.tcl here.
  • The IP address of your nearest twitter server. (nslookup or dig will help you there)

The IOS obviously must support EEM.

Then once you have your twitter account, you need to encode your twitter account’s username:password to a base64 encoded representation.  Could be done using this website. Example:

twitter-username:tweet-password
        gives you
dHdpdHRlci11c2VybmFtZTp0d2VldC1wYXNzd29yZA==

Read the rest of this entry »

Posted in General info, TCL | Tagged , , | Leave a Comment »

h1

Updated Study Guide Chapters

January 19, 2010


I have updated the free Switching and OSPF study guides.

Feel free to download them at these posts:

01- Switching

07- OSPF

Posted in CCIE R&S | Tagged , | Leave a Comment »

h1

PfR Process flow

January 19, 2010


I came across a really good Performance Routing document, that I thought should assist R&S v4 candidates. It has really great examples of the different scenarios along with implementations.

Here is a depiction of the PfR process flow for OER configuration:

Source: Cisco Design Land

Posted in CCIE R&S | Tagged , | Leave a Comment »

h1

Using TCL/EEM to tweet SYSLOG events

January 17, 2010


Staying in the focus of the previous article, this is one of those really cool features, but possible something that you won’t easily use in production. Or maybe you would!

How about taking your routers syslog events and sending them to a twitter account. That way you can easily keep on heights when something in your network goes really wonky.Why would you want to do this?  To have a publicly accessable syslog replacement, or just because you can!

Bruno Klauser from Cisco wrote a TCL script using EEM to tweet routers syslog messages to a twitter account.  Here is an example of one tweeting router:  EASyDMI.

If you want to use this or give it a try, download the script at Cisco Land, and see my post on how to configure this.

Posted in TCL | Tagged , , | Leave a Comment »

h1

Tweeting Bits

January 14, 2010

Routing-Bits is now Tweeting-Bits too.

Yes I know I should have done this ages ago :(

But better late than never.

Feel free to visit twitter.com/routingbits

Posted in General info | Tagged | Leave a Comment »

h1

Route Selection with equal AD’s

January 7, 2010


I had a interesting question from a friend today.

Assume the following scenario:

Im going to exclude any MPLS connectivity, as it is not relevant.
The PE (Router1) connects the CE (Router3) with two links, one serial and one wireless.
This particular ISP runs mostly static routes to client sites (within the VRF’s) or alternatively eBGP.

On a wireless link it is always good practise to run BGP to detect when connectivity with the remote end is lost in the underlying Layer2 network. (Preventing a blackhole)
Regarding routing on the Serial Link, there as a default route out from Router 3 and a static route to 10.33.33.0/24 on Router1 pointing to Router3.

The client wants to load-balance traffic across both links. And the Admin Distance of the static route was set to 20 to match eBGP. (this is the scenario)

So the question : Why does Router1 not install both routes (the eBGP route and the Static), both with an prefix-length of /24,  a Admin Distance of 20, and metric of 0 into the RIB??

Read the rest of this entry »

Posted in General info | Tagged | 1 Comment »

Older Entries »