Tuesday, June 8, 2010

Dynamips with 3725 and NM-16ESW modules

Does no one else have a problem with this setup? I've tried windows. I've tried multiple linux distros. Tried GNS3. I've tried dozens of idle-pc values, several IOS versions, multiple servers, different configurations and I still have issues.

What are the issue? Sometimes layer 2 sometimes layer 3. Sometimes arp entries will be incomplete on one of my four switches to a directly connected neighbor. CDP shows up fine. Almost always, all my layer 3 interfaces on one switch will just fail to work. What gives? The other three switches will be just fine!!

At first, I thought it was load. Ok, I have a few servers laying around. Fired it up, installed ubuntu 10, dynamips and dynagen and fired up only my 4 switch instances. Guess what? Still problems. I tried Windows. Still problems. Tried CentOS. Still problems. It is not just me - I've had several friends who have also had the same issues with NM16-ESW modules with the 3725 images.

So what was my solution? Use the 3640 image. It works EVERY DAMN TIME. Now granted, the feature set is different, and I can't do things like EIGRPv6 and I can't use the now-standard vlan commands in configuration mode like on t he 3725 - but it WORKS. I wasted a whole day trying to get this to work (again) instead of working on a full lab. Sigh. I've searched google to no extent. I've searched the dynamips/dynagen forums, INE's forums and still, I can't find anyone else with this issue.

So if anyone out there has a solution, I would be glad to hear it.

....now back to starting my full labs tomorrow.....

6 comments:

  1. I read on INE somewhere that you need to disable spanning tree on the switches until needed otherwise it uses up all your processing power. I have the same exact problem as you but haven't had time to really work on it.

    ReplyDelete
  2. Just as an update - perhaps the IOS version I was running was tainted. I am now running 12.5(15)T13 and it's running much better. I still come across the occasional problem with L3 interfaces between switches, but if I just bounce the ports - they return to normal and it doesn't happen all the time. Hope this helps someone!

    ReplyDelete
  3. what u mean by bounce the ports?? shut / no shut???

    ReplyDelete
  4. Yes - shut then no shut the interfaces, both at the same time. Sometimes you need to do this twice. Hey - no one said using Dynamips and saving yourself $15,000 in hardware costs was easy...

    ReplyDelete
  5. I spent almost a complete month working this issues, sadly one of the two months I requested at work to finish the CCIE studies (no good results as you might guess). But at least dynamips is now working perfectly. Here are my findings:

    1. Never use L3 interfaces at NM16esw, use L2 interface with vlans L3 instead.

    2. Never use more than one trunk between switches, use single trunk or etherchannel.

    These two recommendations made dynamips to runn for days without issues.

    You will have to do a lot of work with INE scripts by the way, I hope this helps people with same issues.

    ReplyDelete
  6. I second the two points that Luis mentioned. Having spent most of a day tinkering with the 3725 and a 16ESW module, I can also add:

    3) Do not use more one 16ESW per chassis as packets cannot be encapsulated because ARP resolution always fails. Only solution is to remove all but one 16ESW

    ReplyDelete