Getting Multihomed - Part 3/3

Following up to one of my first posts. We FINALLY brought up BGP with all of our providers. A call from our CEO to some people at Verizon got some things moving again. I had the circuit up with BGP the same day. Pretty amazing, huh?

Anyways, now my problem was dealing with the limited memory and tcam allocation for unicast routes. If you recall, I ordered three full BGP feeds from three different providers. With the internet pushing 226,000 routes my 3750G wasn't going to cut it:

sh platform tcam utilization

CAM Utilization for ASIC# 0 Max Used
Masks/Values Masks/values

Unicast mac addresses: 400/3200 13/44
IPv4 IGMP groups + multicast routes: 144/1152 6/26
IPv4 unicast directly-connected routes: 400/3200 13/44
IPv4 unicast indirectly-connected routes: 1040/8320 1023/8134
IPv4 policy based routing aces: 512/512 2/2
IPv4 qos aces: 512/512 8/8
IPv4 security aces: 1024/1024 23/23

Note: Allocation of TCAM entries per feature uses
a complex algorithm. The above information is meant
to provide an abstract view of the current TCAM utilization

So now I've got full feeds from three providers coming in. Luckily I read up on IOS route-map statements before I brought these BGP sessions up. Otherwise things could've gotten ugly. Here's what I started with:

ip as-path access-list 50 permit ^174$
ip as-path access-list 50 permit ^4323$
ip as-path access-list 50 permit ^701$

I started with just getting the ASNs we were directly connected to. And my tcam started to fill, but it wasn't close. I thought, hey, why not get some more routes while I can? I started to read up a bit more on route-maps and I figured out how to get other ASNs into my route table. I only want the networks of providers connected to my providers. Does that make sense?

Without being able to see the full table I would have no idea of what I was doing. What if I wanted Level(3)'s routes, for instance? I needed to see what was going on. Luckily an old client of mine runs FixedOrbit - the coolest site to look at BPG information. All I had to do was query my directly connected ASNs and start picking other routes I wanted. BGP would take care of the rest.

Here is a shortened version of what I ended up with:

ip as-path access-list 50 permit ^174$
ip as-path access-list 50 permit ^174_3356$
ip as-path access-list 50 permit ^174_33363$
ip as-path access-list 50 permit ^4323$
ip as-path access-list 50 permit ^4323_1668$
ip as-path access-list 50 permit ^4323_6983$
ip as-path access-list 50 permit ^4323_11456$
ip as-path access-list 50 permit ^701$
ip as-path access-list 50 permit ^701_19262$
ip as-path access-list 50 permit ^701_3356$

Now I have entries in my route table for my directly connected ASNs (174, 701, 4323) and some ASNs they are peered with - 3356, 33363, 1668, 6983, 11456, 19262. I don't have much room in my tcam but hey, that's what VXRs are for! Wow, I really want one of those (with an NPE-G2, of course) ;).

The Ultimate Geek Watch

I used to think watches were completely unnecessary and stupid. After all, my cellphone is always with me and is always synced to the right time. Why do I need a watch?

Then I found a white G-Shock. This watch has it all:

• It's white
• Atomic synced
• Shock resistant (tough)
• Vibration alarm
  
• Thermometer
• Solar Power
• World Time
• Movement sensor

I bought it a few months ago from amazon.com. Evidently it's imported by Mister Watch from Japan. Sure enough it came in the mail, complete with price tag (in yen) and a Japanese-only user's manual (G-Shock is made by Casio, a Japanese company).

Because I don't read Japanese it has been difficult to discover all of the features of this watch. The "movement sensor" that I described above is a good example. I was out with my friends one night in a dark bar and I lifted my wrist to look at the time. The back light automatically came on once I twisted my wrist a certain way. At first I thought it was a fluke. My friends were convinced I was somehow controlling it with my wrist, mind, etc. Oh no, it was the movement sensor.

I think it works in combination with the solar panel because it only activates when it is dark (beyond a certain point). In a dark enough room, with just the right wrist snap, I never have to manually push the light button to see what time (or temperature) it is. That's a good thing too, because like any good American, I don't want to have to do ANYTHING that I shouldn't have to do and pushing watch buttons is no exception.

It's too bad that you can't get them anymore because many, many people have asked about the white G-Shock. If anyone knows where you can get them in the US please let me know!

UPDATE: Shinya Amano has translated AstLinux documentation for voip-info.jp and he has done some research on this watch for me. He found the English manual. Thanks Amano!

GoDaddy Sucks

As I write this GoDaddy has managed to completely screw up authoritative DNS for krisk.org. They have been my registrar for quite some time. Yesterday I decided to switch my authoritative dns over to them from DynDns. Why would I do this?

I needed e-mail forwarding. KrisK.org used to be hosted on a FreeBSD server that I ran. I started to get more and more busy with AstLinux and other misc. stuff so I moved as many of my services to free or managed solutions.

I couldn't do e-mail forwarding because GoDaddy's recommended MX records were CNAMEs. DynDns does not approve of this (every MX should be an A record). So I needed to move everything to GoDaddy just so I could use their stupid free e-mail forwarding (krisk.org -> gmail).

I tried to get to this blog today and blog.krisk.org wasn't resolving. What gives? I tried digging a few DNS servers that I knew of. All of them returned NXDOMAIN. That's not good. I ran whois and krisk.org to find the authoritative name servers. I was (and still am) on ns5.secureserver.net and ns6.secureserver.net. I tried to do directly against them:

kris@krislap:~$ dig @ns5.secureserver.net

; <<>> DiG 9.3.4 <<>> @ns5.secureserver.net
; (1 server found)
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 11126
;; flags: qr rd; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;. IN NS

;; Query time: 70 msec
;; SERVER: 208.109.78.180#53(208.109.78.180)
;; WHEN: Wed Jul 25 15:22:43 2007
;; MSG SIZE rcvd: 17

Same deal with ns6. That's not good. Some of my other domains on GoDaddy have ns1 and ns2. I tried to dig against those and they worked. The funny thing was ns5 and ns6 were not found as NS records.

I shuddered at the thought of calling GoDaddy support. I certainly don't want to talk to any of "those people". By "those people" I mean script reading drones that would ask me which version of Internet Explorer I was using...

I logged into the extremely horrible GoDaddy portal and clicked "Use default hosting name servers". They were listed as ns5 and ns6. I wasn't hopeful. A few minutes later it appears to be working again:

kris@krislap:~$ dig @ns5.secureserver.net www.krisk.org

; <<>> DiG 9.3.4 <<>> @ns5.secureserver.net www.krisk.org
; (1 server found)
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 11574
;; flags: qr aa rd; QUERY: 1, ANSWER: 2, AUTHORITY: 4, ADDITIONAL: 0

;; QUESTION SECTION:
;www.krisk.org. IN A

;; ANSWER SECTION:
www.krisk.org. 3600 IN CNAME godaddy.krisk.org.
godaddy.krisk.org. 3600 IN A 68.178.211.88

;; AUTHORITY SECTION:
krisk.org. 3600 IN NS NS1.SECURESERVER.NET.
krisk.org. 3600 IN NS NS2.SECURESERVER.NET.
krisk.org. 3600 IN NS ns5.SECURESERVER.NET.
krisk.org. 3600 IN NS ns6.SECURESERVER.NET.

;; Query time: 76 msec
;; SERVER: 208.109.78.180#53(208.109.78.180)
;; WHEN: Wed Jul 25 15:32:13 2007
;; MSG SIZE rcvd: 157


Woah! Look at that! ns1, ns2, ns5, and ns6 are listed as authority. Hmmm...

P.S. - If you are still reading this I need to tell you that DynDns is AWESOME!

AstLinux 0.4.7

AstLinux 0.4.7, with Asterisk 1.2.22 and Zaptel 1.2.19 has been released.

Finding Asterisk

I've run my own Asterisk system for quite some time. My mom lives in Minneapolis, my dad lives in Chicago and my sister lives in Montreal (stinkin' French Canadians - the worst kind). Just kidding, I love my sister and all of my Quebec friends! At this point in time I lived in Wisconsin. Cell phone plans weren't that cheap yet (especially in Canada) and my family was spread out far enough for every call to be long distance.

I started playing with Asterisk in July of 2004. It turns out that I was flying to Montreal to visit my sister during the US Independence Day Holiday (yeah, that makes sense - go to Canada for the 4th). I bought a cheap X100P clone, installed it in a Linux machine, and connected it to the analog line (SBC) at my house in Wisconsin.

I configured a basic (dangerous) Asterisk install over the course of a few days and packed my things for Montreal. Of course I also brought my laptop. I had also managed to install a softphone on it before I left. I got to my sister's apartment, connected my laptop to her VideoTron cable modem and registered with my softphone. Would it actually work? Could I make a call from this softphone to that Linux server (over the internet) and out the PSTN in Wisconsin to one of my friends - for free?

If you are reading this blog you probably know what happened next. It worked perfectly. I was amazed and my friends were impressed. This was awesome.

Over the next couple of weeks I added what I needed to my (lame) Asterisk configuration. Then disaster struck... My air conditioning broke and with it went the hard drive of the Asterisk play server. It was a play server - no backups.

I hate doing things twice. That's exactly how I felt about this. I wasn't going to work through all of that again - compiling, configuration file mess again. I swore I wasn't going to resurrect Asterisk or that machine again.

Then I heard it. Asterisk was calling. I couldn't get the idea of Asterisk out of my mind. I hit the books (voip-info.org, internet mostly) and I learned more about Asterisk - more than enough to simply rebuild my play server. A couple of days later I had it back up and better than before. In the process I also knew what I was going to do with it. I could ship an ATA to each member of my family and we could all call each other, for free. I could also get an account with an ITSP and provide cheap calling to my family members.

Today some instance of that server is still running and my family members can still call each other with a four digit dial, for free.

What's my name?

This is going to be a different kind of post. This post might actually be useful for people trying to solve this problem. Just the facts, ma'am.

One of the things that has repeatedly come up in my line of work is CallerID name delivery in PRI (Primary Rate Interface) ISDN (Integrated Services Digital Network) configurations. I learned more about CallerID name today than I ever wanted to know. Just kidding - I love getting into stuff like this!

PRI is great because call setup is fast and CallerID information is available instantly. Or is it? I always knew that Caller ID name is not carried over the PSTN (usually - in some countries it is). The number does (obviously), but the name is usually looked up in CNAM by the terminating switch, not the originating switch. What I didn't know is that sometimes this isn't done when the initial Q.931 Setup message comes down the PRI to signal a new call.

Sometimes this CNAM lookup takes a little while (fractions of a second) and the name is sent later in a separate Q.931 Facility message. This is true. Cisco says so (PDF). A Cisco ISDN-SIP gateway can be configured to do this one of two ways:

1) Wait until you receive the Q.931 Facility message with name and shove it into the SIP INVITE using either PAI (P-Asserted-Identity) or RPID (Remote-Party-ID). Send the INVITE to the SIP proxy (or wherever).

2) Send the INVITE ASAP, and then send a SIP INFO packet when the name shows up in the Q.931 Facility message.

The default is #2, which is screwy. Very cool, but still screwy. It is much harder to design a SIP platform that can accept the initial INVITE, begin to process the call, and then append the PAI or RPID information received in the later INFO.

Thanks to Cisco I now understand more about Q.931 and ISDN. Now I need to get this "thing" to work.

My test setup:

PRI -> Asterisk -> PRI -> AS5350XM -> SIP -> OpenSER -> SIP -> Device

I need to get Caller ID with name delivery through this whole mess, from the first PRI to the last SIP device.

The LEC provided the PRI coming into the Asterisk machine. I provided everything else. I saw several roadblocks:

1) Get the CID Name from the LEC (via PRI)
2) Pass it through Asterisk
3) Get it to the 5350 (via PRI)
4) Get it to OpenSER (via SIP)

Knowing what I now know about Caller ID with name in ISDN I knew just what to do for Asterisk. In zapata.conf, my incoming context is lec-in. Here it is (from extensions.conf):

[lec-in]

exten => NXXNXXXXXX,1,Wait(1)
exten => NXXNXXXXXX,n,DoSomethingElse

Yep, that's right. All you need is to Wait a little to get that second Facility IE. Asterisk doesn't support getting the Facility IE later and it certainly doesn't support sending a subsequent SIP INFO. That's a good thing because as I said the "other" way (SIP INFO) just seems goofy to me.

Now I needed to get the CallerID name to the 5350. It didn't seem to work. I start looking at "pri debug span 3" output to see the Q.931 goodness coming from Asterisk. I fired up "debug isdn q931" on the 5350. No dice. It looked like this bug in libpri was killing me:

http://bugs.digium.com/view.php?id=9651

This was committed to libpri SVN about a month ago. I update libpri from SVN, recompile Asterisk, and install the new chan_zap.so. I give it another shot. It looks like the 5350 is now getting the name over Q. 931. Using ngrep I look at the SIP INVITE coming into OpenSER from the 5350. I have an RPID header, but it looks strange. The name field in the Remote-Party-ID header is "pending". What the heck is that about? "pending" was not what I was seeing in Asterisk!

I opened up ngrep a bit to let my see any SIP INFO messages that might be coming later. Sure enough shortly after the SIP INVITE comes a SIP INFO message with my Caller ID name. Going back to my two configuration choices on the 5350 I knew I preferred option #1 (send everything in one SIP INVITE), even if it meant there was a little delay before the caller got audio. How could I configure the 5350 to wait a little and put it all in one SIP INVITE before the Cisco fired it off to OpenSER?

I dug around on cisco.com for a bit. Nothing - at least nothing obvious. You have to love Cisco configuration and Cisco docs. I decided to look around the internet and see if anyone else had this problem.

I looked on Google and found this:

http://puck.nether.net/pipermail/cisco-voip/2005-June/005485.html

I wondered if Mr. Adam Rothschild ever found the solution to his (my) problem. I open up another tab and write him an e-mail. Three minutes later (literally) he sends me this configuration snippet:

---Begin IOS Configuration---
interface Serial3/0:23
no ip address
load-interval 30
isdn switch-type primary-ni
isdn incoming-voice modem
isdn supp-service name calling
isdn negotiate-bchan
no isdn outgoing display-ie
no cdp enable
exit
gateway
timer receive-rtp 1200
sip-ua
disable-early-media 180
retry invite 3
retry response 3
retry bye 3
retry cancel 3
timers buffer-invite 500

---End IOS Configuration---

Let's get away from this technical mumbo-jumbo and talk about people for a minute...

Mr. Adam Rothschild got an e-mail from a random stranger across the internet referencing an obscure technical problem that he had over two years ago. In less than three minutes he dug up the solution and wrote me back. I have a SmartNet support contract on this 5350 but I doubt the techs at Cisco could have helped me any better or faster than a nice guy (Adam) helping a stranger (me).

Wipe away your tears, you sentimental fool. We're getting back to configuration. This blog is hardcore. Couldn't you tell?

I applied Adam's config to my AS5350XM running IOS 12.4(15)T. Here is the SIP INVITE from the 5350 to OpenSER:

U 192.168.0.1:61306 -> 192.168.0.10:5060
INVITE sip:9418675309@192.168.0.10:5060 SIP/2.0.
Via: SIP/2.0/UDP 192.168.0.1:5060;x-ds0num="ISDN 3/1:D 3/1:DS1
1:DS0";branch=z9hG4bK901AB9.
Remote-Party-ID: "STAR2STAR COMM"
;party=calling;screen=no;privacy=off.
From: "STAR2STAR COMM" ;tag=971D8C-1203.
To: .
Date: Mon, 16 Jul 2007 21:57:24 GMT.
Call-ID: 5E99C7DC-331E11DC-8126E6C7-399CBB13@192.168.0.1.
Supported: 100rel,timer,resource-priority,replaces.
Min-SE: 1800.
Cisco-Guid: 1586976572-857608668-2150694933-1673067056.
User-Agent: Cisco-SIPGateway/IOS-12.x.
Allow: INVITE, OPTIONS, BYE, CANCEL, ACK, PRACK, UPDATE, REFER,
SUBSCRIBE, NOTIFY, INFO, REGISTER.
CSeq: 101 INVITE.
Max-Forwards: 70.
Timestamp: 1184623044.
Contact: .
Expires: 300.
Allow-Events: telephone-event.
Content-Type: application/sdp.
Content-Disposition: session;handling=required.
Content-Length: 288.
.
v=0.
o=CiscoSystemsSIP-GW-UserAgent 7275 8957 IN IP4 192.168.0.1.
s=SIP Call.
c=IN IP4 192.168.0.1.
t=0 0.
m=audio 20746 RTP/AVP 18 0 101.
c=IN IP4 192.168.0.1.
a=rtpmap:18 G729/8000.
a=fmtp:18 annexb=no.
a=rtpmap:0 PCMU/8000.
a=rtpmap:101 telephone-event/8000.
a=fmtp:101 0-16

Yeah yeah! Look at that Caller ID name in that Remote-Party-ID header! I feel like that's the best looking SIP INVITE I have ever seen. How does one SIP INVITE look better than any other? If you don't know the answer to that question, you haven't been following along.

I wrote Adam back to let him know how it turned out. He wrote me back again, happy to hear that it worked for me. Wow, just wow.

So many things shine through in this post. In one evening I found (and patched) a bug in libpri. I learned more about Q.931 and Caller ID. I found a guy to help me put it all together. The open source development model worked. The promise of easy access to information via the internet skooled me in ISDN. Social networking proved to be very effective, even while using pre-web 2.0 technology (e-mail). Google worked (a lot).

Now I get to put it all together in this blog post to give back a little. Hopefully the next guy (or girl) trying to get some mixed up mess of SIP and ISDN devices to work together with Caller ID Name delivery will get out of the office just a little bit earlier.

AdSense

It has been three days since my first post. I feel like I have done a pretty good job letting Google/AdSense know what this blog is going to be about. I've mentioned Cisco, Asterisk, AstLinux, trixbox, BGP, etc. What gives AdSense? I'd love to specifically mention the ridiculous things you are trying to sell my readers - but I won't. I won't out of fear that you will pick up on this post and not the others, sinking me even deeper into non-relevant AdSense hell. The ads on AstLinux.org are perfect. Give me some of that!

AstLinux and trixbox

I'm convinced that people like to compare things. It doesn't matter if it's an apples to apples comparison, it doesn't matter if it's an apples to Ritz cracker comparison. Someone, somewhere will compare it if they provide even a vaguely similar function. I mean hey, if you're hungry it could come down to choosing between apples and Ritz crackers, right? They are both food, aren't they?

Yesterday someone e-mailed me and started comparing AstLinux and trixbox. This has happened before, many, many times. The difference now? I have a blog! This is where I get to finally rant about all of this.

AstLinux and trixbox are both Linux distributions. This is true. They both run Asterisk. This is true. After establishing these two facts I start to have problems finding what else they have in common.

I'm going to start this off with a disclaimer: I have nothing against trixbox. trixbox is very popular and certainly fills a need. You can't argue with that. I myself have never really used it with users, etc. but I have downloaded the ISOs and VMWare images just to check it out. I also downloaded the "source" once.

trixbox is designed to be a turn-key "do anything and everything have anything to do with Asterisk or a related application". Trixbox is based off of CentOS (a good choice based on what they are trying to do) and includes some other open source software to accomplish their goals, all integrated together.

The person that e-mailed me yesterday also asked why AstLinux was so small. After all, it is pretty much impossible to find a Compact Flash card smaller than 512MB these days. What's with this 50MB business? Why should I care to keep it small?

He then mentioned that his trixbox install was "only" 1.8GB. I had to reread it to make sure he wasn't talking about something else. Only 1.8GB? Sure enough, I look at my Trixbox 2.2 VMWare image and it comes in at about 1.3GB. I guess I never noticed. That's huge.

I very quickly realized that meant that trixbox's install size was 26 times as large as AstLinux. In my VMWare session, it also uses over 200MB of RAM (doing nothing). AstLinux comes in at about 20MB of RAM (doing nothing). That's a smaller but still significant factor of 10.

Ok, now I've done it. I am bashing trixbox. You knew it was coming, you just wanted to see how long it was going to take. You've caught me.

Maybe not. If you need something to do whatever it is trixbox does, 1.8 (or 1.3) GB isn't really that bad. It just further demonstrates, for me, how different trixbox and AstLinux are:

- different users
- different environments
- different hardware
- different goals

It was at this moment a car analogy popped into my head. What is it with people and making computer/car analogies? We've all heard about the Bill Gates car analogy (I think it might be an urban legend, though). For some reason it is perceived that people fundamentally understand cars better than they do computers and when you need to demonstrate some relationship, you can compare a computer to a car. Here's what I came up with.

Ferrari makes some of the best performance cars in the world. These things are hand built with custom selected and designed components. They offer some of the most amazing performance on the road. 0-60 in under four seconds in almost all cases. With the Enzo you can almost break 3 (0-60, 3.14 seconds). That's awesome, unless...

- You have a family of four and a two seat Italian rocket doesn't get them to school
- You can't drive stick/paddle shifters (I can't - it's a hassle)
- You can't handle 500+ horsepower (I can't)
- You don't have a Ferrari dealer near you (where will it get fixed?)

Obviously there is the other issue of cost. All of this speed, performance, and prestige will cost you at least $300,000. But because this analogy talks about software that is free (I am talking about money), we'll leave that part out (for now).

Toyota makes some great cars, trucks, and vans. They consistently outsell many other major brands. If you:

- Have a family of four (and friends)
- Can't/don't want to drive stick (it is a hassle)
- Would like a calm, quiet, smooth ride
- Have a Toyota dealer around the corner

If this is you, you probably want a Toyota Sienna. That's a minivan. It has a variety of options for entertainment (TVs, DVD players, rear seat controls). More cup holders than you can imagine, cargo space, folding seats, etc, etc. All of these features are included. Sure you might not use the DVD player everyday (some people might never use it at all) but it would really help to shut those kids up on a long car trip. C'mon, you know it does!

Back to cost. Ferrari: $300,000. Toyota: $30,000 (that's a guess). Cost, in most cases is a barrier to entry. Billionaires think nothing of it to buy a Ferrari. Why not? They've got the money!

A similar barrier to entry exists in (some) free software. My non-super-technical friends always wonder what I am doing on the computer. It took me a while to figure it out but then one day my friend Kyle just asked me "How do you use a computer without a mouse?". Then the light went on. I've been using free software, Linux, etc for so long that I think nothing of it to sit down and spend hours tinkering with config files, source code, etc. All while never touching the mouse. Now I get it.

I (along with a few other select people) think nothing of it to get an AstLinux system, open up rc.conf, tweak the network interfaces and go to town on Asterisk config files.

My friends would have nothing of it. Give them trixbox and a mouse and they will get it done their way. There is nothing wrong with that.

The end result? We both get a phone system. Mine looks more like a Ferrari (hand tuned, hand written config files, base core, etc). Their's looks more like the Sienna minivan.

In this complicated web of comparisons the difference is I am the billionaire. Only my billions are not dollars, they are hours (ok, maybe minutes or even seconds) of experience and work with software, computers, networks, etc. Billionaires (my kind or the real kind) are not everywhere. Neither are people that can make an AstLinux system work. Knowledge and experience are the barrier to entry here. Not money.

My install is faster. More efficient. Uses much less power. The hardware is cheaper. You also need to keep me around so I can make changes to it (maintenance - I am the Ferrari dealer) when it needs it.

The trixbox install gets the same basic job (phone calls) done. It can also do CRM and all of that fancy jazz if someone ever wants it. That's awesome! My super-non-technical friends provided a great service to someone and made their life/business/etc better. Bravo!

AstLinux and trixbox:
- different users
- different environments
- different hardware
- different goals

Getting Multihomed - Parts 1,2

WARNING: This is going to be a long post. You probably won't make it to the end. I guess that's what happens when I go this long without having a blog.

Since moving into our colo in Tampa way back when, Star2Star has been getting blended bandwidth from our colo facility (E-Solutions). First they had three providers (Verizon/MCI/UUnet, Global Crossing, Level3). Then two (Global Crossing, Level3).

Starting in February, Global Crossing started having some big problems. Mostly packet loss in a router in Miami. Not only did it happen frequently, it was bad (%50 - %60 loss). You can imagine what that does for VoIP...

We are obsessed with quality, so about four months ago we decided to get multihomed. Seems easy enough, right? Get the right equipment, order some circuits, do the BGP thing.

Let's start with the good equipment. We have been using the awesome Cisco Catalyst 3750 to form our redundant switch stack (two 3750G-24-TS-1U configured with STP to the colo's 4500). My buddy Anton Kapela at Five9s Data suggested them. How I love these switches:

- Good stacking (Cisco StackWise)
- Good performance (65.7mpps - that's over 65 million packets per second across the backplane)
- Good performance, with features. That's right, you can do QoS, ACLs, etc at wire speed, per port (within the limits of the TCAM, obviously).
- 24 +4 port density in 1U (24 GigE copper + four GBIC slots)
- More router-type functionality (with EMI software image - gives BGP, etc)

So with a little configuration I should be able to use one of these (right now I just grabbed a spare) to form our BGP capable router to aggregate all of these circuits. Remember those great services I talked about before? Remember the tcam? Turns out that it can only handle about 8,000 unicast routes before it starts to drop into software forwarding/otherwise start to act up. Not that big of a surprise, with the current full BGP table on the internet pushing 225K+ routes the 128MB of RAM in the 3750 wouldn't have done much good anyways. With our configuration (providers directly connected, aggregated routes only) 8000 unicast routes should be just fine. Sure we lose some end to end visibility, but it's still better than what we've got now.

It might not be the perfect equipment (it's no VXR, that's for sure) but it should get us started. Now I have to order some circuits...

We take a look at the customers we have now, the providers they have, the big providers in the area, all of that good stuff. We determine we would like to get (in no order):

- Cogent
- Verizon
- Time Warner

Cogent! Yes, I know, Cogent. Cogent sticks out on that list. Let's start with them.

Dealing with the sales guy was great. Very responsive. The price (I'm sure you know) was tough to beat. Even better than price, there was another perk...

Remember that huge global internet routing table I was talking about? There are many advertised networks in it, all with varying sizes. Some are a full Class A (/8), some are less than a Class C (/24). Or are they? It turns out that most providers/network admins/BGP snobs filter any announcement smaller than a full Class C (/24). Make sense. That table is out of control! Router memory is expensive! There is old equipment! What are those less-than-a-full-class-C small fries doing messing with BGP anyways?

What is someone with a currently small network supposed to do if they want to multihome? We need BGP to control our own routing and peer with other networks and providers. We don't have enough machines to justify the current ARIN/ISP policy of %25/%50 utilization for IP addresses to get a full class C.

It turns out that ARIN has been thinking of us. That's why there is ARIN policy 2001-2. This policy, in short, says that if you can prove you are multihoming, your ISP can give you a full Class C no questions asked. Out of the three providers mentioned above Cogent was the only provider that had even heard of this and they were more than happy to do the allocation. Thanks Cogent! (Why does everyone hate them so much?)

Don't get me wrong. I am really interested in IPv6. I know global IPv4 address space is shrinking. Hacks like NAT are running out too and it is only a matter of time before we run out of IP address space and the internet comes to a halt. Whatever. At the rate Star2Star is growing we'll need all of those IPs soon enough. When the internet really needs IPv6 all of the really smart internet god types will figure it out. I'm not worried.

So we have one provider. We have a class C. Now we need to get an ASN. Before we do that, we need to make the various contacts that ARIN requires. I started with applications for the OrgID and NocID (I think - can't remember exactly now). Much to my surprise, the turnaround time on both of these was less than one hour even though I didn't start until about 5:30 PM on a Friday. I guess they don't believe in P.O.E.T.S Day over at ARIN!

We get approved for the ASN in a day or so. Fax in the contract. Give them their $500. Now we need to wait until the good 'ol US Postal Service delivers TWO copies of our signed contract. I guess ARIN really wants to hold us to that one. Don't worry ARIN I promise we'll keep up our end of the deal. You've been great so far. Two days later (USPS Florida -> Virginia) our ASN (15092) is approved and will be in WHOIS the next day. Fantastic!

We order two more circuits. One from Time Warner, one from Verizon/MCI. Time Warner installs the circuit fairly easily. They give us a /30 and everything is good. Now we just need BGP.

So far I have filled out the Time Warner BGP request form three times. No response, not even an automated one. I have e-mailed tech support. No response, not even an automated one. I have called tech support. They say they can't do anything until I fill out the form. They say they have no requests from me. What gives?!?!? I'm giving up on them for now. At least until next week Monday. I don't give up easily.

Next we deal with Verizon. This has been interesting. Most people know of at least three Verizon-type companies:

- Verizon Wireless (cell phones)
- Verizon local (the ILEC)
- Verizon Business (used to be MCI, I guess)

So far, I have heard the following business names while trying to order/turn up this circuit:

- Verizon
- Verizon Legacy
- Verizon Core
- Verizon Business
- MCI
- UUNET

That's right: UUNET. Are you kidding me? Having six different names for your company is confusing enough. Using UUNET certainly doesn't help. Last time I heard UUNET it was the nineties and I was in middle school. I had to look it up on Wikipedia just to make sure she wasn't totally confused. Turns out it goes something like this:

UUNET -> MCI -> Verizon Business

So far their name hasn't been the only thing they are confused about. I don't even want to get into it right now. I'll make sure to update everyone as the Verizon/MCI/UUNET saga continues.

What is most surprising about all of this? Cogent. With it's horrible reputation and low cost half the people I talk to still cringe at the mention of the "C word". I can tell you this: they have been (by far) the easiest to deal with. Amazing. We'll see how the service is but as of now I am a happy Cogent customer. Anyone that would like to argue about them can try to deal with some of these other characters. Let me know how it goes!

New AstLinux Site

So I've finally put up a new AstLinux site:

http://www.astlinux.org

The wonderful web developer at Star2Star (Gabe Shepard) set me up with a super awesome Drupal install. Man these (Drupal) guys have got the web portal/blog/CMS thing down to a science. I don't have enough experience with it yet to note the details, I just know I like it. It makes me feel good. Isn't that reason enough to love it? Anyways, for now you can enjoy the new site. I plan on spending a good part of the weekend updating it - adding content, fixing outdated info, etc.