Monday, December 10, 2007

Modern Technology

VoIP really bothers me sometimes... Why is it that even after YEARS of dealing with this stuff do some things just seem to be ridiculously complicated? For instance - today a Snom 360 arrived. My goal is to get this thing ready to integrate with Star2Star. That means:

  • Remote firmware upgrades
  • No (little) touch provisioning
  • Speed dials, monitoring, etc
I've had this thing on my desk for a little over an hour and the first requirement (firmware upgrade) cannot be met because the damn things HTTP/HTTPS server disappears a few seconds after the phone boots up. WTF? Yes, I am working on this now, I am writing this now, and I am angry now. I've been working with this stuff for years and I am AMAZED it still takes this long to get a phone working. Call this progress (no pun intended)? I don't think so. Fifty years ago (if I were alive, I suppose) I could go buy any analog phone, plug it in, and carry on with my life. Instead I'm wasting it away with this phone/computer Frankenstein sitting on my desk.

Reminds me of my car (also German - BMW). About a month ago the remotes just stopped working. After taking it in a few times over the course of a couple of weeks, they FINALLY figured out what was wrong. They replaced almost $1500 worth of parts (still under warranty, thank God) and spent days (literally) "upgrading and rebooting" various computer and software components to ensure compatibility with the new hardware. I get the car back and the computer had been completely re-initialized. Everything needs to be replaced and reprogrammed. Even after setting it up again, my Nokia E-70's bluetooth didn't work with the car. It is paired and recognized but any call results in no audio - makes it kind of tough to talk "hands free". Of course it worked quite well before the software upgrade...

Now I'm trying to figure out why the web server on the Snom keeps disappearing. Is it a bug (running firmware 7.1.8)? Some kind of "feature" (another example of German over-engineering, perhaps)? At the moment I'm leaning towards bug because this thing has got some other really interesting quirks... I changed the VLAN setting, rebooted, and still had the DHCP address from the original VLAN but it wasn't reachable. The phone had joined the new VLAN but did not obtain a new DHCP address. If this were in the field, this phone would be bricked (from a network perspective). If this were Grandstream I would understand (expect) this. From someone with a good reputation like Snom it is very disappointing!

VoIP, Bluetooth, Snom, BMW, Nokia. Are our lives REALLY any better?

Tuesday, August 28, 2007

Monitoring BGP Feeds

Two weeks! I can't believe it has been that long since my last post. That's just crazy!

I know that I can't make it up to you but I will share something I just finished up.

BGP is cool. It is so cool to turn it up on a new router and see the entire internet's routing table with a simple command "sh ip bgp". What's cooler than that?

Feeding that table into a database, that's what!

Putting your BGP feed into a database enables all sorts of cool things. As a matter of fact, some of these are so cool I haven't even thought of them yet! For the last hour or so I've been busy working on getting this going. Here's what you will need:

- BGP feed from an upstream provider, connected to a router
- Linux machine running Quagga
- Linux machine with Perl installed (can be same machine as Quagga, mine is)

First you will need to configure your router with BGP enabled:

en

conf t

ip as-path access-list 1 deny .*

router bgp [your ASN]
neighbor [Quagga IP] remote-as 64512
neighbor [Quagga IP] transport connection-mode passive
neighbor [Quagga IP] description Quagga peer
neighbor [Quagga IP] filter-list 1 in


You will want to make sure that this machine is directly connected. If it isn't you need multihop BGP (which I won't cover right now). Here's what we're doing:

- The as-path with deny updates from your Quagga machine. We don't want some misconfiguration to actually affect your network. We just want some routes from the Cisco!

- Create the neighbor with remote-as 64512 (private ASN)

- Don't initiate a connection to this peer, let them connect to us (passive)

- Apply the filter-list to inbound traffic for this neighbor (don't allow updates from Quagga)

Now we need to configure Quagga. First, you will need to install it. Your distribution should have some packages for you. Use yum, apt, etc to grab it.

It will probably install some config files in /etc/quagga. We only want to setup bgp. This should be a good sample bgpd.conf to get you started:

hostname [your hostname]
password changeme
enable password changeme
log stdout
log syslog
service advanced-vty
!
router bgp 64512
bgp router-id [Quagga IP]
neighbor [Cisco IP] remote-as [your real ASN]
neighbor [Cisco IP] description Internet BGP Feed
neighbor 127.0.0.1 remote-as 64512
neighbor 127.0.0.1 description local db hookup
neighbor 127.0.0.1 port 9179
neighbor 127.0.0.1 passive
neighbor 127.0.0.1 filter-list 1 in
neighbor 127.0.0.1 next-hop-self
neighbor [some public ip] remote-as 64513
neighbor [some public ip] description Remote devel
neighbor [some public ip] passive
neighbor [some public ip] ebgp-multihop 255
neighbor [some public ip] filter-list 1 in
neighbor [some public ip] next-hop-self
!
access-list 1 permit [local class C network] 0.0.0.255
access-list 1 permit 127.0.0.1
access-list 1 deny any
access-list 10 permit [local class C network] 0.0.0.255
access-list 10 permit [remote ip]
access-list 10 deny any
!
ip as-path access-list 1 deny .*
!
line vty
access-class 1


After you apply this, you will want to start bgpd: "bgpd -n". It will tell you which vty you can connect to with telnet:

telnet localhost 2605

That should work. At this point, you should have a connection up to your main router:

sh ip bgp sum
BGP router identifier (deleted), local AS number 64512
29 BGP AS-PATH entries
0 BGP community entries

Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
(deleted) 4 15092 580 129 0 0 0 02:06:29 9845
127.0.0.1 4 64512 167 944 0 0 0 00:58:59 0
(deleted) 4 64513 6 616 0 0 0 00:02:56 0

Total number of neighbors 3

Sweet! Hey, where did that connection from 127.0.0.1 come from? That looks strange...

That connection is the whole point of this exercise. Star2Star is interested in quality, internet routing and the relationship between the two (at least I am). Long ago we realized that having a full BGP feed and being able to analyze updates in real time could be a huge asset to our management and monitoring portfolio.

How could we do this? Within the last week it came up on NANOG. A guy named Bill Nash wrote a Perl script a while back using Net::BGP to connect to a BGP peer and dump it's routes into a MySQL database. This made much more sense than trying to pull them via SNMP or some other hackish mechanism.

Bill wrote this for another employer and no longer had access to it. Evidently enough people asked him about it off list for him to consider re-writing it. I wanted him to do more than consider... I wanted that script!

I contacted Bill and offered to support his work any way possible. This included setting up a read-only BGP feed for him. I didn't want to give him (no offense Bill) a direct connection to our Cisco router (crashing that would be BAD) so I came up with the setup above. The Quagga is read only (by configuration and with -n on the command line not even the local kernel can be updated). And the Cisco is read-only to the Quagga peer. Seems safe enough.

The Quagaa instance is merely a distributor for our BGP feed. That way I can mess with it all I want without any fear (or very little fear) of causing any problems for our main router. I can hammer it all I want with some alpha-quality perl scripts. Worst case (hopefully) I'll just hose Quagga if something goes wrong...

While waiting for Bill to get his Perl script going, I Googled BGP Perl to see if there was anything else out there. Sure enough, there is:

http://briangannon.com/2007/04/23/bgp-perl-route-analyzer/


This is a crude version of what I am looking for. I made a few minor changes because I needed it to run on the same machine as Quagga (already using TCP port 179). I also didn't want to have to run the perl script as root. Here is a mini-diff:

line 20:
-my $bgp = new Net::BGP::Process();
+my $bgp = new Net::BGP::Process( Port => 9179 );

Now when you follow the directions on Brian's blog to INSERT the BGP peer into the SQL table, make sure to just use localhost. Then the perl script will use port 9179 for itself. After all, if nothing needs to connect to it, who cares what the local port is (as long as the peer has been configured properly)? Quagga knows that peer 127.0.0.1 is on port 9179, and it works. Check this out:

6147167 | 4 | 216.134.176.0/22 | 2007-08-28 19:39:51 | 2 | Next Hop Changed,Metric Changed |
| 6147168 | 4 | 216.134.180.0/22 | 2007-08-28 19:39:51 | 2 | Next Hop Changed,Metric Changed |
| 6147169 | 4 | 216.85.83.0/24 | 2007-08-28 19:39:51 | 0 | Removal of network |
| 6147170 | 4 | 216.85.83.0/24 | 2007-08-28 19:39:57 | 1 | Added 216.85.83.0/24 |
| 6147171 | 4 | 216.85.83.0/24 | 2007-08-28 19:40:35 | 0 | Removal of network |
| 6147172 | 4 | 216.85.83.0/24 | 2007-08-28 19:41:22 | 1 | Added 216.85.83.0/24 |
| 6147173 | 4 | 207.250.244.0/23 | 2007-08-28 19:48:32 | 1 | Added 207.250.244.0/23

or the routes:

mysql> select * from route limit 9842,100;
+---------+-----------+------------------+-----------+--------+------------+---------------+
| id | router_id | prefix | next_hop | metric | local_pref | as_path |
+---------+-----------+------------------+-----------+--------+------------+---------------+
| 3045536 | 4 | 216.85.190.0/24 | 127.0.0.1 | | 100 | 15092 4323 |
| 3045638 | 4 | 198.102.2.0/24 | 127.0.0.1 | | 100 | 15092 4323 |
| 3045655 | 4 | 195.85.117.0/24 | 127.0.0.1 | | 100 | 15092 174 209 |
| 3045665 | 4 | 216.85.83.0/24 | 127.0.0.1 | 99999 | 100 | 15092 4323 |
| 3045666 | 4 | 207.250.244.0/23 | 127.0.0.1 | 99999 | 100 | 15092 4323 |
+---------+-----------+------------------+-----------+--------+------------+---------------+


That's from my MySQL db. Pretty cool, huh?

I'll be working more on this in the upcoming weeks. Bill will also be working on a much improved version of the BGP Perl script that I am working with now. I'll make sure to let everyone know how it goes!

Tuesday, August 14, 2007

Social Networking

As of today I rounded out my social networking portfolio.

A year ago I didn't belong to any of these "web 2.0"/"social networking" sites. First it was Orkut. I was in Brazil and it kept coming up. Why not? So I joined.

A couple months ago my friends in Sarasota, FL kept started really bothering me about MySpace. Why not? So I joined.

A couple of days ago I got enough LinkedIn invitations for me to break down and create an account.

Today I signed up for Facebook, much to my chagrin. I was already on three other sites, so why not?

I'll tell you why not. Now I am going to have people complaining about my outdated profiles, lack of interest, etc. Why create an account if you can't keep it up to date?

How am I supposed to maintain accounts on my personal/professional life spread across FOUR different social networking sites?!? This is madness. I can't wait to see what everything looks like in a few months...

So anyways if you are on any of these sites you should try to track me down to see how it all unfolds. I am sure it will be interesting!

Monday, August 13, 2007

Update!

I am still alive - barely.

I haven't been able to post over the last couple of weeks because Star2Star was busy getting another release out. We put out another release every six months (depending on schedule and delays) and you guessed it - it's that time of the year again.

Our latest release is 2.1. It includes a lot of fixes and feature improvements to the overall system, everything from Polycom firmware to OpenSER enhancements (lots of them).

Speaking of OpenSER, it looks like I will be working with it quite a bit over the next few weeks and months for the 2.2 release. I'll also have some interesting Cisco experiences, I'm sure...

So between getting this release out, a car accident, and regular life I have not had much time for this blog. Things should be getting back to normal pretty soon. I like it that way.

Tuesday, July 31, 2007

Getting Multihomed - Part 3/3









Following up to one of my first posts. We FINALLY brought up BGP with all of our providers. A call from our CEO to some people at Verizon got some things moving again. I had the circuit up with BGP the same day. Pretty amazing, huh?

Anyways, now my problem was dealing with the limited memory and tcam allocation for unicast routes. If you recall, I ordered three full BGP feeds from three different providers. With the internet pushing 226,000 routes my 3750G wasn't going to cut it:

sh platform tcam utilization

CAM Utilization for ASIC# 0 Max Used
Masks/Values Masks/values

Unicast mac addresses: 400/3200 13/44
IPv4 IGMP groups + multicast routes: 144/1152 6/26
IPv4 unicast directly-connected routes: 400/3200 13/44
IPv4 unicast indirectly-connected routes: 1040/8320 1023/8134
IPv4 policy based routing aces: 512/512 2/2
IPv4 qos aces: 512/512 8/8
IPv4 security aces: 1024/1024 23/23

Note: Allocation of TCAM entries per feature uses
a complex algorithm. The above information is meant
to provide an abstract view of the current TCAM utilization

So now I've got full feeds from three providers coming in. Luckily I read up on IOS route-map statements before I brought these BGP sessions up. Otherwise things could've gotten ugly. Here's what I started with:

ip as-path access-list 50 permit ^174$
ip as-path access-list 50 permit ^4323$
ip as-path access-list 50 permit ^701$

I started with just getting the ASNs we were directly connected to. And my tcam started to fill, but it wasn't close. I thought, hey, why not get some more routes while I can? I started to read up a bit more on route-maps and I figured out how to get other ASNs into my route table. I only want the networks of providers connected to my providers. Does that make sense?

Without being able to see the full table I would have no idea of what I was doing. What if I wanted Level(3)'s routes, for instance? I needed to see what was going on. Luckily an old client of mine runs FixedOrbit - the coolest site to look at BPG information. All I had to do was query my directly connected ASNs and start picking other routes I wanted. BGP would take care of the rest.

Here is a shortened version of what I ended up with:

ip as-path access-list 50 permit ^174$
ip as-path access-list 50 permit ^174_3356$
ip as-path access-list 50 permit ^174_33363$
ip as-path access-list 50 permit ^4323$
ip as-path access-list 50 permit ^4323_1668$
ip as-path access-list 50 permit ^4323_6983$
ip as-path access-list 50 permit ^4323_11456$
ip as-path access-list 50 permit ^701$
ip as-path access-list 50 permit ^701_19262$
ip as-path access-list 50 permit ^701_3356$

Now I have entries in my route table for my directly connected ASNs (174, 701, 4323) and some ASNs they are peered with - 3356, 33363, 1668, 6983, 11456, 19262. I don't have much room in my tcam but hey, that's what VXRs are for! Wow, I really want one of those (with an NPE-G2, of course) ;).

Friday, July 27, 2007

The Ultimate Geek Watch



















I used to think watches were completely unnecessary and stupid. After all, my cellphone is always with me and is always synced to the right time. Why do I need a watch?

Then I found a white G-Shock. This watch has it all:

  • It's white
  • Atomic synced
  • Shock resistant (tough)
  • Vibration alarm
  • Thermometer
  • Solar Power
  • World Time
  • Movement sensor

I bought it a few months ago from amazon.com. Evidently it's imported by Mister Watch from Japan. Sure enough it came in the mail, complete with price tag (in yen) and a Japanese-only user's manual (G-Shock is made by Casio, a Japanese company).

Because I don't read Japanese it has been difficult to discover all of the features of this watch. The "movement sensor" that I described above is a good example. I was out with my friends one night in a dark bar and I lifted my wrist to look at the time. The back light automatically came on once I twisted my wrist a certain way. At first I thought it was a fluke. My friends were convinced I was somehow controlling it with my wrist, mind, etc. Oh no, it was the movement sensor.

I think it works in combination with the solar panel because it only activates when it is dark (beyond a certain point). In a dark enough room, with just the right wrist snap, I never have to manually push the light button to see what time (or temperature) it is. That's a good thing too, because like any good American, I don't want to have to do ANYTHING that I shouldn't have to do and pushing watch buttons is no exception.

It's too bad that you can't get them anymore because many, many people have asked about the white G-Shock. If anyone knows where you can get them in the US please let me know!

UPDATE: Shinya Amano has translated AstLinux documentation for voip-info.jp and he has done some research on this watch for me. He found the English manual. Thanks Amano!

Wednesday, July 25, 2007

GoDaddy Sucks

As I write this GoDaddy has managed to completely screw up authoritative DNS for krisk.org. They have been my registrar for quite some time. Yesterday I decided to switch my authoritative dns over to them from DynDns. Why would I do this?

I needed e-mail forwarding. KrisK.org used to be hosted on a FreeBSD server that I ran. I started to get more and more busy with AstLinux and other misc. stuff so I moved as many of my services to free or managed solutions.

I couldn't do e-mail forwarding because GoDaddy's recommended MX records were CNAMEs. DynDns does not approve of this (every MX should be an A record). So I needed to move everything to GoDaddy just so I could use their stupid free e-mail forwarding (krisk.org -> gmail).

I tried to get to this blog today and blog.krisk.org wasn't resolving. What gives? I tried digging a few DNS servers that I knew of. All of them returned NXDOMAIN. That's not good. I ran whois and krisk.org to find the authoritative name servers. I was (and still am) on ns5.secureserver.net and ns6.secureserver.net. I tried to do directly against them:

kris@krislap:~$ dig @ns5.secureserver.net

; <<>> DiG 9.3.4 <<>> @ns5.secureserver.net
; (1 server found)
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 11126
;; flags: qr rd; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;. IN NS

;; Query time: 70 msec
;; SERVER: 208.109.78.180#53(208.109.78.180)
;; WHEN: Wed Jul 25 15:22:43 2007
;; MSG SIZE rcvd: 17

Same deal with ns6. That's not good. Some of my other domains on GoDaddy have ns1 and ns2. I tried to dig against those and they worked. The funny thing was ns5 and ns6 were not found as NS records.

I shuddered at the thought of calling GoDaddy support. I certainly don't want to talk to any of "those people". By "those people" I mean script reading drones that would ask me which version of Internet Explorer I was using...

I logged into the extremely horrible GoDaddy portal and clicked "Use default hosting name servers". They were listed as ns5 and ns6. I wasn't hopeful. A few minutes later it appears to be working again:

kris@krislap:~$ dig @ns5.secureserver.net www.krisk.org

; <<>> DiG 9.3.4 <<>> @ns5.secureserver.net www.krisk.org
; (1 server found)
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 11574
;; flags: qr aa rd; QUERY: 1, ANSWER: 2, AUTHORITY: 4, ADDITIONAL: 0

;; QUESTION SECTION:
;www.krisk.org. IN A

;; ANSWER SECTION:
www.krisk.org. 3600 IN CNAME godaddy.krisk.org.
godaddy.krisk.org. 3600 IN A 68.178.211.88

;; AUTHORITY SECTION:
krisk.org. 3600 IN NS NS1.SECURESERVER.NET.
krisk.org. 3600 IN NS NS2.SECURESERVER.NET.
krisk.org. 3600 IN NS ns5.SECURESERVER.NET.
krisk.org. 3600 IN NS ns6.SECURESERVER.NET.

;; Query time: 76 msec
;; SERVER: 208.109.78.180#53(208.109.78.180)
;; WHEN: Wed Jul 25 15:32:13 2007
;; MSG SIZE rcvd: 157


Woah! Look at that! ns1, ns2, ns5, and ns6 are listed as authority. Hmmm...

P.S. - If you are still reading this I need to tell you that DynDns is AWESOME!