Monday, December 22, 2008

Introducing Recqual

I've been waiting to talk about this one for a while.

Several months ago Star2Star was having problems with one of our upstream SIP carriers. We were starting to notice a large increase in the number of one way audio calls our customers were reporting.

When most people think of one way calls their first reaction is to blame SIP. Must be NAT! Must be a firewall! SIP sucks! Etc, etc.

I knew that wasn't the case. I just had to prove it.

I was convinced the problem wasn't SIP/UDP/IP related at all. We had multiple pcaps where we were sending RTP to the appropriate gateway. It just wasn't getting to the PSTN. Where was it going? When was this happening? Which gateways (out of hundreds) were the most problematic? We needed to know and we needed to know quickly.

I came up with and "wrote" recqual over a couple of days. After a few runs we were noticing patterns with problematic RTP endpoint IP addresses. Long story short, once these were identified we worked with the carrier to replace various bits of equipment (DSPs, line cards, etc). The one way audio problem has largely disappeared and we continue to run recqual. If this starts happening again we should know /BEFORE/ our customers do.

Of course I'm using Asterisk to place the calls. The best part of using Asterisk is it's multi-protocol flexibility. You should be able to test just about any combination of voice technologies - G.279a, G711, GSM, SIP, IAX, PRI, FXO, FXO, gtalk/jabber/jingle, skype, etc. The possibilities boggle the mind.

I've just been too busy to get it together and release this to the community - until now.

Tarball with instructions here.

Questions? Comments? Suggestions? Drop me a line.

Sunday, December 21, 2008

Consulting Time Available

I haven't blogged in a while but there is some good news...

I have made some time available for consulting work!

You might not be as excited about it as I am but this is a good thing. I'm looking for interesting projects, people, and companies to work with.

If you or anyone you know might be interested please contact me. Resume, references, etc available on request.

I'll be offering bonus time, discounts, and a few other potential incentives to anyone that lets me blog about my projects and/or release any work under a liberal (read: FOSS) license.

Between my change in schedule and some (hopefully) fun new projects you can all expect to see much more frequent blogging soon!

Monday, November 17, 2008

SBCs are Killing SIP

Wow... Over a month since my last post! My how time flies.

No time to reminisce or catch up. I've got a rant that needs to get out - NOW.

SBCs (Session Border Controllers) are killing SIP. Breaking SIP. Smothering SIP. Especially when used by "carriers". Carriers and their SBCs I tells ya.

SBCs, technically, are pretty cool devices. While I certainly understand their purpose they tend to be overused, misconfigured, and misunderstood. Many entities deploy SBCs without any idea of the other components (I'm looking at you, proxies) that make up a well designed SIP network.

Why do I hate SBCs so much?

1) SIP is cool because it is end to end and designed with intelligent endpoints in mind (endpoints that can think for themselves).
2) SIP is very flexible, especially with regards to handling media.
3) Ubiquity.

SBCs (especially when misconfigured) break many of these features:

1) SBCs (by design) hide endpoints from one another. Both endpoints support G.722? The SBC doesn't and it's going to rewrite the SDP with it's capabilities. Too bad.
2) SBCs (by design) handle media. While this can be good often times it isn't and there are other, less drastic ways to ensure quality of media.
3) When the only tool you have is a hammer, every problem starts to look like a nail.

My biggest concerns with SBCs relate to the last point. I swear, there are many providers, enterprises, etc that have deployed SIP in some capacity using ONLY SBCs and simple UACs and UASs. They've never heard of a proxy. Or a registrar. Heck, I'd even go for a signalling-only B2BUA and call it a compromise. Chances are they've never heard of that either.

I have dealt with several devices that break down, utterly fall apart when used with a proxy. I've covered it on this blog before. I'm just too mad to look up the link now. Again, $MANUFACTURER designs and markets a SIP device. They only test it against SBCs and they've (apparently) never heard of a proxy. Guess what happens...

Some poor soul like myself tries to deploy said device in what I consider to be a well designed SIP network. Unfortunately for me, this call path might not involve an SBC. Guess what happens? The device doesn't understand traversing proxies (Record-Route, Via, etc) and does something silly like parse the Contact header when trying to send a response. Call failure and all kinds of brokenness ensue.

So... I talk to $MANUFACTURER and get the standard "We've deployed this device thousands of times and never seen this problem before". Let's assume that's true. I don't know what's more depressing: the fact that they skipped over multiple sections of a basic SIP RFC like 3261 or the fact that no one noticed it for this long because (apparantly) no one uses proxies anymore. Ugh. Gross.

It's not just device manufacturers. Carriers do this too. Often times the actual issue lies with their SBC. Many carriers (especially those using ACME SBCs, it seems) parse To: instead of the Request-URI. Probably because their customers are using SBCs too and Request-URI and To: match. Not so with a proxy. I don't blame the carrier's use of an SBC. This makes sense. That's what they were designed for. However, please test your device and configuration against something other than another SBC.

What happens if your Request-URI and To: don't match? They send a 404! Yet another RFC3261 violation. Section 8.2.2.1 allows for a UAS to route based off To (although it doesn't sound preffered). However, for the love of God, if you are going to deny a request because of the content of a To header, please send a 403 as specified in the RFC. Your 404s are confusing and ignorant. Was it really not found, or are you just routing based off To instead of the Request-URI? Once again I blame SBCs and a world where it's becoming common for SBCs to talk to each other (and nothing else).

This is yet another situation where assumptions are made based on the behavior of SBCs. It's bad. Please stop.

Thursday, October 9, 2008

Submit Your SIP

Ever since I've started blogging and talking about SIP people have come out of the woodwork with SIP interop problems.

After giving a talk about SIP at Astricon 2008 I received several e-mails from audience members with specific SIP issues. I LOVE getting these e-mails.

Why? I love working on SIP issues. With all of the devices using SIP there is no shortage of interop problems. Just today a guy on the Asterisk mailing list had a problem with his Cisco AS5300 and Asterisk 1.2 Usually that wouldn't be a problem at all - many people (including myself) use this combination of hardware with great success.

Why was he having problems? His AS5300 was configured for GTD and Asterisk 1.2 (apparently) doesn't handle multipart SIP bodies very well. I was able to find a patch to Asterisk 1.4 to improve multipart body parsing. That was a fun one.

I got to thinking... There should be a place where people can exchange specific SIP interop tips and notes. Otherwise how are we supposed to get anything to work!?!?

I came up with such a place and it's called SubmitYourSip.com. I' ve started to fill it in a little but hopefully (with time) it will become somewhat of a SIP wiki (with a focus on interop, of course).

I'm just getting started on it but I'll be working on my MediaWiki syntax and going back through my e-mail to dig out some of these examples.

Friday, September 19, 2008

A preview: performance tests

I'm headed out the door for some sushi but I thought I'd drop in to give you an idea of what I'm working on for my next blog post. I'm hungry so let's keep this short and sweet: receive interrupt mitigation and its effects on Linux media applications.

In general I'm a big fan of receive interrupt mitigation. I'll trade some delay for a substantial decrease in system CPU time spent servicing interrupts resulting in the ability to handle more calls. I didn't just come to this one day, I've done some tests in the past to verify this.

However, I've never done a large scale test on regular, server class hardware. Usually just Asterisk on an embedded system. It usually works out well. This is why, by default, all ethernet adapters that support NAPI in Linux are enabled in AstLinux by default.

The folks at TransNexus spend a fair amount of time testing OpenSER/Kamailio/OpenSips performance. Today Jim Dalton posted the results of another test to the Kamailio User's mailing list. I replied to his post with so many questions I figured it might be time for me to lab this up myself and test my theories about interrupt handling (in Linux, specifically).

If those brighly colored rolls of fish weren't so distracting and delicious I'd promise to think about all of this over dinner. Unfortunately it will have to wait until tomorrow...

Friday, September 12, 2008

Distro Wars!

I don't like to get into Distro Wars... Nothing is more pathetic than a bunch of FOSS geeks sitting around getting religious about:

- Distros
- Editors
- Star Wars

Ok, ok I started to embrace the stereotype a little towards the end there but if you've ever seen one of these epic battles with your own eyes you just might relate it to Star Wars too.

There are always far more serious things going on in the world and these software hippies sit around and argue about what software to edit ASCII with. Ridiculous.

However, even I will throw down when people bring up the worst idea for a distro ever:

Fedora

There, I said it. I've officially fired my first shot in a war that has been raging in the Linux community since, oh, 1991 or so.

What's wrong with Fedora, you ask? Of all of the other hundreds of distros, why would I single out Fedora and waste my time writing about it? I'll tell you why:

Much like the iPhone, it's a joke.

I CRINGE. ABSOULTELY CRINGE when I see someone trying to do something serious with Fedora. At this very minute (3am), I am typing a blog post when I should be sleeping. Why you ask? Because I just saw a post on Asterisk-Users with yet another poor soul trying to bring up Asterisk on a Fedora system (Fedora 9). I couldn't possibly sleep knowing the abomination that is Fedora+Asterisk continues.

This is a perfect example to illustrate why Fedora is such a joke. One of the fundamental principles of Linux is reliability. One of the fundamental principles of telephony is reliability. By installing Asterisk on Fedora you are flying in the face of 18 years of Linux and 100+ years of telephony.

Fedora is BLEEDING EDGE. It serves as a test bed for RedHat's next REAL Linux release. How do you feel being a tester for what is sold as a commercial product by a profitable company? Bugs in Fedora are found and fixed quickly...

For six months

You can install Fedora and deal with the usual issues in beta software. Once things finally settle down (after six months or so), you get to upgrade to the next release and start all over again!

Sounds like a great plan for a server or PBX, right?

Hosting companies sell packages based on Fedora. Shame on them. People install PBX systems with Fedora. Double shame on them. The average life span for a PBX is seven years. This means that your Fedora Asterisk system is supported with updates for %7 of the typical PBX lifespan.

CentOS/RHEL/Ubuntu LTS and several other Novell/etc offerings are supported for five years or more.

Granted Fedora has it's place. It makes a nice toy, much like the iPhone. If you want to play with cutting edge Linux, Fedora is for you and it might work on a test system, desktop, laptop, etc.

But please. Please. PLEASE do not install it on a server and don't even think of using it for Asterisk. ANYTHING else will do. Seriously, I don't have a problem with any other distro. Pick one.

Monday, August 25, 2008

Free cell phones

Just in case you thought I was selling out to the US wireless industry with my previous post, check this out:

After looking over some phones on the Nokia website, I thought of all of those "free" phones carriers like to give away.

Here's one for comparison. The Nokia 6085 is offered in the US by AT&T wireless. Their website says it retails for $189.99. My gosh! Oh but don't you worry, as long as you sign up for two years of service (and do so online) we'll discount our $190 phone to $39.99 and then give you a $39.99 discount (you're buying online, remember). OMG! Free phone! See how that works?

Funny enough, Nokia offers the phone (just the phone) on their US website for $118. That means that right off the bat, AT&T wireless is jacking the price of the phone up $60 just for the pleasure of buying it from them.

Some of you might say "Hey, a $60 markup isn't that bad". Yeah right. AT&T wireless is NOT paying $118 for that phone. I wonder how many of them they sell and what kind of special pricing Nokia gives them. Probably not even close. Probably not even half that. I bet AT&T still makes money at the $39.99 price. I also wonder how many they sell at $189.99...

Here's the catch. They are going to give you the same contract and charge you the same price whether you get the "free" phone or bring your own. That's what sucks about the wireless industry in the US. Unlike the rest of the world, Americans can't be bothered to buy their own phone and bring it to the carrier for service. Maybe it's because we've got some different wireless standards and there could be confusion (iDEN, CDMA, GSM, etc). More than likely it's because the wireless carriers know they can make out like bandits otherwise.

The coupling of the phone to the service is inherently wrong and evil. Some carriers (Verizon Wireless) don't even activate phones who's ESNs they don't have in a database. All to "protect the network". Yeah right. If your network is going to go down because I want to use a CDMA cell phone I bought from Sprint a year ago you've got some serious network issues...

At least there is some light at the end of the tunnel. Soon enough cell phone carriers will have to pro-rate early termination fees. It's tough to lock someone into a contract with a $200 early termination fee to cover the cost of the $40 phone you "gave away" almost two years ago.

Nokia isn't ditching SIP

Someone sent me a link to a blog post about Nokia "turning it's back on VoIP". I know I can get pretty emotional from time to time, but at least I'm accurate when I do. At least I feel like I am, and that's all that matters, right?

Anyways, let's look at this post. The author points out that the new N78 and N96 no longer include the Symbian SIP client. This must mean Nokia is finally giving in to pressure from cell carriers in some huge scheme to enslave the mobile phone subscriber and direct all talk time over their network.

Follow up comments clarify the situation a bit. Nokia simply removed the interface to the SIP stack on the N series. It's still in the firmware and available to any third party developers, so the most "threatening" apps (Truphone, Gizmo, etc) will continue to work. People that just want to configure the phone to connect to their corporate/"personal" PBX will be out of luck. I love saying "personal" PBX. I have one. So do many of my friends. How geeky is that?

Why did the N series have a SIP interface in the first place? Anyone who has ever read my iPhone review knows that I believe teenagers and hipsters control a large chunk of the cellphone market which breaks down into three parts:

- Free (throw away) phones for moms, grandparents, and kids. Sign up for a two year contract and they're yours to keep! What a deal.
- Flashy phones for teenagers and hipsters (it's a better MP3 player/camera than phone)
- Serious business phones (Blackberry, Windows Mobile, most Symbian, etc)

The N series is (and always has been) a hipster phone. It's more likely to compete with the iPhone than the Blackberry or Nokia E-Series.

Speaking of the E-Series... The new Nokia E-71 still includes the SIP client. Nokia turning it's back on VoIP? I don't think so. Nokia learning a little more about their customers? Much more likely.

Thursday, August 7, 2008

OpenSER Update

I just got a very interesting Anonymous comment to my last post. One of my BIG questions was answered:

Kamailio or OpenSIPS?

For me that question was answered - OpenSIPS. Star2Star has purchased a block of consulting time from Voice System for OpenSER support. We used it once to flush out a bug in OpenSER 1.2. Other than that we have it for a little business security. It's nice to have an official avenue for support. Plus we like (financially) supporting Open Source software. It gives us so much, we should give a little back.

Anyways, this last post makes it pretty clear: OpenSIPS is Voice System's product. That's all I needed to hear. Thanks Anonymous commenter/blog reader!

Tuesday, August 5, 2008

More OpenSER Drama

My last rant on this blog covered the OpenSER name change to Kamailio. It's pretty obvious how I felt about it and it's even more obvious how upset (for lack of a better word) I was with the selection of the new name.

These new developments make a name change pale in comparison. This time I have something much more serious to fret over:

There has been a fork of OpenSER.

First some history. OpenSER started life as SER. Some time ago OpenSER was forked from SER for good reason (and a common one) - the company "sponsoring" the development of SER didn't understand Open Source. Community input was ignored. Patches took forever to be applied. IPTel just didn't get it. The developers (just about all of them) set out on their own to form OpenSER and create an Open Source friendly company to serve the needs of the OpenSER community - paid support, development, consulting, etc.

Almost everyone I know has (by now) abondoned SER and moved to OpenSER. Which is good because this fork (like any other) created a certain amount of fragmentation in the community:

- documentation
- support (mailing lists, etc)
- thirdy party support
- name recognition

Even with the SER and OpenSER projects generally moving in the same direction it was clear they couldn't be treated the same. Various changes were introduced in the configuration for each piece of software. They were (and are) mostly trivial, but you can't simply move an OpenSER configuration to a SER system (or vice-versa). You also can't ask an OpenSER question on a SER list (obviously). Granted most people are subscribed to both and can provide expertise on either product but it still creates a headache for the user - fragmentation of expertise, documentation, support, etc.

As I've said this doesn't seem to be that much of a problem anymore. OpenSER (Kamailio) is clearly where it's at. It's hard for me to describe how much respect I have for this software and it's developers. It is one of the most impressive products I have ever come across. It does what it is designed to do better than just about anything else I've ever seen.

As I have come to use OpenSER more and understand it better the activist in me begins to emerge. I feel pain anytime I see someone use a product where OpenSER could clearly do the job better. OpenSER doesn't get as much credit or use as it should. I can only theorize why this is but I do know one thing:

Forking doesn't help.

Any traction (that's for you, JJ) OpenSER has made over the last few years is being seriously threatened by these political shenanigans. In the last month or so I've reviewed the first OpenSER book and solicted OpenSER contributions for the upcoming O'Reilly Asterisk Cookbook. I was starting to feel like OpenSER was finally headed towards getting the exposure it deserves.

Most developers probably don't care about these things (exposure). They should, and if they don't the project admins should. And if they don't the biz guys at the company sponsoring the development should. Here's why.

Exposure/"traction":
- Attracts more users
-- More users for testing, bug reports, etc
-- More users to write documentation
-- More users to create revenue for the sponsoring company (consulting, etc)
- Attracts more developers. Open Source development is largely ego driven and the larger and more visible the project, the more developers (both good and bad) you attract.
- Attracts more third party interest in the project. Open Source software has a lot of holes for real business use. There is a huge potential for third party projects for OpenSER. Billing systems, desktop call managers, GUIs, etc.

Neither the SER community, OpenSER community, or OpenSIPS community are large enough to sustain this fragmentation. Sure they very well might survive but they won't be what they should.

Now what? There is an OpenSER book, but no OpenSER. There is Kamailio. I hope %100 of the users figure that out. I hope the publisher of the existing OpenSER book figures out how to deal with that. What happens if everyone (well, almost everyone) bails on OpenSER/Kamailio and moves to OpenSIPS? Now anyone providing support for these other OSPs (Open Source Proxies) has to decide which they will support. SER? Kamailio? OpenSIPS? All of them? None of them? That's what worries me.

I haven't decided what I'm going to do. I still use and recommend OpenSER 1.2 so none of this affects me - yet. Of course I'll be watching this closely. I will tell you one thing... I like the name OpenSIPS more than Kamailio...

Monday, July 28, 2008

OpenSER Name Change

OpenSER has changed their name to Kamailio.

Name changes are tough. This name change is especially tough and dare I even say, personally frustrating.

Working with Star2Star is tough. We have a revolutionary architecture. We operate very differently from most other providers. Heck, we operate differently than most other companies! Don't get me wrong, this is one of the many things I like about us. That's why we're changing what people think about phone (communications) companies.

One of the main drawbacks of these differences (read: advantages) is working with legacy telco. Whether it's legacy telco people, equipment, process, or organization, I have had many, many frustrating clashes with telecom. Frustrations with technology excite me. Frustrations with people drive me crazy. People can't be fixed as easily.

This name change only exacerbates what I see as an already BIG problem. How am I going to talk to veteran telecom executives about a product who's name I can't even pronounce? Even if I do learn how to pronounce it I'll never get past it's awkwardness.

I am very proud of OpenSER. OpenSER has been crucial in making Star2Star what it is. OpenSER is amazing and I have the utmost respect for everyone involved with the project.

I am ashamed of Kamailio. Same project. Same people. Same awesome, quality software. It just doesn't feel the same. Unfortunately this is my emotional reaction to this name change. I can't help how I feel and I'm sure other people (like Alex) feel the same way.

OpenSER is a phenomenal piece of software that commands respect. It's name should too.

Maybe we'll start a new "support" mailing list. Not for people needing technical support for OpenSER/Kamailio, but support for those of us who have been personally affected by this name change.

Wow, now I'm just being dramatic!

Wednesday, July 23, 2008

IPv6

I've been playing with IPv6 a bit in the last couple of days and by playing I mean:

- Setup IPv6 tunnel with Hurricane Electric
- Configured Cisco 2811 for Iv6 tunnels (both ends)
- Subnetted (is that a word?) our /48 from HE
- Configured tunnels in Linux with iproute2
- Used radvd in Linux
- Setup AAAAs for services
- Added/enabled IPv6 in AstLinux
- Played with ip6tables
- Worked on my super-secret IPv6 project (more on that later)

Yep, it's been an IPv6 week. As of right now I've got a main IPv6 tunnel from HE (in Dallas) coming into my 2811 in Tampa. I have a /48 routing down that to other tunnels providing multiple /64s to several locations. I'm setup for providing more IPv6 tunnels and /64s in the future from my 2811 (with or without connectivity to the IPv6 net at large).

I've got IPv6 in the datacenter. I've got IPv6 in the office. I've got IPv6 at home. I've got IPv6 everywhere and a TON of IP addresses to boot. It's really pretty cool and other than my funky tunnel configuration (which I actually kind of like) it's pretty easy. Once I've setup the tunnels I just route the appropriate /64s down to each PtP address for each tunnel at each location. It's a bit of a hub and spoke configuration but it works very well so far. Of course it helps when your tunnel gateway (my 2811) has seven upstream IPv4 carriers.

I've also added IPv6 to AstLinux:

- Kernel (IPv6, mobile IPv6, "41" tunnels, netfilter, etc)
- C library (uClibc)
- Busybox (apps in general, ping6, etc)
- mini_httpd
- OpenSSH
- ntpd
- stunnel
- rsync
- php
- libpcap
- tcpdump
- dnsmasq (needs testing)
- nmap
- radvd
- and more

The IPv6 kernel module alias is disabled by default. Anyone that wants to use IPv6 in AstLinux will have to enable it via (you guessed it) rc.conf. It could use some more testing (hint, hint) but so far it looks pretty good.

There was one more thing I was going to talk about... Of yeah, my "super-secret IPv6 project"... I'll have another post for that soon...

Thursday, July 17, 2008

One Week

One week - almost. That's how long it took me to realize that Maddox is right.

This time last week I was plotting how I could sneak into line to get one of those new iPhones. No it's not because I'm some Apply fanboy or some moron that can't do basic math (I'm not paying for the phone or the service). It's because my company is excited about what we might be able to do with the Apps store and the iPhone SDK. Trust me, it's really cool. That's all I can say right now.

I knew we needed at least a couple of these things (iPhones) and I didn't want to get stuck without them. The app we want to do is awesome and I wanted to get to work on it. That meant I was getting an iPhone. I had two choices:

1) Carry two phones
2) Port my number to our AT&T corporate plan and cancel T-Mobile

I wasn't going to carry two phones. A guy I work with carries at least two and it seems like a hassle. I HATE hassle. So it looks like I was going to port my number to AT&T and leave T-Mobile.

I'll skip the next part. We all know what happened here. Quick summary:

- In line at 7:45am
- Out of store by 10:30am (16GB white)
- Phone activated by 3:30pm

Let's just say I couldn't believe the activation problems. I guess Apple and AT&T were too busy marketing the hell out of this thing to remember to buy more servers or upgrade their network. Unbelievable.

Now that I've had the iPhone 3G for almost a week, here's what I've learned. Looking around the web, everyone already knows this:

- The camera sucks. Stills are grainy and often blurry. It can't do video at all. The Motorola RAZR could do video - five years ago. Today any free phone with a service plan can do video.

- Battery life sucks. For the first time in years, I actually worry about my cell phone battery.

- I don't care what anyone says, the keyboard sucks. It does ok considering there are no buttons but Apple needs to learn when they're wrong. They were wrong with the first gen, and they still are. It is not usable for any real work. I almost got to the point where I started sending incomprehensible teenage TXT: "OMG! WHR R U? LOL!".

Obvious so far. Here's where people might start to disagree with me...

What really surprises me about the iPhone is how little Apple got right. The following things also suck:

- The phone. There is a delay when picking up a new call and audio quality still sucks, regardless of method - ear buds, bluetooth, ear piece.

- No MMS for txt. Plain text SMS like it's 1999.

- Bluetooth. It's borderline useless. No profiles other than headset and A2DP (so I hear). Wow, I'm so glad my company paid so much for such a crippled device.

- The iPod. I don't HOW they screwed this one up... With just under 1,000 songs, the iPod software freezes the entire phone for a few seconds upon startup. Navigating around the various functions shows more of the same behavior - random freezes. The audio quality is horrible. HORRIBLE. Regardless of EQ settings or type of music, audio sounds weak and distorted. Don't ask me how that's possible at the same time, it just is. Any application of volume shows clipping and distortion, regardless of source material. The rotate function will drive you crazy. Listen to a song. Toss the iPhone in your pocket at any angle. When you pull it out the screen will be rotated horizontally in some kind of artist/album/song selection menu that scrolls around. Yes, it's very pretty but functionally, it's crap. Just like the rest of this device. God forbid you need to get out of this travesty and back to the somewhat functional vertically oriented interface. Good luck.

- Just like the iPod app, Contacts takes at least 3-5 seconds to start - EVERY TIME. I don't have that many contacts - just under 300 (297). I can't wait for a few seconds every time I need to interact with that database. I just can't.

- The calendar. This thing might work for you if all you have to do is set a reminder for the slumber party at your BFFs house this weekend. After all, you're a 13 old girl and that's all you have to worry about. That's obviously Apple's target market, and they NAILED it. Let's see - scroll wheels for setting dates and times are superfluous eye candy (more of the same from Apple). The alarm for events is useless. First of all, because it isn't persistent. It doesn't require you to acknowledge anything and rarely repeats. Secondly, it provides fixed options for notification before an event. I know I live 22 minutes away from a meeting. I want to set an alarm for 22 minutes. Too bad. I have an iPhone. Apple knows 13 year old girls don't need that and probably couldn't figure that interface out anyway. Setting schedules for recurring events - daily, weekly, etc. What if I want an event M-F, and another event M, W, F. How about another Saturday and Sunday. These recurring events are not supported. I have to create a new event for each day. Absolute crap. If I depended on this to make it to meetings and conference calls I'd be out of a job soon.

- The interface. I know I'm not the average user. I work (and play) in a command line environment all day and have for years. That means I'm used to having somewhat of an intimate connection with my devices. I type commands, hit enter, and it does what it is told. The man-machine communication is there. It's like driving a BMW. You really almost do feel one with the machine. I don't think I've ever felt so disconnected from a machine like I do with the iPhone. I never got the sense that I was really controlling this thing. From the keyboard to the excessive scrolling, rotating, blinking and other useless eye candy, to the lack of configurable options in everything I've come to the conclusion that I can't interface with this thing.

- Rotate. Make rotate system wide or don't support it at all. Very few apps actually support rotate. Again, I don't know what Apple was thinking here.

I've got plenty more but I'm done complaining for now. I came from a Nokia E-70. Let me tell you why this phone is awesome:

- Real keyboard. I can smoke ANYONE typing on this thing. Seriously, try me. Did I mention it also has a full phone keypad?

- Full bluetooth. Transfer files, push contacts, tether to your laptop. ANYTHING you want. It's even the little things... For instance - Symbian supports "profiles". Profiles allow you to configure just about any alert or setting on the device. You store them in profiles and can change carious settings with a single button. Think of memory seats in your car. It's basically that for phone settings. Anyway, Symbian supports automatically changing the profile based on which bluetooth device it's connected with. Basically, I used this to change the settings on my phone once it was connected to my car. I can never feel vibrate in my car for some reason. I can hear the tones I've assigned to this profile. I don't need to think about it. Get in the car, settings change on the phone. Get out, and they revert back to what they were before. Brilliant.

- PIM functions (Contacts, Calendar) have never done me wrong in over a year of ownership.

- MMS. Good MMS too.

- Awesome camera. 2MP, complete with night mode and video.

- 3G, 802.11g wifi, SIP.

- Real battery. Expandable storage.

Did I mention this phone is over two years old?

I'm serious about the iPhone being a play toy for teenagers. Why do you think it comes with Youtube? Once you're done playing Light Saber get a real phone - Blackberry or something running Symbian. That's it and that's why my iPhone SIM is in my E-70 and my iPhone is sitting on the table where I dump all of my junk mail when I walk in the door.

Seriously, if I didn't need this for my project I'd prop it up somewhere and unload some buckshot on it. Or maybe I'd give it to my friend Jason. He teaches middle school after all. Maybe he can sell it to one of his students and get this thing where it belongs.

This is the ONLY time you will find any profanity on this blog. Get an iPhone, and you'll see how justified it is:

The iPhone is a piece of shit, and so is your face.

Maddox, you are the man!

Thursday, July 10, 2008

Asterisk Cookbook

We've been trying to get the Asterisk Cookbook going. It's the same format as other O'Reilly cookbooks - practical instructions for specific problems posted as recipes, just like a cookbook you'd use in the kitchen (at least that's what people usually do with them).

I haven't contributed much but I have an excuse - I don't really work with Asterisk that much anymore. So, a couple of days ago I asked the other authors and editors if we could widen the scope of the book to include some of my favorite technologies. Anyone that reads this blog knows what that means - Cisco, OpenSER, etc. I plan on writing quite a bit about OpenSER. As I said in my OpenSER book review, OpenSER needs more press and documentation. Let's work on that!

I've now contributed three recipes! My SIP DoS/DDoS mitigation script and a couple OpenSER scripts. Some of the OpenSER stuff is a little obscure and probably won't make it to the final book. I'll be adding more OpenSER, Cisco, and SIPp stuff over the next days/weeks/months. Keep checking here and there for updates.

Oh, and if you feel like you have something to add, get a hold of one of the authors (myself, Jim, Leif, Matt, Jared, etc) and we can figure out how to get you an account on the wiki.

Book Review: Building Telephony Systems with OpenSER

I've been looking for and wondering about an OpenSER book for quite some time...

Everyone knows Asterisk is king when it comes to anything open source VoIP or telephony. OpenSER rocks but it just seems so, well, hardcore.

I can't imagine anyone walking into a Borders or Barnes and Noble, grabbing a book about OpenSER and cuddling up in one of those comfy, over-stuffed chairs on a rainy afternoon for some reading. Ok, I don't know anyone that does that anyway, and certainly not anyone with a book about OpenSER. I don't know about you but my tech books always end up getting rammed into my carry on for reading during "electronics prohibited" periods of air travel. That's all the time I care to dedicate to dead tree reads. But hey, it beats SkyMall (Jim Gaffigan: "Hey buddy, I work for SkyMall and I don't appreciate you jabbing us").

Bringing this back into focus (and on topic), is there a market for an OpenSER book? I've worked with two publishing companies in the past doing various amounts of writing and technical reviewing. Mostly O'Reilly but lately another company: Packt Publishing. Well, Packt decided there was a market for an OpenSER book and they decided to publish one. It's called Building Telephony Systems with OpenSER.

I found out about this book and knew I had to read it. What would an OpenSER book look like? What OpenSER features would be explored? What would it cost? What would it smell like? Ok, I'm just kidding about that last one but I was very interested.

I happened to be working on a couple of Packt Publishing projects at the time. I thought I'd contact one of my editors to see if they could hook me up. Turns out the book had already gone to press but they were willing to send me one. Cool!

Before I go any further I need to tell you: I won't be bought and I'm not easily influenced. Just because I've worked with Packt in the past doesn't mean I'm going to spare them any criticism or cut them any slack. If anything I'll go a little harder on them because they usually know how to get good authors and reviewers (me). ;) (Kidding, of course).

So they sent me a book. It's 295 pages and written by Flavio E. Goncalves. Flavio is a Brazilian that runs VoIP training in Brazil. Florianopolis, to be exact. Florianopolis is gorgeous. If you ever go to Brazil check it out. Hey, while you're there take an OpenSER training class from Flavio and get your company to pay for the whole thing!

Anyways, this is where my criticism starts to come in... I'm not one of these American English Nazis (does that make any sense?) that stands in line at the grocery store and freaks out when someone speaks Spanish or any other foreign language. However, I do believe that language exists as a mechanism to effectively communicate between people and sometimes that can be a little tricky even between two speakers of the same language. How many misunderstandings have you had with your friends, relatives, strangers etc while speaking the same language?

Brazilians speak Portuguese and I love it. It's one of the best sounding languages around. Being Brazilian, Flavio's first language is of course, Portuguese. It's also the language this book (Building Telephony Systems with OpenSER) was originally written in. My understanding from reading the book is that the book was developed out of Flavio's original (Portuguese) training course work and later translated to English. This makes for some very interesting and (sometimes) difficult to understand English language usage. There are several instances in the book that are difficult to follow because the language is, well, awkward. It looks like there was a proofreader... Maybe his style is different from mine but I'm pretty sure most people agree with me.

Part of me feels like this criticism is unfair and I wrestled with even mentioning it. Here's my chance to elaborate on this and explain myself.

Neither the author, editor, nor reviewers are native English language speakers yet this is what the book was written in. That's amazing to me. These people, in addition to being remarkably talented in their respective fields (publishing, technology) can all speak multiple languages and work on a team, around the world (India, Romania, Brazil) in a second language. The world is flat, my friend.

I feel like I can barely work and speak English half the time. Most Americans are the same way. We're all lazy, fat, SUV driving slobs that grunt back and forth in something that resembles English (which is already pretty "ugly"). Ok it's not that bad but that's what it's often made out to be... Anyone ever seen Idiocracy?

So for a technical book, my main criticism is English language usage and that's not even that bad. So far they're doing pretty well! On to the technical stuff...

I thought the book did a great job explaining the use of various OpenSER modules and OpenSER scripting in general. I also like how Flavio introduced readers to the various (relevant) SIP RFCs along the way. In working with OpenSER and SIP in general I can't stress how important this is. Many SIP platforms isolate you from the RFCs as much as possible. Everything is taken care of and it magically works (even with Asterisk when compared to OpenSER). With OpenSER this is not the case. You better know your RFCs. You better know the difference between components in a SIP network (Proxy, UAC, UAS, B2BUA, etc). You better know some core aspects of RFC3261 (stateful vs. stateless, strict routing, loose routing, methods, dialogs, domains, etc). Flavio isn't afraid to present this to the reader and he does it in a way that doesn't come across as snobby. Depending on the situation I've been known to quote RFCs and sections before. I probably (almost always) end up looking like a jerk but sometimes, hey, it needs to be done!

While reading I was disappointed to see so much focus on MediaProxy. I don't like MediaProxy. It's written in Python. Yuck. Enough said. I kept reading and by the end of the book Flavio admitted the mistake of more or less ignoring RTPProxy and agreed it was mostly superior. Bravo! No one knows everything, not even the guys that write the books. It was refreshing to see this in print. Once again, well done.

A couple of specific (nit picky) problems... The Asterisk config for a PSTN gateway on pg. 147 is suspect. The type doesn't make it clear which file you are working in and I doubt anyone would really use that SIP configuration in production. There's also no Zaptel config. I know this isn't an Asterisk book (there are plenty of those, the best one is free) but there should at least be a pointer to an online resource for Asterisk/Zaptel configuration or something.

The same thing goes for the Cisco config. If you look closely you'll see the dial-peer uses sip-notify for dtmf-relay. No one uses this and we certainly haven't configured OpenSER for it. Again, it's nit picky but some user could end up pulling all of his hair out trying to get this to work.

I didn't go over the OpenSER script bracket by bracket but it looked pretty good. That's even tougher to comment on because it's practically a programming language and no programmer ever completely loves the way someone else wrote something. That's just how it goes.

Overall, this is a very good book and it deserves to do well, for Flavio, everyone at Packt, OpenSER and the community in general. People who should be running OpenSER aren't. Hopefully this book can help change that!

Thank you to Flavio, the OpenSER team, and everyone at Packt Publishing!

Monday, July 7, 2008

An update on SIP DoS mitigation

After spending a couple of hours on the VoIP User's Conference last week I thought I'd keep working on my SIP DoS/DDoS script a bit and get it to the point where I'd like to run it on some of my systems (if only to collect statistics).

The new version includes several new features, the most exciting (and certainly controversial) are changes to the string pattern matching for SIP requests. I now block ALL tel: URIs by default. I don't like tel: URIs. I think they're anti-SIP and you shouldn't use them. Now my script won't let you (unless you disable it, of course).

Anyways, I had to do this because (as I've already mentioned), I changed the way pattern matching runs on SIP requests. Two big changes:

1) Support a (configurable) offset for searches into the packet.
2) Update the SIP method matches to match "$METHOD sip:"

First of all, we now (by default) only search the first 65 bytes of the packet. That should be more than enough to search the first line for the SIP method and URI. Speaking of URI, I now match the URI along with the method to prevent false matches. Before we were only matching on the method and it was causing some false positives because of things like the Allow: header (where all of the supported methods are listed).

We'll see how this goes.

One thing I wanted to bring up in the VoIP User's call last week but failed to do so is the possible use of OpenSER to protect Asterisk (and other systems) from attack. In addition to supporting cool things like SIP message length filtering (msg:len) you can also use the pike module for some basic (but slightly more intelligent) SIP rate limiting. Of course then you need to support an OpenSER config, which a lot of people don't want to do...

What else is new in the script? Basic support for udp and/or tcp, configurable bursting, fixes to the FORWARD support and more. Check it out, it's free!

Wednesday, July 2, 2008

SIP DoS/DDoS Mitigation

An interesting thread came up on Asterisk-Users the other day...

Someone had problems with some miscreant attempting (and apparently succeeding) to bruteforce a SIP account on his Asterisk system.

That's a problem. Asterisk currently has no means to protect against any type of SIP flooding/bruteforce/DoS (other than running out of resources/crashing). That's ok because some people (like myself) would argue that these types of attacks are best handled via other means...

Like the kernel, or preferably the kernel of a completely seperate box.

It got me to thinking - maybe I could whip up a script using some of the cool stuff in iptables/netfilter to mitigate these SIP DoS attacks in the kernel. I don't have a ton of time to elaborate on the script but for now here it is:

http://admin.star2star.com/sipdos

It's not great. It's not very SIP-specific. Who knows how accurate/effective/resource intensive it is. It hasn't been tested (much). For all I know, it will just make things worse. However, I think it is a step in the right direction and hopefully one of many tools that can be used to protect Asterisk and other SIP/VoIP systems.

By the way, I call it "SIP DoS Mitigation" because with any large enough DDoS attack, you are toast. ..

Speaking of other tools... We're all going to celebrate America's birthday this Friday by getting together on a conference call/IRC to talk over some of these issues with the VoIP User's Conference. Hear you there?

Monday, March 24, 2008

Q.931 Problems - Solved, sort of

It's been a long day so I'll keep this one short... Upgrading to Asterisk (libpri|zaptel) 1.4 solved my Cisco AS5350XM problem. However, now I've got a new one.

Zaptel 1.4 required a new version of Wanpipe. The new version of Wanpipe does not seem to work with mixed Ts. My T1 happens to have 12 DS0s for PRI and 12 for PPP.

I'm currently working with Sangoma to get this resolved so I can have it all!

Friday, March 21, 2008

Why I (and presumably other people) hate telecom

Q.931.

That's it. Q-freaking-9-3-1. I suppose it's not bad considering it was developed so long ago, but really for me this protocol (and all it stands for) is why so many people (myself included) get frustrated with telecom.

I spent two hours today trying to get a call up (over PRI) from an Asterisk system to a Cisco gateway. I covered all of the basics:

Asterisk - pri_net - check
Cisco - pri_cpe - check
Asterisk - master clocking - check
T1 params - B8ZS, ESF - check
Switchtype - national/NI2 - check
D chan - 24 - check
T1 crossover cable - check

Voila! D channel up in no time (seriously, five minutes). Try to send a call - SURE, everything looks good (including caller id). The call gets setup and once there is about 1 sec of audio it gets torn down. Hmmm... Granted I am running SIP on the other side of the Cisco gateway (AS5350XM) so I start there. Let's face, if you're reading this blog you know I deal with SIP quite a bit. I'd much rather look at it than the alternative - Q.931...

Sure enough everything looks good on the SIP side. Perfect, actually. Ok fine, I guess we're going to have to go Q.931. I enable Q.931 debugging:

Asterisk:
asterisk -r
pri debug span 3

Cisco:
debug isdn q931
term mon (to log to my SSH session)

Try the call again. Same thing - one second of audio, disconnect. Here is what I get:

Cisco debug:

Mar 21 23:13:45.307: ISDN Se3/0:23 Q931: Applying typeplan for sw-type 0xD is 0x2 0x1, Calling num 5043223199
Mar 21 23:13:45.311: ISDN Se3/0:23 Q931: Applying typeplan for sw-type 0xD is 0x2 0x1, Called num 19412340001
Mar 21 23:13:45.311: ISDN Se3/0:23 Q931: TX -> SETUP pd = 8 callref = 0x00A3
Bearer Capability i = 0x8090A2
Standard = CCITT
Transfer Capability = Speech
Transfer Mode = Circuit
Transfer Rate = 64 kbit/s
Channel ID i = 0xA18381
Preferred, Channel 1
Display i = 'Pcom2'
Calling Party Number i = 0x2180, '5043223199'
Plan:ISDN, Type:National
Called Party Number i = 0xA1, '19412340001'
Plan:ISDN, Type:National
Mar 21 23:13:45.323: ISDN Se3/0:23 Q931: RX <- CALL_PROC pd = 8 callref = 0x80A3
Channel ID i = 0xA98381
Exclusive, Channel 1
Mar 21 23:13:46.371: ISDN Se3/0:23 Q931: RX <- CONNECT pd = 8 callref = 0x80A3
Channel ID i = 0xA98381
Exclusive, Channel 1
Progress Ind i = 0x8182 - Destination address is non-ISDN
Mar 21 23:13:46.371: ISDN Se3/0:23 Q931: TX -> CONNECT_ACK pd = 8 callref = 0x00A3
Mar 21 23:13:46.379: ISDN Se3/0:23 Q931: RX <- STATUS pd = 8 callref = 0x80A3
Cause i = 0x80E2 - Message not compatible with call state or not implemented
Call State i = 0x0A
Mar 21 23:13:46.379: ISDN Se3/0:23 Q931: TX -> RELEASE pd = 8 callref = 0x00A3
Cause i = 0x80E408 - Invalid information element contents
Mar 21 23:13:46.403: ISDN Se3/0:23 Q931: RX <- RELEASE_COMP pd = 8 callref = 0x80A3
Cause i = 0x8190 - Normal call clearing

Asterisk debug:
< Protocol Discriminator: Q.931 (8) len=50
< Call Ref: len= 2 (reference 164/0xA4) (Originator)
< Message type: SETUP (5)
< [04 03 80 90 a2]
< Bearer Capability (len= 5) [ Ext: 1 Q.931 Std: 0 Info transfer capability: Speech (0)
< Ext: 1 Trans mode/rate: 64kbps, circuit-mode (16)
< Ext: 1 User information layer 1: u-Law (34)
< [18 03 a1 83 81]
< Channel ID (len= 5) [ Ext: 1 IntID: Implicit, PRI Spare: 0, Preferred Dchan: 0
< ChanSel: Reserved
< Ext: 1 Coding: 0 Number Specified Channel Type: 3
< Ext: 1 Channel: 1 ]
< [28 05 50 63 6f 6d 32]
< Display (len= 5) [ Pcom2 ]
< [6c 0c 21 80 35 30 34 33 32 32 33 31 39 39]
< Calling Number (len=14) [ Ext: 0 TON: National Number (2) NPI: ISDN/Telephony Numbering Plan (E.164/E.163) (1)
< Presentation: Presentation permitted, user number not screened (0) '5043223199' ]
< [70 0c a1 31 39 34 31 32 33 34 30 30 30 31]
< Called Number (len=14) [ Ext: 1 TON: National Number (2) NPI: ISDN/Telephony Numbering Plan (E.164/E.163) (1) '19412340001' ]
-- Making new call for cr 164
-- Processing Q.931 Call Setup
-- Processing IE 4 (cs0, Bearer Capability)
-- Processing IE 24 (cs0, Channel Identification)
-- Processing IE 40 (cs0, Display)
-- Processing IE 108 (cs0, Calling Party Number)
-- Processing IE 112 (cs0, Called Party Number)
> Protocol Discriminator: Q.931 (8) len=10
> Call Ref: len= 2 (reference 164/0xA4) (Terminator)
> Message type: CALL PROCEEDING (2)
> [18 03 a9 83 81]
> Channel ID (len= 5) [ Ext: 1 IntID: Implicit, PRI Spare: 0, Exclusive Dchan: 0
> ChanSel: Reserved
> Ext: 1 Coding: 0 Number Specified Channel Type: 3
> Ext: 1 Channel: 1 ]
-- Accepting call from '5043223199' to '19412340001' on channel 0/1, span 3
-- Executing Wait("Zap/49-1", "1") in new stack
-- Executing Answer("Zap/49-1", "") in new stack
> Protocol Discriminator: Q.931 (8) len=14
> Call Ref: len= 2 (reference 164/0xA4) (Terminator)
> Message type: CONNECT (7)
> [18 03 a9 83 81]
> Channel ID (len= 5) [ Ext: 1 IntID: Implicit, PRI Spare: 0, Exclusive Dchan: 0
> ChanSel: Reserved
> Ext: 1 Coding: 0 Number Specified Channel Type: 3
> Ext: 1 Channel: 1 ]
> [1e 02 81 82]
> Progress Indicator (len= 4) [ Ext: 1 Coding: CCITT (ITU) standard (0) 0: 0 Location: Private network serving the local user (1)
> Ext: 1 Progress Description: Called equipment is non-ISDN. (2) ]
-- Executing MusicOnHold("Zap/49-1", "") in new stack
-- Started music on hold, class 'default', on channel 'Zap/49-1'
< Protocol Discriminator: Q.931 (8) len=5
< Call Ref: len= 2 (reference 164/0xA4) (Originator)
< Message type: CONNECT ACKNOWLEDGE (15)
> Protocol Discriminator: Q.931 (8) len=12
> Call Ref: len= 2 (reference 164/0xA4) (Terminator)
> Message type: STATUS (125)
> [08 02 80 e2]
> Cause (len= 4) [ Ext: 1 Coding: CCITT (ITU) standard (0) 0: 0 Location: User (0)
> Ext: 1 Cause: Wrong message (98), class = Protocol Error (6) ]
> [14 01 0a]
> Call State (len= 3) [ Ext: 0 Coding: CCITT (ITU) standard (0) Call state: Active (10)
< Protocol Discriminator: Q.931 (8) len=10
< Call Ref: len= 2 (reference 164/0xA4) (Originator)
< Message type: RELEASE (77)
< [08 03 80 e4 08]
< Cause (len= 5) [ Ext: 1 Coding: CCITT (ITU) standard (0) 0: 0 Location: User (0)
< Ext: 1 Cause: Invalid information element contents (100), class = Protocol Error (6) ]
< Cause data 1: 08 (8)
-- Processing IE 8 (cs0, Cause)
-- Channel 0/1, span 3 got hangup
-- Stopped music on hold on Zap/49-1
== Spawn extension (pri-in, 19412340001, 3) exited non-zero on 'Zap/49-1'
NEW_HANGUP DEBUG: Calling q931_hangup, ourstate Null, peerstate Release Request
> Protocol Discriminator: Q.931 (8) len=9
> Call Ref: len= 2 (reference 164/0xA4) (Terminator)
> Message type: RELEASE COMPLETE (90)
> [08 02 81 90]
> Cause (len= 4) [ Ext: 1 Coding: CCITT (ITU) standard (0) 0: 0 Location: Private network serving the local user (1)
> Ext: 1 Cause: Normal Clearing (16), class = Normal Event (1) ]
NEW_HANGUP DEBUG: Calling q931_hangup, ourstate Null, peerstate Null
NEW_HANGUP DEBUG: Destroying the call, ourstate Null, peerstate Null
-- Hungup 'Zap/49-1'
s2s-srq-co*CLI>

Everytime I have to look at Q931 I just cringe. I MEAN CRINGE. I hate it. Look at it - all of these goofy messages, number plans, and the worst - IEs (Information Elements). Compare this to SIP (there are plenty of examples on this blog). I guess I just take it for granted. I can fire up ngrep on a network interface, turn on interpretation of carriage returns/linefeeds and go to town with something that just makes more sense. Sure to some people it probably still looks like gibberish but anyone that complains certainly hasn't seen much Q.931!

Let's look at my current problem... It appears that once the call is setup, Asterisk sends a CONNECT message, to which the Cisco quickly acknowledges with a CONNECT ACKNOWLEDGMENT (big surprise there). Here's where things get a little strange... The next message is from Asterisk (STATUS) complaining with "Message not compatible with call state or not implemented". That is pretty helpful, I'll give them that. However, what's not compatible about a standard, simple CONNECT ACK?!?!

This is wear I start to get angry. I'd like to see what's going on a little bit more. I know I have some flexibility with my IEs, for example. If I turn up the debug high enough in Asterisk I can see which IEs Asterisk identifies. I just can't see the data. From a debug standpoint, Cisco appears to give me even less. Cisco does, however, give me some pretty decent control of newer and less than standard IEs to send (and to a lesser extent, receive). That's gotta have something to do with it, I'm sure. It's even telling me "invalid information element contents". Too bad I can't actually see the IE content... Being severely limited with the tools at hand, I began to cycle through all of my IE, number plan, etc options on both sides. I even got lazy at one point and tried to set my switchtype to dms100! No dice.

Want to know the craziest part about all of this? I've done this several times before. I've brought up many a PRI to carriers, with all kinds of switchtypes, b/d channel configs, into all kinds of equipment (including Asterisk and Cisco). I've gone from Asterisk PRI -> Cisco PRI, Cisco PRI -> Cisco PRI, Asterisk PRI -> Asterisk PRI and every other combination you could imagine (although I think I covered all of them). I've never really had problems, although whenever there is a problem it means going through lines and lines of less-than-helpful Q.931 messaging to identify the problem, which gets me back to my point.

That's just the problem with telecom. It's old, slow, and inflexible. If this were a problem with SIP I could grab the packet stream with tcpdump and load it into Wireshark if I got really desperate (I'm rarely that desperate). I could watch it in real time with ngrep, complete with regex matching on payload and BPF syntax. I could try multiple SIP libraries and multiple clients. I could even tap into the WEALTH of information about SIP, including the various RFCs. Sure I know there are a lot of them but hey, at least you know where to look. I could even try different hardware very easily because hey, you don't need a $500 T1 card to play with SIP. Heck you don't even need a network. VmWare or even good ol' fashioned lo0 work just fine.

How can I do this with ISDN? Buy a T-Berd? No thanks. So here it is, Friday night, and I'm obsessing over the PRI that kicked my ass today. Anyone have any ideas?

Wednesday, March 19, 2008

More OpenSER Work

I've been working with OpenSER a lot over the last couple of weeks. I upgraded to 1.2.3, enabled TLS support, and implemented some really cool new features:

- Serial forking class
- English error messages
- Numerous code cleanups
- Much more

I'll try to find out how much of this I can share and get back to you. In the meantime I'm having a problem calling next_gw() from the LCR module. I suspect it's a db problem but I can't be sure yet...

Friday, February 22, 2008

Obscure RFC 3398 Support with OpenSER

I hate SS7. Let me say that again. I HATE SS7. I mean it's cool and everything, but there are parts of it that are unnecessarily confusing...

For instance, this week I got very confused over use of the term "cic" in telco-speak. I had thought cic was "circuit identification code". So, when one of our partners asked me to support embedding the cic in the incoming SIP INVITE, I freaked out. Why is that my responsibility? Why should my proxy ("class 5 switch") have to maintain state of individual DS0s on YOUR media gateways. This sucks! I bitterly started reading RFC 3398 (particularly section 7.2.1.1) and gave up halfway through it, becoming frustrated with the overall language of the section. After all, keeping track of DS0s just isn't my thing, and that's what CICs are about, right?

Turns out I was wrong. Keeping tracks of CICs is my thing. The problem is, these CICs are "Carrier Identification Codes", not "Circuit Identification Codes". I know everyone in networking and telecom loves to have an acronym for everything, but for GODs SAKE PEOPLE don't use the same acronym in the same context. Reading on in RFC 3398, they specifically warn for this confusion:

..
 If the 'cic=' parameter is present in the Request-URI, the
gateway SHOULD consult local policy to make sure that it is
appropriate to transmit this Carrier Identification Code (CIC, not to

be confused with the MTP3 'circuit identification code') in the IAM;

..

Good point. Why didn't you say something sooner? So, now it turns out all I have to do is append a cic= (CARRIER identification code) parameter to the SIP RURI (request uniform resource identifier) on the outbound INVITE to the SIP <-> SS7 gateway. No problem. Thanks to OpenSER, module textops, and subst_uri, this is all I need to do:

avp_printf("$avp(s:cic)", "+15062"); # Set the cic value in an AVP
if (subst_uri('/^sip:([0-9]+)@(.*)$/sip:\1;cic=$avp(s:cic)@\2/i')) {
xlog("Added CIC $avp(s:cic) to RURI"); # tell us about it
};

Here is the INVITE going out to the gateway (cic in red):

U a.b.c.d:5060 -> w.x.y.z:5060
INVITE sip:14145551212;cic=+15062@w.x.y.z:5060 SIP/2.0.
Record-Route: .
Via: SIP/2.0/UDP a.b.c.d;branch=z9hG4bKf1e2.8a4d41b6.0.
Via: SIP/2.0/UDP e.f.g.h:5060;branch=z9hG4bK4e28b419;rport=5060.
From: "Polycom 320" ;tag=as477acc5f.
To: .
Contact: .
Call-ID: 343b661709e271c30bdfafe923a82adb@e.f.g.h.
CSeq: 103 INVITE.
User-Agent: Star2Star StarBox astlinux-s2s-1438-net4801.
Max-Forwards: 69.
Remote-Party-ID: "Polycom 320" ;privacy=off;screen=no.
Date: Fri, 22 Feb 2008 19:48:44 GMT.
Allow: INVITE, ACK, CANCEL, OPTIONS, BYE, REFER, SUBSCRIBE, NOTIFY.
Content-Type: application/sdp.
Content-Length: 265.
.
v=0.
o=root 6183 6184 IN IP4 e.f.g.h.
s=session.
c=IN IP4 e.f.g.h.
t=0 0.
m=audio 19770 RTP/AVP 18 0 101.
a=rtpmap:18 G729/8000.
a=fmtp:18 annexb=no.
a=rtpmap:0 PCMU/8000.
a=rtpmap:101 telephone-event/8000.
a=fmtp:101 0-16.
a=silenceSupp:off - - - -.

Yeah, yeah, you like that?!?

I'm still not sure of the actual CIC format... That isn't EXACTLY what is shown in section 8.2.1.1 of RFC 3398 but it does look like what Cisco expects from their docs I've read (from the BTS 10200). After all, this particular SIP <-> SS7 gateway is made by them (PGW 2200).

Obviously you can get fancy with that and do all sorts of conditionals and DB calls to get the CIC, only append it on certain calls, different CICs for different sources, destinations, etc but the point is you can tweak up the RURI and insert it. I still haven't tested this against Cisco but I'm pretty sure it will work...

After I sorted out my "CIC confusion" it took me less than 30 minutes to implement this. Thanks OpenSER. You make SS7 a lot easier to deal with (as long as SIP is involved)!

Wednesday, February 20, 2008

Missing SIP traffic?

SIP can be cool because it resembles HTTP. OpenSER and SER are cool because they are so powerful. For instance, in OpenSER you don't need to authenticate calls. Or you can specify which request URIs should be challenged with an auth (or which method types, like REGISTER). This is usually done like so:

if (is_method("INVITE")) {

if (uri=~"^sip:1000@") {
rewritehostport("192.168.1.1:5060"); #set destination
route(1); #t_relay, etc is in route(1)
}

if (!allow_trusted()) {
if (!proxy_authorize("star2star.com","subscriber")) {
proxy_challenge("star2star.com","0");
return;
} else if (!check_from()) {
sl_send_reply("403", "Use From=ID");
return;
};

xlog("Creds are good\n");
consume_credentials();

};

if (uri=~"^sip:1[0-9]{10}@") {
route(5); #goto LCR
return;
};
};

So, in this example any SIP endpoint that can reach this proxy can hit RURI:1000 and be forwarded to the voicemail server with no authentication. As we step through this example, the only other URIs that match are after the allow_trusted or proxy_authorize checks. Basically, your source IP address has to be in the trusted table or you have to successfully respond to a 407 Proxy Authentication Required from the proxy.

I've seen this work perfectly between an Asterisk client and OpenSER hundreds of times. Most of the time it works. MOST OF THE TIME. I've noticed a scenario where it does not work, and I am struggling to figure out why...

Here's the architecture:

User's Phone -> Asterisk --(internet)--> OpenSER -> Misc. other systems

Like I said, normally this works and it looks like this (get ready for some SIP):

#
U a.b.c.d:5060 -> e.f.g.h:5060
INVITE sip:9415551212@e.f.g.h SIP/2.0.
Via: SIP/2.0/UDP a.b.c.d:5060;branch=z9hG4bK2e8b0432;rport.
From: "User Phone" ;tag=as3474f6c4.
To: .
Contact: .
Call-ID: 2ad4212700f61c3e1294439f1c72c479@a.b.c.d.
CSeq: 102 INVITE.
User-Agent: Asterisk PBX.
Max-Forwards: 70.
Remote-Party-ID: "User Phone" ;privacy=off;screen=no.
Date: Tue, 05 Feb 2008 18:24:11 GMT.
Allow: INVITE, ACK, CANCEL, OPTIONS, BYE, REFER, SUBSCRIBE, NOTIFY.
X-s2s-region: 1.
Content-Type: application/sdp.
Content-Length: 216.
.
v=0.
o=root 1429 1429 IN IP4 a.b.c.d.
s=session.
c=IN IP4 a.b.c.d.
t=0 0.
m=audio 19424 RTP/AVP 0 101.
a=rtpmap:0 PCMU/8000.
a=rtpmap:101 telephone-event/8000.
a=fmtp:101 0-16.
a=silenceSupp:off - - - -.

#
U e.f.g.h:5060 -> a.b.c.d:5060
SIP/2.0 100 Trying.
Via: SIP/2.0/UDP a.b.c.d:5060;branch=z9hG4bK2e8b0432;rport=5060.
From: "User Phone" ;tag=as3474f6c4.
To: .
Call-ID: 2ad4212700f61c3e1294439f1c72c479@a.b.c.d.
CSeq: 102 INVITE.
Server: OpenSer (1.1.0-notls (i386/linux)).
Content-Length: 0.
Warning: 392 e.f.g.h:5060 "Noisy feedback tells: pid=4363 req_src_ip=a.b.c.d req_src_port=5060 in_uri=sip:9415551212@e.f.g.h out_uri=sip:9415551212@e.f.g.h via_cnt==1".
.

#
U e.f.g.h:5060 -> a.b.c.d:5060
SIP/2.0 407 Proxy Authentication Required.
Via: SIP/2.0/UDP a.b.c.d:5060;branch=z9hG4bK2e8b0432;rport=5060.
From: "User Phone" ;tag=as3474f6c4.
To: ;tag=0dd4490c85a9eb8d48ff967a8700cef0.fcb4.
Call-ID: 2ad4212700f61c3e1294439f1c72c479@a.b.c.d.
CSeq: 102 INVITE.
Proxy-Authenticate: Digest realm="star2star.com", nonce="valid_nonce".
Server: OpenSer (1.1.0-notls (i386/linux)).
Content-Length: 0.
Warning: 392 e.f.g.h:5060 "Noisy feedback tells: pid=4363 req_src_ip=a.b.c.d req_src_port=5060 in_uri=sip:9415551212@e.f.g.h out_uri=sip:9415551212@e.f.g.h via_cnt==1".
.

#
U a.b.c.d:5060 -> e.f.g.h:5060
ACK sip:9415551212@e.f.g.h SIP/2.0.
Via: SIP/2.0/UDP a.b.c.d:5060;branch=z9hG4bK2e8b0432;rport.
From: "User Phone" ;tag=as3474f6c4.
To: ;tag=0dd4490c85a9eb8d48ff967a8700cef0.fcb4.
Contact: .
Call-ID: 2ad4212700f61c3e1294439f1c72c479@a.b.c.d.
CSeq: 102 ACK.
User-Agent: Asterisk PBX.
Max-Forwards: 70.
Remote-Party-ID: "User Phone" ;privacy=off;screen=no.
Content-Length: 0.
.

#
U a.b.c.d:5060 -> e.f.g.h:5060
INVITE sip:9415551212@e.f.g.h SIP/2.0.
Via: SIP/2.0/UDP a.b.c.d:5060;branch=z9hG4bK57af1f44;rport.
From: "User Phone" ;tag=as3474f6c4.
To: .
Contact: .
Call-ID: 2ad4212700f61c3e1294439f1c72c479@a.b.c.d.
CSeq: 103 INVITE.
User-Agent: Asterisk PBX.
Max-Forwards: 70.
Remote-Party-ID: "User Phone" ;privacy=off;screen=no.
Proxy-Authorization: Digest username="cpesource", realm="star2star.com", algorithm=MD5, uri="sip:9415551212@e.f.g.h", nonce="valid_nonce", response="valid_response", opaque="".
Date: Tue, 05 Feb 2008 18:24:12 GMT.
Allow: INVITE, ACK, CANCEL, OPTIONS, BYE, REFER, SUBSCRIBE, NOTIFY.
X-s2s-region: 1.
Content-Type: application/sdp.
Content-Length: 216.
.
v=0.
o=root 1429 1430 IN IP4 a.b.c.d.
s=session.
c=IN IP4 a.b.c.d.
t=0 0.
m=audio 19424 RTP/AVP 0 101.
a=rtpmap:0 PCMU/8000.
a=rtpmap:101 telephone-event/8000.
a=fmtp:101 0-16.
a=silenceSupp:off - - - -.

#
U e.f.g.h:5060 -> a.b.c.d:5060
SIP/2.0 100 Trying.
Via: SIP/2.0/UDP a.b.c.d:5060;branch=z9hG4bK57af1f44;rport=5060.
From: "User Phone" ;tag=as3474f6c4.
To: .
Call-ID: 2ad4212700f61c3e1294439f1c72c479@a.b.c.d.
CSeq: 103 INVITE.
Server: OpenSer (1.1.0-notls (i386/linux)).
Content-Length: 0.
Warning: 392 e.f.g.h:5060 "Noisy feedback tells: pid=4361 req_src_ip=a.b.c.d req_src_port=5060 in_uri=sip:9415551212@e.f.g.h out_uri=sip:9415551212@e.f.g.h via_cnt==1".
.

#
U e.f.g.h:5060 -> a.b.c.d:5060
SIP/2.0 100 trying -- your call is important to us.
Via: SIP/2.0/UDP a.b.c.d:5060;branch=z9hG4bK57af1f44;rport=5060.
From: "User Phone" ;tag=as3474f6c4.
To: .
Call-ID: 2ad4212700f61c3e1294439f1c72c479@a.b.c.d.
CSeq: 103 INVITE.
Server: OpenSer (1.1.0-notls (i386/linux)).
Content-Length: 0.
Warning: 392 e.f.g.h:5060 "Noisy feedback tells: pid=4361 req_src_ip=a.b.c.d req_src_port=5060 in_uri=sip:9415551212@e.f.g.h out_uri=sip:18135551212@w.x.y.z:5060;transport=udp via_cnt==1".
.

#
U e.f.g.h:5060 -> a.b.c.d:5060
SIP/2.0 183 Session Progress.
Via: SIP/2.0/UDP a.b.c.d:5060;branch=z9hG4bK57af1f44;rport=5060.
From: "User Phone" ;tag=as3474f6c4.
To: ;tag=F6697AC-1DCD.
Date: Tue, 05 Feb 2008 18:24:12 GMT.
Call-ID: 2ad4212700f61c3e1294439f1c72c479@a.b.c.d.
Server: Cisco-SIPGateway/IOS-12.x.
CSeq: 103 INVITE.
Allow: INVITE, OPTIONS, BYE, CANCEL, ACK, PRACK, COMET, REFER, SUBSCRIBE, NOTIFY, INFO, UPDATE, REGISTER.
Allow-Events: telephone-event.
Contact: .
Record-Route: .
Content-Disposition: session;handling=required.
Content-Type: application/sdp.
Content-Length: 238.
.
v=0.
o=CiscoSystemsSIP-GW-UserAgent 2171 1394 IN IP4 w.x.y.z.
s=SIP Call.
c=IN IP4 w.x.y.z.
t=0 0.
m=audio 18722 RTP/AVP 0 101.
c=IN IP4 w.x.y.z.
a=rtpmap:0 PCMU/8000.
a=rtpmap:101 telephone-event/8000.
a=fmtp:101 0-16.

#
U e.f.g.h:5060 -> a.b.c.d:5060
SIP/2.0 200 OK.
Via: SIP/2.0/UDP a.b.c.d:5060;branch=z9hG4bK57af1f44;rport=5060.
From: "User Phone" ;tag=as3474f6c4.
To: ;tag=F6697AC-1DCD.
Date: Tue, 05 Feb 2008 18:24:12 GMT.
Call-ID: 2ad4212700f61c3e1294439f1c72c479@a.b.c.d.
Server: Cisco-SIPGateway/IOS-12.x.
CSeq: 103 INVITE.
Allow: INVITE, OPTIONS, BYE, CANCEL, ACK, PRACK, COMET, REFER, SUBSCRIBE, NOTIFY, INFO, UPDATE, REGISTER.
Allow-Events: telephone-event.
Contact: .
Record-Route: .
Content-Type: application/sdp.
Content-Length: 238.
.
v=0.
o=CiscoSystemsSIP-GW-UserAgent 2171 1394 IN IP4 w.x.y.z.
s=SIP Call.
c=IN IP4 w.x.y.z.
t=0 0.
m=audio 18722 RTP/AVP 0 101.
c=IN IP4 w.x.y.z.
a=rtpmap:0 PCMU/8000.
a=rtpmap:101 telephone-event/8000.
a=fmtp:101 0-16.

#
U a.b.c.d:5060 -> e.f.g.h:5060
ACK sip:18135551212@w.x.y.z:5060 SIP/2.0.
Via: SIP/2.0/UDP a.b.c.d:5060;branch=z9hG4bK38cd41fe;rport.
Route: .
From: "User Phone" ;tag=as3474f6c4.
To: ;tag=F6697AC-1DCD.
Contact: .
Call-ID: 2ad4212700f61c3e1294439f1c72c479@a.b.c.d.
CSeq: 103 ACK.
User-Agent: Asterisk PBX.
Max-Forwards: 70.
Remote-Party-ID: "User Phone" ;privacy=off;screen=no.
Content-Length: 0.
.

#
U a.b.c.d:5060 -> e.f.g.h:5060
BYE sip:18135551212@w.x.y.z:5060 SIP/2.0.
Via: SIP/2.0/UDP a.b.c.d:5060;branch=z9hG4bK48cf5abc;rport.
Route: .
From: "User Phone" ;tag=as3474f6c4.
To: ;tag=F6697AC-1DCD.
Call-ID: 2ad4212700f61c3e1294439f1c72c479@a.b.c.d.
CSeq: 104 BYE.
User-Agent: Asterisk PBX.
Max-Forwards: 70.
Remote-Party-ID: "User Phone" ;privacy=off;screen=no.
Proxy-Authorization: Digest username="cpesource", realm="star2star.com", algorithm=MD5, uri="sip:18135551212@w.x.y.z:5060", nonce="valid_nonce", response="valid_response", opaque="".
Content-Length: 0.
.

#
U e.f.g.h:5060 -> a.b.c.d:5060
SIP/2.0 200 OK.
Via: SIP/2.0/UDP a.b.c.d:5060;branch=z9hG4bK48cf5abc;rport=5060.
From: "User Phone" ;tag=as3474f6c4.
To: ;tag=F6697AC-1DCD.
Date: Tue, 05 Feb 2008 18:24:21 GMT.
Call-ID: 2ad4212700f61c3e1294439f1c72c479@a.b.c.d.
Server: Cisco-SIPGateway/IOS-12.x.
Content-Length: 0.
CSeq: 104 BYE.
.

INVITE comes in, 407 goes out and gets ACKd by remote Asterisk instance, INVITE comes back, this time with a Proxy-Authorization: header attached. The call gets forwarded, setup, and everything is fine.

HOWEVER - some times the 407 doesn't make it back to Asterisk (for whatever reason) and the call dies:

#
U a.b.c.d:5060 -> e.f.g.h:5060
INVITE sip:8135551212@e.f.g.h SIP/2.0.
Via: SIP/2.0/UDP a.b.c.d:5060;branch=z9hG4bK3beb8c07;rport.
From: "User Phone" ;tag=as664fbdbc.
To: .
Contact: .
Call-ID: 165e2edb63fd81b30e629055245b8b28@a.b.c.d.
CSeq: 102 INVITE.
User-Agent: Asterisk PBX.
Max-Forwards: 70.
Remote-Party-ID: "User Phone" ;privacy=off;screen=no.
Date: Tue, 05 Feb 2008 18:22:47 GMT.
Allow: INVITE, ACK, CANCEL, OPTIONS, BYE, REFER, SUBSCRIBE, NOTIFY.
X-s2s-region: 1.
Content-Type: application/sdp.
Content-Length: 216.
.
v=0.
o=root 1306 1306 IN IP4 a.b.c.d.
s=session.
c=IN IP4 a.b.c.d.
t=0 0.
m=audio 19154 RTP/AVP 0 101.
a=rtpmap:0 PCMU/8000.
a=rtpmap:101 telephone-event/8000.
a=fmtp:101 0-16.
a=silenceSupp:off - - - -.

#
U e.f.g.h:5060 -> a.b.c.d:5060
SIP/2.0 100 Trying.
Via: SIP/2.0/UDP a.b.c.d:5060;branch=z9hG4bK3beb8c07;rport=5060.
From: "User Phone" ;tag=as664fbdbc.
To: .
Call-ID: 165e2edb63fd81b30e629055245b8b28@a.b.c.d.
CSeq: 102 INVITE.
Server: OpenSer (1.1.0-notls (i386/linux)).
Content-Length: 0.
Warning: 392 e.f.g.h:5060 "Noisy feedback tells: pid=4363 req_src_ip=a.b.c.d req_src_port=5060 in_uri=sip:8135551212@e.f.g.h out_uri=sip:8135551212@e.f.g.h via_cnt==1".
.

#
U e.f.g.h:5060 -> a.b.c.d:5060
SIP/2.0 407 Proxy Authentication Required.
Via: SIP/2.0/UDP a.b.c.d:5060;branch=z9hG4bK3beb8c07;rport=5060.
From: "User Phone" ;tag=as664fbdbc.
To: ;tag=0dd4490c85a9eb8d48ff967a8700cef0.d2fe.
Call-ID: 165e2edb63fd81b30e629055245b8b28@a.b.c.d.
CSeq: 102 INVITE.
Proxy-Authenticate: Digest realm="star2star.com", nonce="valid_nonce".
Server: OpenSer (1.1.0-notls (i386/linux)).
Content-Length: 0.
Warning: 392 e.f.g.h:5060 "Noisy feedback tells: pid=4363 req_src_ip=a.b.c.d req_src_port=5060 in_uri=sip:8135551212@e.f.g.h out_uri=sip:8135551212@e.f.g.h via_cnt==1".
.

#
U a.b.c.d:5060 -> e.f.g.h:5060
CANCEL sip:8135551212@e.f.g.h SIP/2.0.
Via: SIP/2.0/UDP a.b.c.d:5060;branch=z9hG4bK3beb8c07;rport.
From: "User Phone" ;tag=as664fbdbc.
To: .
Call-ID: 165e2edb63fd81b30e629055245b8b28@a.b.c.d.
CSeq: 102 CANCEL.
User-Agent: Asterisk PBX.
Max-Forwards: 70.
Remote-Party-ID: "User Phone" ;privacy=off;screen=no.
Content-Length: 0.
.

#
U e.f.g.h:5060 -> a.b.c.d:5060
SIP/2.0 483 Too Many Hops.
Via: SIP/2.0/UDP a.b.c.d:5060;branch=z9hG4bK3beb8c07;rport=5060.
From: "User Phone" ;tag=as664fbdbc.
To: ;tag=0dd4490c85a9eb8d48ff967a8700cef0.f74a.
Call-ID: 165e2edb63fd81b30e629055245b8b28@a.b.c.d.
CSeq: 102 CANCEL.
Server: OpenSer (1.1.0-notls (i386/linux)).
Content-Length: 0.
Warning: 392 e.f.g.h:5060 "Noisy feedback tells: pid=4365 req_src_ip=e.f.g.h req_src_port=5060 in_uri=sip:8135551212@e.f.g.h out_uri=sip:8135551212@e.f.g.h via_cnt==71".
.


So, in this scenario the INVITE comes in and OpenSER responds with a 407. The remote endpoint (Asterisk) never receives the 407, gives up on the request, sending a CANCEL after about 60 seconds. At this point I'm not really sure what happens but something loops in OpenSER until Max-Forwards: is exceeded.

We can verify from packet captures on the remote Asterisk system that the 407 is not being received and therefore, Asterisk isn't resending the INVITE with auth.

What's doing on here? We're not seeing any other messages being lost, we're not seeing packet loss, what's going on?