Building a Startup (the right way)

(Continued from Building a Startup)

 

Our way wasn’t working.  To put it mildly our “business grade” solution didn’t perform much better than Vonage.  We became to exemplify VoIP - jittery calls, dropped calls, one way calls, etc, etc, etc.  Most of this was because of the lack of quality ITSPs at that time.  Either way our customers didn’t care.  It was us.  If we went to market with what we had the first time around we were going to loose.

The problem was the other predominant architecture at the time was “hosted”.  Someone hosts a PBX for you and ships you some phones.  You plug them in behind your router and magically you have a phone system.  They weren’t doing much better.  Sure, their sales looked good but even then it was becoming obvious customer churn was quite high.  People didn’t like hosted either, and for good reason.  Typically they have less control over the call than we do.

As I’ve eluded before I thought there was a better way.  We needed to host the voice applications where it made the most “sense”.  We were primarily using Asterisk and with a little creative provisioning, a kick-ass SIP proxy, and enough Asterisk machines we could build the perfect business PBX - even if that meant virtually none of it existed at the customer premise.  Or maybe all of it did.  That flexibility was key.  After a lot of discussions, whiteboard sessions, and late nights everyone agreed.  We needed a do-over.

So we got to work and slowly our new architecture began to take shape.  We added a kick-ass SIP proxy (OpenSER).  OpenSER would power the core routing between various Asterisk servers each meeting different needs - IVR/Auto Attendant, Conferencing, Voicemail, remote phones (for “hosted” phones/softphones), etc.  The beauty was the SIP proxy could route between all of these different systems including the original AstLinux system at the customer premise.  Customer needs to call voicemail?  No problem - the AstLinux system at the CPE fires an INVITE off to the proxy and the proxy figures out where their voicemail server is.  The call is connected and the media goes directly between the two endpoints.  Same thing for calls between any two points on the network - AstLinux CPE to AstLinux CPE, PSTN to voicemail, IVR to conference.

This is a good time to take a break and acknowledge what really made this all possible - OpenSER.  While it’s difficult to explain the exact history and family tree with any piece of SER software I can tell you one thing - this company would not be possible without it.  There is no question in my mind.  It’s now 2011 and whether you select Kamailio or OpenSIPS for your SIP project you will not be sorry.  Even after five years you will not find a more capable, flexible, scalable piece of SIP server software.  It was one of the best decisions we ever made.

Need to add another server to meet demand for IVR?  No problem, bring another server online, add the IP to a table and presto - you’re now taking calls on your new IVR.  Eventually a new IVR lead to several new IVRs, voicemail servers, conference systems, web portals, mail servers, various monitoring systems, etc.

What about infrastructure?  Starting at our small scale, regional footprint, and focus on quality we began to buy our own PRIs and running them on a couple of Cisco AS5350XM gateways.  This got us past our initial issues with questionable ITSPs.  Bandwidth was becoming another problem...  We had an excellent colocation provider that provided blended bandwidth but still we needed more control.  Here came BGP, ARIN, AS numbers, a pair of Cisco 7206VXRs w/ G2s, iBGP, multiple upstream providers, etc.

At times I would wonder - whatever happened to spending my time worrying about cross compilers?  Looking back I’m not sure which was worse - GNU autoconf cross-compiling hell or SIP interop, BGP, etc.  It’s fairly safe to say I’m a sadomasochist either way.

Even with all of the pain, missteps, and work we finally had an architecture to take to market.  It would be the architecture that would serve us well for several years.  Of course there was more work to be done...

Building a Startup

(Continued from Starting a Startup)

After several days of meetings in Sarasota we determined:

1)  I was moving to Sarasota to start a company with Norm and Joe.
2)  We were going to utilize open source software wherever possible (including AstLinux, obviously).
3)  The Internet was the only ubiquitous, high quality network to build a nationwide platform.
4)  The Internet was only getting more ubiquitous, more reliable, and faster in coming months/years/decades/etc.
5)  We were going to take advantage of as much of this as possible.

These were some pretty lofty goals.  Remember, this is early 2006.  Gmail was still invitation-only beta.  Google docs didn’t exist.  Amazon EC2 didn’t exist.  “Cloud computing” hadn’t come back into fashion yet.  The term itself didn’t exist.  The Internet was considered (by many) to be “best effort”, “inherently unreliable”, and “unsuitable” for critical communications (such as real time business telephony).  There were many naysayers who were confident this would be a miserable failure.  As it turns out, they were almost right.

We thought the “secret sauce” to business grade voice over the internet was monitoring and management.  If one could monitor and manage the internet connection business grade voice should be possible.  Of course this is very ambiguous but it lead to several great hires.  We hired

Joe had already deployed several embedded Asterisk systems to various businesses in the Sarasota area.  They used an embedded version of Linux he patched together and a third party (unnamed) “carrier” to connect to the PSTN.  The first step was upgrading these machines and getting them on AstLinux.  Once this was accomplished we felt confident enough to proceed with our plan.  This was Star2Star Communications and in the beginning of 2006 it looked something like this:

1)  Soekris net4801 machines running AstLinux on the customer premise.
2)  Grandstream GXP-2000 phones at each desk.
3)  Connectivity to a third party “ITSP”.
4)  Management/monitoring systems (check IP connectivity, phone availability, ITSP reliability, local LAN, etc).
5)  Central provisioning of AstLinux systems, phones, etc.

This was Star2Star and there was something I really liked about it - it was simple.  Anyone who knows me or knows of my projects (AstLinux, for example) has to know I favor simplicity whenever possible.  Keep it simple, keep it simple, keep it simple (stupid).

As time went on we started to learn that maybe this was too simple.  We didn’t have enough control.  Out monitoring wasn’t as mature as it should be.  We didn’t pick the right IP phones.  These could be easily fixed.  However, we soon realized our biggest mistake was architecture (or lack thereof).  This wasn’t going to be an easy fix.

We couldn’t find an ITSP that offered a level of quality we considered to be acceptable.  Very few ITSPs had any more experience with VoIP, SIP, and the internet than we did.  More disturbing, however, was an almost complete lack of focus on quality and reliability.  No process.

What we (quickly) discovered is the extremely low barrier to entry for ITSPs, especially back then.  Virtually anyone could install Asterisk on a $100/mo box in a colo somewhere, buy dialtone from someone (who knows) and call themselves an ITSP.  After going through several of these we discovered we needed to do it ourselves.

Even assuming we could solve the PSTN connectivity problem we discovered yet another issue.  All of the monitoring and management in the world cannot make up for a terrible last mile.  If the copper in the ground is rotting and the DSL modem can only negotiate 128kbps/128kbps that’s all you’re going to get.  To make matters worse in the event of a cut or outage the customer would be down completely.  While that may have always happened with the PSTN and an on premise PBX we considered this to be unacceptable.

So then, in the eleventh hour, just before launch I met with the original founders and posed a radical idea - scrap almost everything.  There was a better way.
(Continued in Building a Startup (the right way))