Chewing the Cud (Posts about Work)

Sponsor Me!

James Aimonetti — Thu, 31 Oct 2019 22:55:23 GMT

Recently learned that I was accepted to Github's beta sponsorship program which allows me to define sponsorship tiers!

https://github.com/sponsors/jamesaimonetti

My hope is that folks that have appreciated my work with KAZOO and help in the forums, documentation, and IRC, might throw a bone or two of thanks my way. If there's a better tier that would get you on board, let me know.

Comparing Kazoo and other CPaaS solutions

James Aimonetti — Thu, 31 May 2018 22:07:29 GMT

Introduction

First, what is a CPaaS? Communication Platform as a Service is the expanded acronym but it boils down to APIs and programmatic control of the phone system.

The power of CPaaS is giving you the control of each call in a way that makes sense for your (or your business') needs. It lets you access information outside of the phone system (such as CRMs and other databases) to make decisions about how to route a particular call.

Another feature of CPaaS is the ability to get events out of the phone system - via webhooks or over a websocket typically. This gives you the ability to track a call's progress through the callflow, updating information in CRMs and other databases and lets you build robust applications on top of the phone network.

CPaaS providers often offer API access to manipulate running calls (such as programmatically hanging up a call, initiating a transfer, etc), fetching CDRs (call detail records - typically used when billing for calls), initiating new calls, and more.

A newer feature, a requirement of CPaaS providers these days, is having WebRTC-based phones (or a phone in your browser), minimizing the software needed to be installed and allowing companies to provide a branded phone in their web apps.

Most CPaaS providers allow you to process phone calls without being a telecom junkie by providing a simplified set of actions that may be performed on a given call. No longer do you need to know what the PSTN is or how SIP or SDP work - you get to build on top of an abstracted phone system and focus more on your business needs.

The competitive landscape

When you view the competitive landscape in telecom, it is easy to see that the big players (traditional telcos and UCaaS providers like Avaya, Broadsoft, and others) are recognizing the value of adding a CPaaS-type offering by purchasing CPaaS providers like Zang and Nexmo (via Cisco acquisition). Twilio stands apart as an independent CPaaS provider that IPO'd in 2016.

The market has shifted and CPaaS + UCaaS is the norm going forward; UCaaS companies are racing to catch up while CPaaS companies are trying to expand into more traditional telephony environments.

Technical Evaluation

Let's dive into a bit of the nitty-gritty and see how a handful of providers compare.

Real-time Call Control

Most CPaaS offer a small list of actions/verbs that can be returned from your program. The basics are covered by most providers.

Connecting callers to endpoints

When a caller calls in, typically you want them to talk to someone (or several someones). Below is a quick chart on the various endpoints supported by CPaaS providers for connecting the caller:

Action	Kazoo	Twilio	Plivo	Nexmo	Tropo
Registered Device	✔	X	X	X	X
SIP Endpoint	✔	✔	✔	✔	✔
Conference	✔	✔	✔	X	✔
DID(s)	✔	✔	✔	✔	✔
Call Queue	-	✔	X	X	X
User (find-me/follow-me)	✔	X	X	X	X
Group	✔	X	X	X	X
Page group	✔	X	X	X	X

Some notes:

Kazoo is the only platform that allows devices to register directly to the platform to receive calls. Registration is the phone's way of saying "Send calls for me to this IP/port please". Devices can then move around (say a softphone on a laptop) but still be able to receive calls easily. The SIP endpoint, in contrast, is typically in a fixed location and is harder to move once setup. An existing PBX is the typical destination for SIP endpoints.
Twilio is the only provider that offers a "Queue" option in the public docs. Kazoo has a community-supported queuing applications called ACDc as well as a commercial offering called Qubicle, which can be leveraged for call queues. You can also simulate queues with conferences.
The 'User' endpoint is the ability to associate multiple devices with a user and ring all relevant devices when the user is dialed. Most providers allow you to specify multiple endpoints to dial but that association must be tracked on your side. With Kazoo, you can create a User and use that in your callflow instead.
The 'Group' endpoint is similar to the User in that you can add devices and users to a group and all associated devices will be rung.

The biggest differentiate among the providers in this case comes down to the need for registration, insight into why a call failed to connect (some providers are quite opaque about the reason for failure), and additional features offered on each endpoint type (like setting custom ringtones).

Caller interactions

Most providers allow you to interact with the caller by playing audio files, generating dynamic text to read to the caller (TTS - text to speech), and collecting touch tones (DTMF) from the caller.

Action	Kazoo	Twilio	Plivo	Nexmo	Tropo
Collect DTMF	✔	✔	✔	✔	✔
Play Audio (MP3 or WAV)	✔	✔	✔	✔	✔
TTS	✔	✔	✔	✔	✔

Other basics offered

Action	Kazoo	Twilio	Plivo	Nexmo	Tropo
Hangup a call	✔	✔	✔	X	✔
Pause execution	✔	✔	✔	X	✔
Record the caller	✔	✔	✔	✔	✔
Redirect to different URL	✔	✔	✔	X	✔
Reject a call	✔	✔	✔	X	✔

Kazoo Differentiators

While CPaaS allows you to interact with phone calls at a higher level than processing SIP packets, the features above are fairly low-level as far as business concerns go. They make for great demos and getting a prototype out the door quickly.

Invariably, once you gain traction, feature requests come in and more functionality is needed. The question to ask is, do you have to do the work to implement the functionality or is it provided already?

For instance, say your prototype connects the caller, based on some fancy criteria, to an agent/sales person/fortune teller. You gain traction and now your callees (the agents/sales people/fortune tellers) would love to have a voicemail service in case they can't answer the phone (since most are using a personal mobile phone and don't want the caller to hear their personal voicemail).

Do you want to build a voicemail box?

With Kazoo, you can configure a voicemail box via API and add it to the callflow with minimal effort. With the other providers, you're looking at having to get the IVR right, get the prompts recorded, test all the paths through a voicemail box, figure out storage of the voicemail recordings, build an API for providing the voicemail messages, build voicemail->email and attach the recording, archive/delete old/deleted messages, and more. This is not quick to build and, frankly, most people don't consider voicemail a feature worth paying much, if anything, for; is it a good use of your time?

Kazoo offers a number of callflow actions, like voicemail, that allow you to add higher level, business-focused, actions to your call processing capabilities. A short list of highlights:

Call Parking
Directed / Group Call Pickup
Eavesdrop
Faxing
Hotdesking
IVRs
Manual presence updates
Time-of-day routing
Voicemail

Each of these features represents days to weeks of work to implement yourself (if you even can given the primitives offered by the other providers) and each has its own set of edge cases and intricacies that come with it. Instead, why not use the pre-built, heavily-tested callflow actions provided by Kazoo?

API control

The single largest difference between Kazoo's REST API and other providers is the sheer breadth of APIs exposed by Kazoo. Remember, Kazoo is both a CPaaS and UCaaS platform; on top of that, everything is API driven so you while you might start with the CPaaS-specific features, you can easily grow into UCaaS type services (like office PBX features).

Call Control APIs

Each provider allows you to list and control active calls. What controls you have vary; some allow you to redirect the executing callflow to your server for new callflow instructions; others expose the actions you can overlay on the call via the API and that's it.

Similarly, conferences and conference participants can be managed through most providers' APIs.

All in all, there isn't much that differentiates the basics among providers.

Kazoo API Differentiators

Multi-tenancy

Kazoo is designed to be multi-tenant; you can have as deep or as broad of an account tree as you like. Most providers, like Twilio, Nexmo, and Tropo, don't provide multi-tenancy at all (Plivo does appear to though). As a service integrator, you will have to manage that hierarchy on your side. This means you have to build infrastructure to manage billing, resource usage, configuration, etc.
Carrier management

Kazoo allows you to define your own upstream carriers, so even if you're using a hosted instance of Kazoo (versus running it yourself), you should be able to add your own carriers to you account and use them instead of the system's carriers. You can even setup hybrid situations where you route certain destinations to your carriers and use the system's as the default carrier group.

None of the other CPaaS providers offer BYOC (Bring Your Own Carrier). This means that as you grow your voice traffic, you will not have levers to pull to manage costs. With BYOC, when it makes sense to, you can find your own deals on connectivity and seamlessly switch to your own carriers to save costs.
Finer-grained features
Each API entity (such as devices or conferences) in Kazoo offers significantly more functionality compared to the other providers' equivalent entities. Of course there are some similarities but, for instance looking at a "device" in Kazoo, you can optionally define:
- Call forwarding settings, including for failover (or just straight call-forwarding)
- Call recording (both inbound and outbound) for the device
- Call restrictions based on number classifiers (can call US DIDs, can't call international, etc)
- Caller ID settings
- Dialplan settings (allow users to dial shorter numbers and "fix" them - for instance the US used to allow 7-digit dialing and the area code was assumed based on the phone's location).
- Do Not Disturb settings
- Codecs available (both audio and video)
- Custom music on hold
- Custom presence ID (instead of the SIP username)
There are other Kazoo-specific features on devices as well. Since users can own devices, you can set similar properties on the user and have them propagate to all devices (and define properties on the devices to override user settings).
Class-4 and Class-5 Switching

CPaaS providers are typically concerned with a handful of phone system features - dialing endpoints, conferences, call recording, call queue, and phone number management most commonly.

Kazoo, being a hybrid class-4 and class-5 switch, offers so much more:

PBX functionality includes configuring users, devices, and callflows, blacklists, directories, faxes and faxboxes, IVRs, voicemail, and more.

Configure connectivity features like carriers (resources in Kazoo parlance), ratedecks, PBX trunking.

Configure reseller features like branding email notification templates and white-labeling Kazoo services.

Questions to ask a CPaaS provider

These are some good questions to ask your CPaaS provider(s) and to ask yourself how important the answers are to your needs:

Where are the voice servers located, geographically?
Where are the API servers located? Hint, some providers may have voice spread out around the world but API servers only in geographic location.
Where are the servers located (the data centers) relative to your users? To your carriers?
How is the networking built? Some hosted infrastructure providers, like AWS/GCE aren't well suited to VoIP traffic - are CPaaS providers piggybacking off that infrastructure or are they building out their own racks?
Is the CPaaS provider reselling another provider's service with better lipstick?
Can you extend the platform?

Conclusion

The market has spoken and they want UCaaS+CPaaS offered under one roof. The big UCaaS players have bought their CPaaS plays and are busy integrating them into the fold with varying success. Meanwhile Twilio remains an outlier and looks to be doubling down on being the CPaaS for the masses.

Kazoo offers you CPaaS + UCaaS from day one in a cohesive system. You can use as many or as few APIs and callflow actions as you need to accomplish your goals, while knowing that as you grow and need new functionality, Kazoo is there ready to provide it. On the off chance Kazoo isn't able to provide a feature, you can rest easy knowing the open-source nature of the project gives you the option to build it yourself and keep it private, build it and release it upstream for all to benefit, or support the project by contracting with 2600Hz to build the functionality for you.

Ready to kick the tires?

Check out the architecture page.
Spin up a CentOS 7 virtual machine and install Kazoo.
Contact sales for a demo.
Check out the Git repo
Sign up for the forums

Using ibrowse to POST form data

James Aimonetti — Thu, 26 Apr 2012 15:22:31 GMT

It is not immediately obvious how to use ibrowse to send an HTTP POST request with form data (perhaps to simulate a web form post). Turns out its pretty simple:

ibrowse:send_req(URI, [{"Content-Type", "application/x-www-form-urlencoded"}], post, FormData)

Where URI is where you want to send the request ("http://some.server.com/path/to/somewhere.php") and FormData is an iolist() of URL-encoded values ("foo=bar&fizz=buzz"). There's obviously a lot more that can be done, but for a quick snippet, this is pretty sweet.

Emulating Webmachine's {halt, StatusCode} in Cowboy

James Aimonetti — Thu, 15 Mar 2012 12:26:05 GMT

At 2600Hz, we recently converted our REST webserver from Mochiweb/Webmachine to Cowboy, with cowboy\_http\_rest giving us a comparable API to process our REST requests with. One feature that was missing, however, was an equivalent to Webmachine's {halt, StatusCode} return. While there has been chatter about adding this to cowboy\_http\_rest, we've got a function that emulates the behaviour pretty well (this is cleaned up a bit from our actual function, removing project-specific details).

-spec halt/4 :: (#http_req{}, integer(), iolist(), #state{}) -> {'halt', #http_req{}, #state{}}.
halt(Req0, StatusCode, RespContent, State) ->
    {ok, Req1} = cowboy_http_req:set_resp_body(Content, Req0),
    {ok, Req2} = cowboy_http_req:reply(StatusCode, Req1),
    {halt, Req2, State}.

Obviously you can omit setting the response body if you don't plan to return one.

CouchDB/BigCouch Bulk Insert/Update

James Aimonetti — Fri, 27 Jan 2012 19:57:28 GMT

While writing a bulk importer for Crossbar, I took a look at squeezing some performance out of BigCouch for the actual inserting of documents into the database. My first time running all the documents into BigCouch at the same time resulted in some poor performance, so I went digging around for some ideas on how to improve the insertions. Reading up on the High Performance Guide for CouchDB (which BigCouch is API-compliant with), I started to play with chunking my inserts up to get better overall execution time. Note: the following are very unscientific results, but I think are fairly instructive for what one might expect.

Docs Per Insertion	Elapsed Time (ms)
26618	107176
1000	8325
1500	5679
2000	3087
2500	1644
Docs Per Insertion	Elapsed Time (ms)

Based on the CouchDB guide, I decided to not pursue this further, as dropping insertion time 2 orders of magnitude was fine enough for me! I may have to bake this into the platform natively. For those interested in the Erlang code, it is pretty simple. Taking a list of documents to save, use lists:split/2 to try and split the list. By catching the error, we can know that the list is less than our threshold, and can save the remaining list to BigCouch. Otherwise, lists:split/2 chunks our list into one for saving, and one for recursing back into the function. Since we don't really care about the results of couch\_mgr:save\_docs/2, we could put the calls in the second clause of the case in a spawn to speed this up (relative to the calling process).

-spec save_bulk_rates/1 :: (wh_json:json_objects()) -> no_return().
save_bulk_rates(Rates) ->
    case catch(lists:split(?MAX_BULK_INSERT, Rates)) of
        {'EXIT', _} ->
            couch_mgr:save_docs(?WH_RATES_DB, Rates);
        {Save, Cont} ->
            couch_mgr:save_docs(?WH_RATES_DB, Save),
            save_bulk_rates(Cont)
    end.

cURL stripping newlines from your CSV or other file?

James Aimonetti — Thu, 26 Jan 2012 19:33:43 GMT

I'm in the process of writing a REST endpoint for uploading CSVs to Crossbar as part of our communications platform at 2600hz. Not wanting to invoke the full REST client interface, I generally use cURL to send the HTTP requests. Today, however, I had quite the time figuring out why my CSV files were being stripped of their newline characters. The initial invocation:

$> curl http://localhost:8000/v1/path/to/upload -H "Content-Type: text/csv" -X POST -d @file.csv

Walking through the code, from where I was processing the CSV down to the webserver handling the connection itself, looking for who was stripping the newlines, I determined it was coming in sans-newlines and decided to check out cURL's man pages for what might be amiss. I quickly found that the -d option was treating the file as ascii, and although the docs don't explicitly say so, it appears this option will strip the newlines. The resolution is to use the –data-binary flag so cURL doesn't touch the file before sending it to the server.

Cron and infinite loops do not mix

James Aimonetti — Wed, 09 Mar 2011 17:50:00 GMT

More "expert" code time! From the "expert":

Please put this script in a cron to run every minute

while true; do
  rsync -a server:remote_dir local_dir
  sleep $freq
done

local\_dir is going to be really, really, really up to date after a few minutes…the server crash will be epic. Perhaps we should write a script to find and kill these rogue processes and run it every minute too, but stagger it with the other cron…

You get paid for this?

James Aimonetti — Mon, 07 Mar 2011 13:08:37 GMT

Spotted in some high-priced "expert"'s code:

switch ($retcode)
{
    case -1:
    case -3:
        if ($retcode==-1)
            log("SOME_CODE", "SOME MSG");
        else
            log("SOME_OTHER_CODE", "SOME OTHER MSG");
...

Resolving Dialyzer "Function foo/n has no local return" errors

James Aimonetti — Tue, 23 Nov 2010 12:03:31 GMT

Dialyzer is a great static analysis tool for Erlang and has helped me catch many bugs related to what types I thought I was passing to a function versus what actually gets passed. Some of the errors Dialyzer emits are rather cryptic at first (as seems commonplace in the Erlang language/environment in general) but after you understand the causes of the errors, the fix is easily recognized. My most common error is Dialyzer inferring a different return type that what I put in my -spec, followed by Dialyzer telling me the same function has no local return. An example:

foo.erl:125: The specification for foo:init/1 states that the function might also return {'ok',tuple()} but the inferred return is none()
foo.erl:126: Function init/1 has no local return

The init/1 function (for a gen\_server, btw):

-spec(init/1 :: (Args :: list()) -> tuple(ok, tuple())).
init(_) ->
  {ok, #state{}}.

And the state record definition:

-record(state, {
  var_1 = {} :: tuple(string(), tuple())
  ,var_2 = [] :: list(tuple(string(), tuple()))
}).

Spot the error? In the record definition, var\₁ is initialized to an empty tuple and var\₂ is initialized to an empty list, yet the spec typing for the record does not take that into account. The corrected version:

-record(state, {
  var_1 = {} :: tuple(string(), tuple()) | {}
  ,var_2 = [] :: list(tuple(string(), tuple())) | []
}).

And now Dialyzer stops emitting the spec error and the no local return error.

Still Kicking

James Aimonetti — Fri, 17 Sep 2010 13:06:14 GMT

I am still alive and well; just busy. I did write a blog entry for my company, 2600hz. More to come…eventually.