News from Industry

If you are new to WebRTC then you have missed out on years of drama in the standards bodies over various issues like SDP and codecs. These standards dictate what vendors must implement so they ultimately dictate the industry roadmap.  To get a deep perspective and appreciation of the issues, we like to ask Dan Burnett, W3C editor to comment on where we are at with the standardization process. I caught up with Dan at this year’s IIT Real Time Communications Conference and had the more detailed Q&A with him shortly thereafter.

We asked Dan to comment on recent spec changes, ORTC, the next version of WebRTC, codecs, Apple, when the 1.0 spec might ever be finalized, and a whole lot more.

{“editor”, “chad hart“}

New Governance

webrtcHacks: Hi Dan. Can you describe some of the recent changes to the W3C WebRTC governance?

Dan: Yes. There was a long-running but productive discussion among the members of the WebRTC Working Group (WG), ORTC Community Group (CG), and the some of the members of the W3C advisory committee – which is the group that officially determines group charters.

As part of the Charter renewal process, we decided that there would be one additional Chair of the WebRTC Working Group –  Eric Lagerway of Hookflash who was one of the initiators of ORTC. Also the decision was that the WebRTC WG is the official group where all future standardization work in WebRTC will happen, meaning the ORTC work will gradually fold into that group.

Additionally, the group was chartered to work on another version beyond 1.0 – WebRTC Next Version or WebRTC-NV.

There are 2 requirements on that version:

1. There is no requirement that new features introduced in the specification have an SDP equivalent
2. WebRTC NV is not a replacement for WebRTC 1.0 – it is an extension. It is expected that all browsers that support WebRTC NV will support 1.0 functionality as well.

One other thing has happened that is not official, but is probably good is that Bernard Aboba from Microsoft has joined the WebRTC 1.0 editing team.

The Next Version

webrtcHacks: yeah, Bernard mentioned that in the interview I did with him last week. Can you explain WebRTC NV? Why didn’t you just call it 2.0, or 1.1, or whatever?

Dan: I have been working on standards for a long time. I have seen groups spend ridiculous amounts of time deciding on a name for a specification.  In this particular case a “1.1” sounds like a minor change from “1.0” while “2.0” sounds like a major change. Some people want a minor change. Some people want a major change. If enough people want different minor changes it will end up being a 2.0 anyway because of the number of changes. The goal was to avoid that disagreement now so that we can move forward,.

webrtcHacks: So what is WebRTC NV then, beyond what you stated earlier about no SDP?

Dan: Nothing is officially decided but I expect that there will continue to be more low-level controls as in ORTC. This is complicated by the fact that new feature proposals are continuing to come in for 1.0. Many of these features are from ORTC.

In the Sapporo meeting coming up, Google will be sharing their idea for what should go into WebRTC-NV when we finally start working on it.

Dan at the IIT-RTC Conference

webrtcHacks: How do you see ORTC influencing the WebRTC spec? Is WebRTC-NV really just ORTC?

Dan: If it had to summarize WebRTC-NV I would say that it is the combination of WebRTC 1.0 and ORTC. It is a requirement that 1.0 applications continue to work in WebRTC-NV implementations. It is not required that ORTC applications work directly in WebRTC-NV.

I believe the ORTC community intends to modify ORTC as necessary to remain consistent with WebRTC as it evolves.

webrtcHacks: Is there an end-date to ORTC-then? When it is mostly merged with WebRTC-NV will it cease to exist?

Dan: I can’t speak for the ORTC group. I have not heard of an end date. You’ll have to ask one of the primary ORTC contributors.

Spec Changes

webrtcHacks: What are some of changes made to the specs recently. Particularly those that impact the developers out there?

Dan: First I would like to give a little plug for my webrtcstandards.info site where I have been putting exactly that sort of information over the past few months. I will mention some things here, but you can get more details on that site.

webrtcHacks: ok, we’ll give you one plug (laughs)

Dan: One of the biggest and most relevant changes on what we were just talking about is the introduction of the RTCsenders and RTCreceivers. These are objects that allow for both information and more direct control over how tracks are sent over a PeerConnection. Notice as part of this that we have moved from a stream based API to a track based API.

webrtcHacks: And what advantage does the track approach provide?

Dan: It turns out developers want to have more control over exactly how tracks are sent and received. For example being able to specify which codecs are to be used and the parameters used to configure those codecs. They should be able to configure some transport properties as well on a per track basis such as FEC, retransmission, and bandwidth. Because of this it really didn’t make sense to talk about streams as the primary primitive being sent over a PeerConnection since they are really just a collection of tracks.

One of Peter Thatcher ORTC update slide’s showing the differences between the WebRTC and ORTC API. source: IIT-RTC 2015

webrtcHacks: So the others?

Dan: First, on the one we just mentioned – that was a foundational change where we are going to be seeing many other changes later on. Now I’ll talk about the others that are not related to that.

One big change was the API’s have been converted to use ECMAScript Promises. I think I mentioned this last year.

webrtchacks: You did.

Dan: It has happened. It is now in the specifications.

Promises are now the recommended mechanism for WebRTC specifications and for web specifications in general for dealing with asynchronous function calls. Not so much for things that generate multiple events, but definitely for any single asynchronous function call.

This is part of the move of ECMAscript toward truly asynchronous function calls as you can see if you look at some of the thoughts or future versions of ECMAscript.

The original callback based API’s currently still exist but will eventually be deprecated. Developers should start using the Promise versions.

webrtcHacks: I know media capture from the DOM is another one.

Dan: There has been good progress on capturing media directly from media elements such as audio, video and canvas. Developers have had to use hacks up to this point to be able to capture a canvas for example. Maybe they would take snapshots, but that is not the same as a realtime media stream as you would get from a getUserMedia call.

The major changes going into the specification soon  are to try to reproduce the resulting media stream as faithfully as possible to what a user would experience from that element. For example, if the user is playing a video and pauses it and then resumes, the resulting stream should show the paused video for the amount of time it was paused and then resume again.

This seems to be what developers are most interested in.

webrtcHacks: can you talk about some of the use cases that are being referenced around this feature?

Dan: Shared whiteboard is probably the best example, but there maybe some instances for training purposes where you want to capture how the user has interacted with existing elements – video or audio.

webrtcHacks: What about screensharing?

Dan: There is good progress happening there as well on the specification. It still has some tricky issues in terms of what apps should be able to request to be shared and what users should have control over. An example of this is Microsoft Powerpoint – if a user has 3 powerpoint documents up – say different presentations for different clients; they are likely to only want to share one one of those presentations – one window of that application. That works great until they go into presentation mode, which is far as the computer is concerned is a different window. So is this a case where the user should decide or is this a case where the application should decide what is shared?

In general the WG believes that the user should have the control, but browsers may have to make special cases for known applications such as Powerpoint so that it just works.

webrtcHacks: How about simulcast?

Dan: At the Seattle meeting there were some strong opinions on how simulcast should work and some proposals. Each time we get to the details the discussions diverge rather than converge. We all want it but we do not agree on how it should be signaled.

Timelines

webrtcHacks: Now for an easier one. When will 1.0 be done?

(laughs)

Dan: I am tempted to give a similar answer as last year.

There are 2 primary specifications. The media capture specification is right now finishing up addressing the comments from its Last Call review which is the wide range review that is required in order to go forward. There aren’t any new features being requested by group members – it’s just cleaning up and fixing.

It probably will be stable within another 6 months.

webrtcHacks: Stable meaning not changing any more?

Dan: Yes – meaning no contentful changes. Only editorial fixes.

Now the WebRTC specification has the problem that new features keep coming in.

werbrtcHacks: Just to clarify – the Media Capture group is the getUserMedia API and when you WebRTC, that means the RTCPeerConnection and DataChannel related API’s?

Dan: Yes.

These are features that have come from ORTC. At each meeting we have tried to finalize the list, but new proposals continue to creep in. Within 6 months we will know whether the chairs have been able to hold the line on the most recent list agreed to in Seattle.

webrtcHacks: So is this why it is taking so long?

Dan: Yes.. The good news about it is that the features that are going in are the most requested ones from ORTC.

IP Leakage

webrtcHacks: The IP leakage issue was a hot topic on webrtcHacks and elsewhere? Many have labeled it as a flaw; other say this behaviour was by design? Can you share the “standards” perspective on this topic and the considerations that were discussed?

Dan: The summary is this – there are 2 problems with IP leakage:

One kind is the leakage of public addresses that the user doesn’t want leaked. This can happen when a user is using a VPN and not all of the traffic is sent over the VPN – a so called split tunnel VPN.  This is an issue if the user doesn’t want their non-VPN public address to be revealed. This is not a WebRTC problem; this is a split tunnel VPN problem. That doesn’t mean that people don’t blame the browser vendors even though it’s not their fault (laughs}

Technically any application running on your machine could do the same thing if you’re running a split tunnel VPN. There are extensions to turn off WebRTC for people who are very concerned about this.

The other kind of leakage is leakage of your local IP address. the reason this concerns some people is that it can be used to map the topology of your local network, say within an enterprise. However it turns out that applications can use an XmlHttpRequest to do the same thing.  Despite that, the browser vendors are working on ways to turn off the reporting of these local addresses.

There will be more details coming up in an upcoming post on my site.

Dan talking to webrtcHacks guest author Alan Jonhston at the IIT-RTC show

What’s Apple Doing?

webrtcHacks: Now the only major browser vendor left is Apple. Can you comment on public participation by Apple?

Dan: It is clear that people from Apple are continue to follow the work, but they still don’t contribute.

webrtcHacks: Do you know if they contribute to other WG more actively.

Dan: Yes, Apple does contribute more actively in other WG within W3C.

Codecs

webrtcHacks: Anything new with video codecs now that the market has had some time to react to the decision to include both VP8 & H.264 for browsers? How is the VP9 vs. H.265 and Alliance for Open Media (AOM) discussion changed the discussion?

Dan: The gauntlet has been thrown for the creation of free and open source video codecs. MPEG-LA needs to take notice that the media producers and distributors are serious about coming up with lower cost alternatives. This pressure just continually increases. The AOM is a prime example of that.

webrtcHacks: Has the Alliance for Open Media come up in standards discussion? In the past I know there was discussion of just allowing software codecs that could defined on the fly.

Dan: Codecs still need to be created.  The discussions of VP8 vs H.265 and VP9 vs. H.265 are not really technical discussions. They are all about intellectual property because of the cost of licensing the codecs. The issue is not being able to select a codec – the issue is having a codec that you want to choose.

One API change that is just gone in is being able to choose which codec of the browser supported ones to use.

Microsoft

webrtcHacks: Anything else to add?

Dan: I think we’re finally on a good track in respect to a path forward for ORTC and WebRTC and thus the inclusion of Microsoft as a true and complete WebRTC vendor eventually. We just need the feature inflow from ORTC to stop right now to be able to declare victory and move on.

I think this is evidence that the industry really does want this to happen.

I spoke with a number of people who talk to HTML developer groups and they all agree that even today no more than 50% of the developers have heard of WebRTC – still! It is likely that one reason for that is for many developers a technology isn’t real until it is in Internet Explorer or its successor – Edge.

So having Microsoft fully engaged on a plan that we can all agree on now is a good thing for everyone.

 

{
  “Q&A”:{
    “interviewer”:“chad hart“,
    “interviewee”:“Dan Burnett“
  }
}

Want to keep up on our latest posts? Please click here to subscribe to our mailing list if you have not already. We only email post updates. You can also follow us on twitter at @webrtcHacks for blog updates and news of technical WebRTC topics or our individual feeds @chadwallacehart, @victorpascual and @tsahil.

The post Are we There Yet? WebRTC standards Q&A with Dan Burnett appeared first on webrtcHacks.

On Wednesday, October 21, 2015, at 17:00GMT (12:00CT, 18:00 London, 19:00 Berlin), the Cluecon weekly conference call will focus on Kamailio and FreeSwitch. I will be joining the call, answering the questions about Kamailio and its options to integrate with FreeSwitch. Expect the FreeSwitch core developers to be around to handle the questions about their project.Participation is open for anyone, you can dial in for audio or video sessions using a SIP phone or webrtc capable browser:

• sip:888@conference.freeswitch.org
• https://cantina.freeswitch.org/vc

More dial in options (PSTN) are presented at:

• https://freeswitch.org/confluence/display/FREESWITCH/ClueCon+Weekly+Conference+call

Enjoy!

A lot more than pure messaging.

Messaging used to be about presence and IM. Then the VoIP people came and placed the voice and video chat stickers on it. That then became unified communications. Which is all nice and well, but it is both boring and useless at this point. Useless not because the utility of the service isn’t there, but because the expectation of such a service is to be free – or close to that. Or as I like saying, it has now become a feature within another service more than a service in its own right.

While this is killing unified communications, it doesn’t seem to be making much of a dent on messaging just yet. And the reason I think is the two very different trajectories these are taking:

• Unified Communications is focused on being the one true source of everything that gets federated with all other communication means
• Messaging shifted towards becoming platforms, where the size of the ecosystem and its utility outweighs any desire or need to federate with other similar services

This migration of messaging towards becoming platforms isn’t so easy to explain. There’s no silver bullet of how this is done. No secret recipe that gets you there.

Here are a few strategies that different messaging platforms are employing in their attempt to gain future growth.

Whatsapp and Simplicity

Whatsapp is all about simplicity. It offers pure messaging that replaces the SMS for many, coupled with group messaging that makes it sticky and viral in many countries.

Features don’t make it into Whatsapp fast. The only thing that was added in the past two years of any notable value is voice calling.

With this approach, Whatsapp still is the largest player in town when it comes to messaging; and it is probably doing so with the smallest possible team size.

The problem with such an approach, is that there isn’t enough room for many such players – and soon, to be a viable player in this domain will require a billion monthly active users.

Apple and iMessage

In that same token, the Apple iMessage is similar. It is simple, and it is impossible to miss or ignore if you have an iPhone.

But it is limited to Apple’s ecosystem which only runs on iOS devices.

Google Hangout (and now Jibe Mobile)

Google Hangouts was supposed to do the same/similar on Android, but didn’t live up to the expectation:

• Unlike Whatsapp, group chat is available in Hangouts, but isn’t viral or “mandatory”
• Unlike Apple iMessage, the user needs to make a mental note of using Hangouts instead of the SMS app. There are two of those, and as a user, you are free to choose which one to us. Choice adds friction and omplexity

With the acquisition of Jibe Mobile, this may change in the future. Will others follow suit? Is there enough utility and need in connecting messaging with Telco messaging, and especially with RCS, that many (myself included, at least until this acquisition) see as dead on arrival?

Facebook and Artificial Intelligence

Facebook is experimenting with artificial intelligence that is embedded into their Facebook Messenger service – not the social network where e-commerce is the current focus.

This new AI initiative is called Facebook M and is planned to be driven by part machine part humans.

In many ways, this is akin to the integration LivePerson (a chat widget for contact centers) has with knowledge bases that can cater to customer’s needs without “harassing” live agents in some cases. But this one is built into the messaging service the customer uses.

It is compared to Siri and Cortana, but you can also compare it to Google Now – once Facebook fleshes out the service, they can open up APIs for third parties to integrate to it, making it a platform for engaging with businesses.

WeChat and the Digital Life Platform

WeChat is large in Asia and dominant in many ways. It is an e-commerce platform and a digital life ecosystem.

Connie Chan of Andreessen Horowitz gives a good overview of what makes WeChat a platform:

Along with its basic communication features, WeChat users in China can access services to hail a taxi, order food delivery, buy movie tickets, play casual games, check in for a flight, send money to friends, access fitness tracker data, book a doctor appointment, get banking statements, pay the water bill, find geo-targeted coupons, recognize music, search for a book at the local library, meet strangers around you, follow celebrity news, read magazine articles, and even donate to charity … all in a single, integrated app.

WeChat transitioned from being a communication tool to becoming a platform. It has APIs that makes it easy for third parties to integrate with it and offer their own services on top of WeChat’s platform.

While I use the term “from service to feature” when talking about VoIP and WebRTC, Connie Chan uses “where social is just a feature” to explain the transition WeChat has made in this space.

The ability to send messages back and forth and communicate in real time via voice and video is now considered table stakes. It is also not expected to be a paid service but a feature that gets monetized elsewhere.

Meanwhile in Enterprise Messaging

Slack, which Connie Chan also briefly notes in his account of WeChat, is the guiding light of enterprise messaging these days.

Unlike other players in this space, Slack has built itself around the premise of three strong characteristics:

• Integration – third parties can integrate their apps into Slack, and in many cases, Slack integrates automatically through links that get shared inside messages. Integrations that make sense and bring value to larger audiences of Slack gets wrapped into Slack – the acquisition of Screenhero and the plans to enhance it to video conferencing shows this route
• Omnisearch – everything in Slack is searchable. Including the content of links shared on Slack. This makes for a powerful search capability
• Slackbot – the slackbot is a Slack bot you can interact with inside the service. It offers guidance and some automation – and is about to enjoy artificial intelligence (or at the very least machine learning)

The enterprise platform is all about utility.

Slack is introducing AI and has its own marketplace of third party apps via integrations. The more enterprises use it, the more effect these two capabilities will have in enforcing its growth and effectiveness.

While the fight seems to be these days between Unified Communications and Enterprise Messaging, I believe that fight is already behind us. The winner will be Enterprise Messaging – either because UC vendors will evolve into Enterprise Messaging (or acquire such vendors) or because they will lose ground fast to Enterprise Messaging vendors.

The real fight will be between modern Enterprise Messaging platforms such as Slack and consumer messaging platforms such as WeChat – enterprises will choose one over the other to manage and run their internal workforce.

 

Kranky and I are planning the next Kranky Geek - Q1 2016. Interested in speaking? Just ping me through my contact page.

The post The Future of Messaging is… appeared first on BlogGeek.me.

Hello, again. This past week in the FreeSWITCH master branch we had 25 commits. Our feature this week in the verto communicator is the addition of ngSanitize as a dependency, vertoFilters module and picturify filter, and a change to the chat image display behavior (break line before rendering).

Join this Wednesdays at 12:00 CT for some more FreeSWITCH fun with Daniel-Constantin Mierla! And head over to freeswitch.com to learn more about FreeSWITCH support.

New features that were added:

• FS-8030 [verto_communicator] Added ngSanitize as a dependency, vertoFilters module and picturify filter and changed chat image display behavior (break line before rendering).

Improvements in build system, cross platform support, and packaging:

• FS-8194 FS-7910 FS-7937 Various systemd service improvements
• FS-8350 [build] Fix Windows build errors.
• FS-7928 FS-7618 [Debian] Systemd and package build improvements

The following bugs were squashed:

• FS-8328 [mod_conference] Fixed missing ‘else’ keyword
• FS-8306 [mod_amqp] If the exchange doesn’t exist, then create it, else fail. This resolves several error cases. And now command queues can specify the queue to subscribe to. This enables very interesting use cases that would involve single job queue, and multiple consumers.
• FS-8331 [verto_communicator] Do not show reconnect splash when user has clicked logout
• FS-8335 [mod_easyroute] Fixed a small error check that results in error message not being displayed
• FS-6833 FS-6834 [mod_sofia] Found and fixed a few missing content-types in requests/responses with SDP that were outside the norm
• FS-8338 [core] Fix for ringback not working correctly on stereo channels
• FS-7834 [mod_sofia] Fixed MOH not working with inbound-bypass-media and resume-media-on-hold
• FS-8287 [mod_local_stream] Fixed a segfault from refactor

And, this past week in the FreeSWITCH 1.4 branch we had 2 new commits merged in from master. And the FreeSWITCH 1.4.23 release is here! Go check it out!

The following bugs were squashed:

• FS-8338 [core] Fix for ringback not working correctly on stereo channels
• FS-8354 [mod_conference] Reverted a back ported patch for rate change detection because it introduced a regression that caused an audio issue

We have an immediate WebRTC development contract opportunity that has just come up in the Seattle area. The contract requires 4-5 full-time developers onsite, remote will not fit the bill on this one.

For this contract we are looking for a team lead, 2 x Node.js, 2 x common JS developers

You have built commercial web applications using WebRTC libraries and are intimately familiar with the WebRTC and ORTC specs and respective libraries.

Start date: ASAP

If you are interested please forward your resume elagerway@gmail.com

Time for another WebRTC Basics: Video Codecs

I’ve been yapping about video codec more than once here on this blog. But what is it exactly?

If you’re a web developer and you are starting to use WebRTC, then there’s little reason (until now) for you to know about it. Consider this your primer to video coding.

Definition

A video codec takes the raw video stream, which can be of different resolution, color depth, frame rate, etc. – and compress it.

This compression can be lossless, where all data is maintained (so when you decompress it you get the exact same content), BUT it is almost always going to be lossy. The notion is that we can lose data that our human eye doesn’t notice anyway. So when we compress video, we take that into account, and throw stuff out relative to the quality we wish to get. The more we throw – the less quality we end up with.

The video codec comes in two pieces:

1. Encoder – takes the raw video data and compresses it
2. Decoder – takes the compressed data created by an encoder and decompresses it

The decoded stream will be different from the original one. It will be degraded in its quality.

The Decoder is the Spec

The thing many miss is that in order to define a video codec, the only thing we have is a specification for a decoder:

Given a compressed video stream, what actions need to take place to decompress it.

There is no encoder specification. It is assumed that if you know how the compressed result needs to look like, it is up to you to compress it as you see fit. Which brings us to the next point.

Generally speaking, decoders will differ from each other by their performance: how much CPU they take to run, how much memory they need, etc.

The Encoder is… Magic

Or more like a large set of heuristics.

In a video codec, you need to decide many things. How much time and effort to invest in motion estimation, how aggressive to be when compressing each part of the current frame, etc.

You can’t really get to the ultimate compression, as that would take too long a time to achieve. So you end up with a set of heuristics – some “guidelines” or “shortcuts” that your encoder is going to take when he compresses the video image.

Oftentimes, the encoder is based on experience, a lot of trial and error and tweaking done by the codec developers. The result is as much art as it is science.

Encoders will differ from each other not only by their performance but also by how well they end up compressing (and how well can’t be summed up in a single metric value).

Hardware Acceleration

A large piece of what a codec does is brute force.

As an example, most modern codecs today split an image into macroblocks, each requiring DCT. With well over 3,000 macroblocks in each frame of 720p resolution that’s a lot that need to get processed every second.

Same goes for motion estimation and other bits and pieces of the video codec.

To that end, many video codec implementations are hardware accelerated – either the codec runs completely by accelerated hardware, or the ugly pieces of it are, with “software” managing the larger picture of the codec implementation itself.

It is also why hardware support for a codec is critical for its market success and adoption.

Bandwidth Management

A video codec doesn’t work in a void. Especially not when the purpose of it all is to send the video over a network.

Networks have different characteristics of available bandwidth, packet loss, latency, jitter, etc.

When a video encoder is running, it has to take these things into account and compensate for them – reducing the bitrate it produces when there’s network congestion, reset its encoding and send a full frame instead of partial ones, etc.

There are also different implementations for a codec on how to “invest” its bitrate. Which again brings us to the next topic.

Different Implementations for Different Content Types (and use cases)

Not all video codec implementations are created equal. It is important to understand this when picking a codec to use.

When Google added VP9 to YouTube, it essentially made two compromises:

1. Having to implement only a decoder inside a browser
2. Stating the encoder runs offline and not in real-time

Real-tme encoding is hard. It means you can’t think twice on how to encode things. You can’t go back to fix things you’ve done. There’s just not enough time. So you use single-pass encoders. These encoders look at the incoming raw video stream only once and decide upon seeing a block of data how to compress it. They don’t have the option of waiting a few frames to decide how to compress best for example.

Your content is mostly static, coming from a Power Point presentation with mouse movements on top? That’s different from a head-shot video common in web meetings, which is in turn different than the latest James Bond Spectre trailer motion.

And in many ways – you pick your codec implementation based on the content type.

A Word about WebRTC

WebRTC brings with it a huge challenge to the browser vendors.

They need to create a codec that is smart enough to deal with all these different types of contents while running on variety of hardware types and configurations.

From what we’ve seen in the past several years – it does quite well (though there’s always room for improvement).

 

Next time you think why use WebRTC and not build on your own – someone implementing this video codec for you is one of the reasons.

 

Planning on introducing WebRTC to your existing service? Schedule your free strategy session with me now.

 

 

The post WebRTC Basics: What’s a Video Codec Anyway? appeared first on BlogGeek.me.

Chrome, Firefox, and Edge are all on the same party line. Image from Pillow Talk (1959)

For the first time, Chrome, Firefox and Edge can “talk” to each other via WebRTC and ORTC. Check the demo on Microsoft’s modern.ie testdrive.

tl;dr: don’t worry, audio works. codec interop issue…

Feature Interoperability Notes ICE yes Edge requires end-of-candidate signaling DTLS yes audio yes using G.722, Opus or G.711 codecs video no standard H.264 is not supported in Edge yet DataChannels no Edge does not support dataChannels

As a reader of this blog, you probably know what WebRTC is but let me quote this:

WebRTC is a new set of technologies that brings clear crisp voice, sharp high-definition (HD) video and low-delay communication to the web browser.

In order to succeed, a web-based communications platform needs to work across browsers. Thanks to the work and participation of the W3C and IETF communities in developing the platform, Chrome and Firefox can now communicate by using standard technologies such as the Opus and VP8 codecs for audio and video, DTLS-SRTP for encryption, and ICE for networking.

This description is taken from the early-2013 Chromium blog post that announced interoperability between Chrome and Firefox. And now Edge?

Codecs…

So we have interoperability – for audio calls.  It is just audio. No video interoperability yet. Now this is just an issue of all vendors implementing at least one common video codec:

• Edge currently implements a Microsoft variant of H264 called H264UC which adds some features like SVC
  • Adding H264 is work in progress
  • While there is a VP9 decoder for playing videos, that is not usable for ORTC so don’t get too excited
  • See Bernard’s comments for more information
• Chrome implements VP8; H264 is work in progress
• Firefox implements VP8 and H264

Audio interoperability is currently using G.722 instead of Opus because Edge still prefers Silk and G.722 over Opus.

APIs

But wait, how can those browsers talk if they do not agree on APIs?

Well, I implemented the PeerConnection API on top of ORTC. The gory details can be found here as part of a pull request for adapter.js. It has undergone a quite critical review and improved as a result of that. This process also showed some issues in the ORTC specification. While there has always been the assumption that it would be possible to implement the PeerConnection API using the lower-level ORTC API, nobody had actually done it.

The functionality provided is limited. More than a single audio and video track has not been tested and, since this is using an SDP similar to what is specified in the Unified Plan draft would likely not be interoperable with Chrome. But this is sufficient for quite a number of applications that are simple enough not to benefit from ORTC natively.

SDP!

Using this Javascript implementation, Edge will generate something that is close enough to the SDP used by the PeerConnection API:

v=0 o=thisisadapterortc 8169639915646943137 2 IN IP4 127.0.0.1 s=- t=0 0 m=audio 9 UDP/TLS/RTP/SAVPF 104 9 106 0 103 8 97 13 118 101 c=IN IP4 0.0.0.0 a=rtcp:9 IN IP4 0.0.0.0 a=rtpmap:104 SILK/16000 a=rtcp-fb:104 x-message app send:dsh recv:dsh a=rtpmap:9 G722/8000 a=rtcp-fb:9 x-message app send:dsh recv:dsh a=rtpmap:106 OPUS/48000/2 a=rtcp-fb:106 x-message app send:dsh recv:dsh a=rtpmap:0 PCMU/8000 a=rtcp-fb:0 x-message app send:dsh recv:dsh a=rtpmap:103 SILK/8000 a=rtcp-fb:103 x-message app send:dsh recv:dsh a=rtpmap:8 PCMA/8000 a=rtcp-fb:8 x-message app send:dsh recv:dsh a=rtpmap:97 RED/8000 a=rtpmap:13 CN/8000 a=rtpmap:118 CN/16000 a=rtpmap:101 telephone-event/8000 a=rtcp-mux a=ice-ufrag:lMRF a=ice-pwd:NR15fT4U6wHaOKa0ivn64MtQ a=setup:actpass a=fingerprint:sha-256 6A:D8:7D:05:1A:ED:DB:BD:6A:60:1A:BC:15:70:D1:6C:A1:D9:00:79:E5:5C:56:15:73:80:E2:82:9D:B9:FB:69 a=mid:nbiwo5l60z a=sendrecv a=msid:7E4272C7-2B6C-49BD-BF7A-A3E7B8DD44F5 D2945771-D7B4-4915-AC29-CEA9EC51EC9E a=ssrc:1001 msid:7E4272C7-2B6C-49BD-BF7A-A3E7B8DD44F5 D2945771-D7B4-4915-AC29-CEA9EC51EC9E a=ssrc:1001 cname:3s6hzpz1jj

Check the anatomy of a WebRTC SDP post to find out what each of these lines mean.

This allows quite a number of the WebRTC PeerConnection samples to work in Edge, just like many of the getUserMedia samples already work.

With that working, the next big challenge was browser interoperability. Would this underspecified blob of text be good enough to be accepted by Chrome and Firefox?

It turned out to be good enough. After adding ICE candidates on both sides the ice connection and DTLS states soon changed to completed and connected. Yay. In Chrome at least.
Firefox did not work because of trivial mistakes that took a while to figure out. But then, it just worked as well.

As far as I am concerned this shows the hard part, making ICE and DTLS interoperable, is solved. The rest is something for codec folks to work out. Not my area of interest

{“author”: “Philipp Hancke“}

Want to keep up on our latest posts? Please click here to subscribe to our mailing list if you have not already. We only email post updates. You can also follow us on twitter at @webrtcHacks for blog updates and news of technical WebRTC topics or our individual feeds @chadwallacehart, @victorpascual and @tsahil.

The post Hello Chrome and Firefox, this is Edge calling appeared first on webrtcHacks.

Here’s a thought. Why not get WebRTC to the operating system level and be done with it?

Today, there are different ways to get WebRTC going:

1. Use a browser…
2. Compile the code and link it to your own app (PC or mobile)
3. Wrap the browser within an app (PC)
4. Use a webview (Android)

That last option? This is the closest one to an OS level integration of WebRTC. You assume it is there and available, and use it in your app somehow.

But what if we could miraculously get the WebRTC APIs (Javascript or whatever) from he operation system itself? No compilation needed. No Cordova plugins to muck around with. Just good ol’ “system calls”?

While I don’t really expect this to happen, here’s what we’d gain from having that:

1# Smaller app sizes

Not needing to get WebRTC on a device means your app takes up less space. With the average app size on the increase, this is always a good thing.

The OpenH264 codec implementation binary alone is around 300k, depending on the platform. Assume you need 3-4 more codecs (and that number will be growing every couple of years), the other media algorithms, all the network implementation, code to integrate with device drivers, WebRTC specific wrappers, … – lots and lots of size.

And less app size means more space for other app and less app data to send over the network when intsalling the app.

2# Less variability

While the first one is obvious, and somewhat nagging – so it takes a second more to install an app – who cares?

This point has a lot more of a reason for it.

If there’s a single implementation of WebRTC, maintained by the OS itself, there’s a lot less hassle of dealing with the variance.

When people port WebRTC on their own and use it – they make changes and tweaks. They convince themselves (with or without any real reason) that they must make that small fix in that piece of algorithm in WebRTC – after all, they know their use case best.

But now, it is there, so you make do with what you have. And that piece of code gets updated magically and improves with time – you need not upgrade it manually and re-integrate all the changes you’ve made to it.

Less variability here is better.

3# Shorter TTM

Since you don’t need to muck around with the work of porting and integration – it takes less time to implement.

I’ve been working with many vendors on how to get WebRTC to work in their use case. Oftentimes, that requires that nasty app to get a WebRTC implementation into it. There’s no straightforward solution to it. Yes – it is getting easier with every passing day, but it is still work that needs to be done and taken into account.

Back to reality

This isn’t going to happen anytime soon.

Unless… it already has to some extent and in some operating systems.

Chrome is an OS – not only Chrome OS but Chrome itself. It has WebRTC built in – in newer Android versions as well, where you can open up webviews with it.

For the rest, it is unlikely to be the path this technology will be taking.

 

Planning on introducing WebRTC to your existing service? Schedule your free strategy session with me now.

The post 3 Advantages of WebRTC Embedded in the OS appeared first on BlogGeek.me.

Voxter - Building Value with Kazoo - KazooCon 2015 from 2600Hz

The Voxter team discuss code contributions to the Kazoo platform, discuss how they are utilizing the platform, and give an in-depth demo of WhApps

SIPLABS - Hard Rocking Kazoo - KazooCon 2015 from 2600Hz

Founder and CEO Mikhail Rodionov discusses all the projects and code contributions that they have built for Kazoo over the past year.

Telnexus - Quote to Cash – KazooCon 2015 from 2600Hz

Telnexus CEO Vernon Keenan discuss how he built the Managed Service Provider Telnexus from the ground up and the lessons he has learned in the process.

Least Cost Routing in the Cloud from 2600Hz

The ThinQ team discuss how to set up your routing profile, carrier selection, high volume traffic management, and LCR routing

VirtualPBX - Back Office, Delivering Voice in a Competitive Market - KazooCon 2015 from 2600Hz

In a competitive market, high quality voice services alone are rarely enough. Lon will speak about the customer lifecycle, back office systems from Sales to CRM to deployment, and how to drive profitable growth while delivering an excellent customer experience.

Something is brewing at Google.

Last week it was announced that Symphony just raised another $100M lead by Google. Not Google Ventures mind you – Google Inc.

Who is Symphony?

• High profile Silicon Valley startup (obviously), soon to become a unicorn, if it isn’t already
• Well known founder from the Unified Communications industry – David Gurle
• Have been around for only a year
• Already has over 100 employees, most of them engineers
• Focused on enterprise messaging, and targeting highly regulated and security sensitive industries

The Symphony Service

The service itself is targeted at the enterprise, but a free variant of it is available. I tried logging into it, to see what is all about. It is a variant of the usual messaging app on the desktop, with bits and pieces of Facebook and Slack.

On face value, not much different than many other services.

Symphony Foundation

Symphony decided to build its service on top of an open source platform of its own, which it calls Symphony Foundation. It includes all the relevant washed-out words required in a good marketing brochure, but little else for now: a mission statement, some set of values. That’s about it.

It will be open source, when the time comes. It will be licensed under the Apache license (permissive enough). And you can leave an inquiry on the site. In the name of openness… that’s as open as Apple’s FaceTime protocol is/was supposed to be. I’ll believe it when I see it.

Why Invest in Symphony?

This is the bigger question here. Both for why Google put money in it, as well as others.

With a total of $166M of investment in two rounds and over 100 employees recruited in its first year of existence, there seems to be a gold rush happening. One that is hard to explain.

As a glaring reminder – Whatsapp on acquisition day had 32 developers and around 50 employees. Symphony has twice that already, but no active user base to back it up.

It might be because of its high profile. After all, this is David Gurle we’re talking about. But then again, Talko has Ray Ozzie. But they only raised $4M in the past 3 years, and have less than 10 employees (if you believe LinkedIn).

The only other reason I can see is the niche they went for.

The financial industry deals with money, so it has money. It also has regulations and laws, making it a hard nut to crack. While most other players are focused on bringing consumer technology to the SMB, Symphony is trying to start from the top and trickle to the bottom with a solution.

The feature set they are putting in place, based on their website, include:

• Connectivity across organizations, while maintaining “organizational compliance”
• Security and privacy
• Policy control on the enterprise level
• Oh… and it’s a platform – with APIs – and developers and partners

The challenge will be keeping a simple interface while maintaining the complex feature set regulated industries need (especially ones that love customization and believe they are somehow special in how they work and communicate).

On Messaging and Regulation

The smartphone is now 8 years old, if you count it from the launch of the iPhone.

Much has changed in 8 years, and most of it is left unregulated still.

Messaging has moved from SMS to IP based messaging services like Whatsapp in many countries of the world. Businesses are trying to kill email with tools like Slack. We now face BYOD phenomena, where employees use whatever device and tools they see fit to get their work done – and enterprises find it hard to force them to use specific tools.

If Hillary Clinton can use her own private email server during the course of her workday, what should others say or do?

While regulation is slow to catch up, I think some believe the time is ripe for that to happen. And having a messaging system that is fit for duty in those industries that are sensitive today means being able to support future regulation in other/all industries later.

This trend might raise the urgency or the reason for the capital that Symphony has been able to attract.

Google

Why did Google invest here? Why not Google Ventures? It doesn’t look like an Alphabet investment but rather a Google one. And why invest and not acquire?

Google’s assets in messaging include today:

• Google Hangouts
• The recent acquisition of Jibe Mobile and their RCS chops
• The acquisition of Firebase, a messaging API platform that hasn’t changed much under Google as far as I can tell
• Gmail to some extent

Jibe/RCS is about consumer and an SMS replacement in the long run. It may be targeted at Apple. Or Facebook. Or Skype. Or all of them.

None of its current assets is making a huge impact. They aren’t dominant in their markets.

And messaging may be big in the consumer, but the money is in the enterprise – it can be connectivity to enterprises, ecommerce or pure service. Google is finding it difficult there as well.

Symphony is a different approach to the same problem. Targets the enterprise directly. Focusing on highly regulated customers. Putting money into it as an investment is a no-brainer, especially if it includes down the road rights of first refusal on an acquisition proposal for example. So Google sits and waits, sees what happens with this market, and decides how to continue.

Is this a part of a bigger picture? A bigger move of Google in the messaging space? Who knows? I still can’t figure out the motivation behind this one…

Messaging and me

I’ve been writing on general messaging topics on and off throughout the years on this blog.

It seems this space is becoming a lot more active recently.

Expect more articles here about this topic of messaging from various angles in the near future.

 

Planning on introducing WebRTC to your existing service? Schedule your free strategy session with me now.

The post Google Goes All in for Messaging, Invests in Symphony appeared first on BlogGeek.me.

Hello, again. This past week in the FreeSWITCH master branch we had 51 commits! There were some very important changes this week to the Debian packaging system. The default is now set to build packages with the upstream FS package repos. Since the system dependencies have been removed from the FS codebase the 1.6 branch is now required to use the FS public repo for dependencies.  The notable feature for this week is the addition of the variable media_mix_inbound_outbound_codecs, which mixes inbound and outbound codecs, and this is a behavior change.

Join us on Wednesdays at 12:00 CT for some more FreeSWITCH fun! And head over to freeswitch.com to learn more about FreeSWITCH support.

New features that were added:

• FS-8290 [verto_communicator] Automatically mark dedicated encoder if out/in bandwith isn’t set to ‘Server default’
• FS-8290 [verto_communicator] Adding help text on how to enable dedicated remote encoder
• FS-8321 [core] Add variable media_mix_inbound_outbound_codecs to mix inbound and outbound codecs. BEHAVIOR CHANGE

Improvements in build system, cross platform support, and packaging:

• FS-8316 [build][Debian] Fixed new build warning from latest clang and resolved the build warnings in the modules too
• FS-8271 [Debian] Adding some logging, and more cautious handling of spaces in parameters. Now the default will build packages with the upstream FS package repos. This is a change in the default behavior of the Debian packaging system with the justification that in 1.6 it is now required to use the FS public repo for dependencies because system dependencies have been removed from the FS codebase which used to be included. And defaulting to automatically download the binary dependencies because without major changes to package building in cowbuilder(which is the primary supported method of building FS packages), you can’t access the network to build the binary packages from the source package. If using system apt repo list, then include the supplementary ones too
• FS-8233 [automation] In order to clean up build dependencies for the automated tests, convert the tests/*/Makefile.am into an include file for the top level Makefile.am. This will greatly simplify dependency tracking, and allow tests to be rerun easily on FS source code changes.
• FS-7820 [automation] Use a more appropriate function for printing diagnostics

The following bugs were squashed:

• FS-8243 [mod_opus] Adding back the missing part removed in 8b088c2 so FEC works in most surroundings
• FS-8295 [mod_opus] FMTP fixes to continue the cleanup of FEC
• FS-8302 [mod_opus] Fix some printing/logging because switch_opus_show_audio_bandwidth() was not returning TRUE/FALSE as expected
• FS-8130 FS-8305 [mod_opus] Fix some warnings and errors caused by dtx and/or jittery webrtc, refactor of last patch, and add suppression of scary harmless message about opus FEC
• FS-8296 [mod_opus] Improve the way Opus is initialized when a call comes in
• FS-8179 [mod_opus] Fixed a regression setting fec_decode breaking output on stereo calls
• FS-8297 [mod_conference] A fix for auto STUN switching IPs quickly and WebRTC video not working
• FS-8130 [mod_conference] Fix for micro cut-offs and unstable voice issues
• FS-8317 [mod_conference] Fix for playing multiple files at once to stack them for immediate playback, sometimes breaking and the floor layer becoming unusable for the rest of the conference.
• FS-8067 [verto_communicator] When no email is present make sure mm is the default avatar in the circle this way the talk indicator works on PSTN and SIP callers.
• FS-8247 [verto_communicator] When websocket disconnects go to splash screen to wait for the reconnect
• FS-8300 [verto_communicator] Fixing reload bug so reloading twice is no longer needed
• FS-8315 [core] Fix for rtp_media_timeout not working
• FS-8304 [core] Fix for choppy audio during calls
• FS-8320 [core] Fixed broken ZRTP not responding to HELLO packet
• FS-8311 [mod_voicemail] Fix for leave-message event not containing verbose data for a forwarded voicemail
• FS-8318 [mod_av] Fix for recording being out of sync when video from chrome has packet loss
• FS-7929 [mod_sofia] Fixed an issue when processing SIP messages while using camp-on
• FS-6833 [mod_sofia] Add content-type header to ack with sdp

 

And, this past week in the FreeSWITCH 1.4 branch we had 3 new commits merged in from master. And the FreeSWITCH 1.4.23 release is here! Go check it out!

The following bugs were squashed:

• FS-8246 [mod_json_cdr] Use seconds as default value for delay parameter
• FS-8282 [core] Fix for sleep is not allowing interruption by uuid_transfer
• FS-8166 [core] Mute/unmute while shout is playing audio fails because the channel “has a media bug, hard mute not allowed”

 

We have been waiting a long time for Microsoft to add WebRTC to its browser portfolio. That day finally came last month when Microsoft announced its new Windows 10 Edge browser had ORTC. This certainly does not immediately address the Internet Explorer population and ORTC is still new to many (which is why we cover it often). On the positive side, interoperability between Edge, Chrome, and Firefox on the audio side was proven within days by multiple parties. Much of ORTC is finding its way into the WebRTC 1.0 specification and browser implementations.

I was with Bernard Aboba, Microsoft’s WebRTC lead at the IIT Real Time Communications Conference (IIT-RTC) and asked him for an interview to cover the Edge implementation and where Microsoft is headed. The conversation below has been edited for readability and technical accuracy. The full, unedited audio recording is also available below if you would rather listen than read. Warning – we recorded our casual conversation in an open room off my notebook microphone, so please do not expect high production value.

https://webrtchacks.com/wp-content/uploads/2015/10/Bernard-Aboba-QA.mp3

We cover what exactly is in Edge ORTC implementation, why ORTC in the first place, the roadmap, and much more.

You can view the IIT-RTC ORTC Update presentation slides given by Bernard, Robin Raymond of Hookflash, and Peter Thatcher of Google here.

{“editor”, “chad hart“}

Micosoft’s Edge is hungry for WebRTC

Intro to Bernard

webrtcHacks: Hi Bernard. To start out, can you please describe your role at Microsoft and the projects you’ve been working on? Can you give a little bit of background about your long time involvement in WebRTC Standards, ORTC, and also your new W3C responsibilities?

Bernard: I’m a Principal Architect at Skype within Microsoft, and I work on the Edge ORTC project primarily, but also help out other groups within the company that are interested in WebRTC. I have been involved in ORTC since the very beginning as one of the co-authors of ORTC, and very recently, signed up as an Editor of WebRTC 1.0.

webrtcHacks:  That’s concurrent with some of the agreement around merging more of ORTC into WebRTC going forward. Is that accurate?

Bernard: One of the reasons I signed up was that I found that I was having to file WebRTC 1.0 API issues and follow them. Because many of the remaining bugs in ORTC related to WebRTC 1.0, and of course we wanted the object models to be synced between WebRTC 1.0 and ORTC, I had to review pull requests for WebRTC 1.0  anyway, and reflect the changes within ORTC.  Since I had to be aware of WebRTC 1.0 Issues and Pull Requests to manage the ORTC and Pull Requests, I might as well be an editor of WebRTC 1.0.

Bernard Aboba of Microsoft and Robin Raymond of Hookflash discussing ORTC at the IIT Real Time Communications Conference (IIT-RTC)

What’s in Edge

webrtcHacks:  Then I guess we’ll move on to Edge then. Edge and Edge Preview are out there with varying forms of WebRTC. Can you walk through a little bit of that?

Bernard: Just also to clarify for people, Edge ORTC is in what’s called Windows Insider Preview.  Windows Insider Preview builds are only available to people who specifically sign up to receive them.  If you sign up for the Windows Insider Preview program and install the most recent build 10547, then you will have access to the ORTC API in Edge. In terms of what is in it, the audio is relatively complete. We have:

• G.711,
• G.722,
• Opus,
• Comfort Noise,
• DTMF, as well as the
• SILK codec.

Then on the video side, we have an implementation of H.264/SVC, which does both simulcast and scalable video coding, and as well as forward error correction (FEC), known as H.264UC. I should also mention, we support RED and forward error correction for audio as well. 

That’s what’s you will find in the Edge ORTC API within Windows Insider Preview, as well as support for “half-trickle” ICE, DTLS 1.0, etc.

webrtcHacks: I’ll include the slide from your presentation for everyone to reference because there’s a lot of stuff to go through. I do have a couple of questions on a few things for follow up. One was support on the video side of things for. I think you mentioned external FEC and also talked about other aspects of robustness, such as retransmission?

Bernard’s slide from IIT-RTC 2015 showing Edge’s ORTC coverage

Bernard: Currently in Edge ORTC Insider Preview, we do not support generic NACK or re-transmission.  We do support external forward error correction (FEC), both for audio and video.   Within Opus as well as SILK we do not support internal FEC, but you can configure RED with FEC externally.  Also, we do not support internal Discontinuous Operation (DTX) within Opus or SILK, but you can configure Comfort Noise (CN) for use with audio codec, including Opus and SILK.

Video interoperability

webrtcHacks: Then could you explain H.264 UC? The majority of the people out there that aren’t familiar with the old Lync or Skype for Business as it is now called.

Bernard: Basically, H.264 UC supports spatial simulcast along with temporal scalability in H.264/SVC, handled automatically “under the covers”.  These are basically the same technologies that are in Hangouts with VP8.   While the ORTC API offers detailed control of things like simulcast and SVC, in many cases, the developer just basically wants the stack to do the right thing, such as figuring out how many layers it can send. That’s what H.264UC does.  It can adapt to network conditions by dropping or adding simulcast streams or temporal layers, based on the bandwidth it feels is available. Currently, the H.264UC codec is only supported by Edge.

webrtcHacks:  Is the base layer H.264?

Bernard: Yes, the base layer is H.264 but RFC 6190 specifies additional NAL Unit types for SVC, so that an implementation that only understands the base layer would not be able to understand extension layers.  Also, our implementation of RFC 6190 sends layers using distinct SSRCs, which is known as Multiple RTP stream Single Transport (MRST).  In contrast, VP8 uses Single RTP stream Single Transport (SRST).

We are going to work on an implementation of H.264/AVC in order to interoperate.  As specified in RFC 6184 and RFC 6190, H.264/AVC and H.264/SVC have different codec names.

webrtcHacks:  For Skype, at least, in the architecture that was published, they showed a gateway. Would you expect other people to do similar gateways?

Bernard: Once we support H.264/AVC, developers should be able to configure that codec, and use it to communicate with other browsers supporting H.264/AVC.  That would be the preferred way to interoperate peer-to-peer.  There might be some conferencing scenarios where it might make sense to configure H.264UC and have the SFU or mixer strip off layers to speak to H.264/AVC-only browsers, but that would require a centralized conferencing server or media relay that could handle that. 

Roadmap

webrtcHacks:  What can you can you say about the future roadmap? Is it basically what’s on the dev.modern.ie page?

Bernard: In general, people should look at the dev.modern.ie web page for status, because that has the most up to date. In fact, I often learn about things from the page. As I mentioned, the Screen Sharing and Media Recorder specifications are now under consideration, along with features that are in preview or are under development.  The website breaks down each feature.  If the feature is in Preview, then you can get access to it via the Windows Insider Preview.  If it is under development, this means that it is not yet in Preview.  Features that are supported have already been released, so if you have Windows 10, you should already have access to them. 

Slide from Bernard’s IIT-RTC 2015 presentation covering What’s in Edge

In terms of our roadmap, we made a roadmap announcement in October 2014 and are still executing on things such as H.264, which we have not delivered yet.  Supporting interoperable H.264 is about more than just providing an encoder/decoder, which we have already delivered as part of H.264UC.  The IETF RTCWEB Video specification provides guidance on what is needed to provide interoperable H.264/AVC, but that is not all that a developer needs to implement – there are aspects that are not yet specified, such as bandwidth estimation and congestion control.

Beyond the codec bitstream, RTP transport and congestion control there are other aspects as well.  For example, I mentioned robustness features such as Forward Error Correction and Retransmission.   A Flexible FEC draft is under development in IETF which will handle burst loss (distances greater than one).  That is important for robust operation on wireless networks, for both audio and video.  Today we have internal FEC within Opus, but that does not handle burst loss well.

webrtcHacks: Do you see Edge pushing the boundaries in this area? 

Bernard: One of the areas where Edge ORTC has advanced the state of the art is in external forward error (FEC) correction as well as in statistics.  Enabling external FEC to handle burst loss, provides additional robustness for both audio and video.  We also support additional statistics which provide information on burst loss and FEC operation.  What we have found is that burst loss is a fact of life on wireless networks, so that being able to measure this and to address it is important. The end result of this work is that Edge should be more robust than existing implementations with respect to burst loss (at least with larger RTTs where retransmission would not be available).  We can also provide burst loss metrics, which other implementations cannot currently do.  I should also mention that there are metrics have been developed in the XRBLOCK WG to address issues of burst loss, concealment, error correction, etc.

Why ORTC?

webrtcHacks:  You have been a long time advocate for ORTC. Maybe you can summarize why ORTC was a good fit for Edge? Why did you start with that spec versus something else? What does it enable you to do now as a result?

Bernard: Some of the advantages of ORTC were indeed advantages, but in implementation we found there were also other advantages we didn’t think of at the time.

Interoperability

Bernard: ORTC doesn’t have SDP [like WebRTC 1.0]; the irony is ORTC allowed us to get to WebRTC 1.0 compatibility and interoperability faster than we would have otherwise. If you look at the adapter.js, it’s actually interesting to read that code- the actual code for Edge is actually smaller than for some of the other browsers. One might think that’s weird – why would it take less adaptation for Edge than for anything else? Are we really more 1.0 compatible than 1.0? The answer is, to some respects, we are, because we don’t generate SDP than somebody needs to parse and reformat. It certainly saves a lot of development to not have to write that code and have control in JavaScript, and also be easy to modify in case people find bugs in it.

The irony is ORTC allowed us to get to WebRTC 1.0 compatibility and interoperability faster than we would have otherwise

Connection State Details

The other thing we found about ORTC that we didn’t quite understand early on was it gives you detailed status of each of the transports- each of your ICE transports. Particularly when you’re dealing with situations like multiple interfaces, you actually get information about failure conditions that you don’t get out of WebRTC 1.0. 

It’s interesting to look at 1.0 – one of the reasons that I think people will find the objects interesting in 1.0 is because you actually need that kind of diagnostic information. The current connection state [in the current WebRTC] is not really enough – it’s not even clear what it means. It says in the spec that it’s about ICE, but it really combines ICE and DTLS. With the object model, you know exactly what ICE transport went down or if DTLS is in some weird state. Actually for diagnostics, details of the connection state is actually pretty important. It’s one of the most frequently requested statistical things. That was a benefit we didn’t anticipate, that we found is pretty valuable and will be coming into 1.0.

Many simple scenarios

Bernard: Then there were the simple scenarios. Everyone said, “I don’t need ORTC because I don’t do scalable video coding and simulcast” Do you ever do hold? Do you ever do changing owners of codecs? All illustrations that Peter [Thatcher] showed in his WebRTC 1.0 presentation. The answer is, a lot of those things are, in fact, common, and were not possible in 1.0. There is a lot of fairly basic benefits that you get as well. 

How is Edge’s Media Engine built

webrtcHacks:  In building and putting this in the Edge, you had a few different media engines you could choose from. You had the Skype media engine and a Lync media – you combine them or go and build a new one. Can you reveal the Edge media architecture and how you put that together?

Bernard: What we chose to do in Skype is move to a unified media engine. What we’ve done is, we’ve added WebRTC capabilities into that media engine. That’s a good thing because, for example, things like RTCP MUX and things like BUNDLE are now part of the Skype media engine so we can use them. The idea was to produce something that was unified and would have all the capabilities in one. It took a little bit longer to do it that way, but the benefit is that we get to produce a standardized compliant browser and we also get to use those technologies internally. Now we do not have 3 or 4 different stacks that we would have to rationalize later.

right now, our focus is very much on video, and trying to get that more solid, and more interoperable

Also, I should mention that one thing that is interesting about the way we work is we produce stacks that are both client and server capable. We don’t just produce pure client code that wouldn’t, for example, be able to handle load. Some of those things can go into back-end components as well. That is also true for DTLS and all that. Whether or not we use all those things in Skype is another issue, but it is part of the repertoire for apps. 

More than Edge

webrtcHacks: Is there anything else that’s not on dev.modern.ie that is exposed that a developer would care about? Any NuGet packages with these API’s for example?

Bernard: There is a couple of things. dev.modern.ie does not cover non-browser things in Windows platform. For example, currently we support DTLS 1.0. We do want to support 1.2, because there’s additional cipher  suites that are important. For example, the Elliptic Curve stuff we’re seeing going into all the browsers. I think Mozilla already has it, or Chrome has it, or if they don’t, they will very soon. That is actually very important. Elliptic Curve turned out to be more than just a cipher suite issue – the time and effort it takes to generate more secure certificates is large. For RSA-2048 you can actually block the UI thread if you thread the object. Anyway, those are very important things that we don’t cover on dev.modern.ie, but those are the things we obviously have to do. 

There’s a lot of work and a lot of thinking that’s been going on in the IETF if relating to ICE and how to be better for mobile scenarios. Some of that I don’t think is converged yet, but there’s a new ICE working group. Some of that is in the ortc-lib implementation yet. Robin [Raymond] likes to be on the cutting edge so he has done basically the first implementation of a lot of those new technologies. That’s something, I think is of general interest – particularly as ORTC moves to mobile.

I should mention, by the way, that the Edge Insider Preview was only for desktop. It does not run on Windows Phone just to clarify that. 

webrtcHacks:  Any plans for embedding the Edge ORTC engine as a IE plugin?

Bernard: An external plugin or something?

webrtcHacks:  Yeah, or a Microsoft plugin for IE that would implement ORTC. 

Bernard: Basically at this point, IE is frozen technology. All the new features, if you look on the website, they all go into Edge. That’s what we’ve been developing for. I never say Microsoft will never do anything, but currently that’s not the thinking. Windows 10 for consumers is a free upgrade. Hopefully, people will take advantage of that and get all the new stuff, including Edge.

Is there an @MSEdgeDev post on the relationship between this and InPrivate? pic.twitter.com/bbu0Mdz0Yd

— Eric Lawrence (@ericlaw) September 22, 2015

A setting discovered in Internet Explorer that appears to address the IP Address Leakage issue. Validating ORTC

webrtcHacks:  Is there anything you want to share?

Bernard: I do want to clarify a little bit, I think adapter.js is a very important thing because it validates our original idea that essentially WebRTC 1.0 could be built into the JavaScript layer with ORTC. 

webrtcHacks:  And that happened pretty quick – with Fippo‘s help. Really quick. 

Bernard: Fippo has written all the pull requests. We’re paying a lot of attention to the bugs he’s finding. Obviously, he’s finding bugs in Edge, which hopefully we’ll fix, but he’s also finding spec bugs. It really helps make sure that this compatibility that we’ve promised is actually real. It’s a very interesting process to actually reduce that to code so that it’s not just a vague promise. It has to be demonstrated in software.

Of course what we’ve done is currently with audio. We know that video is more complicated, particularly as you start adding lots and lots of codecs to get that level of compatibility. I wouldn’t say that when Fippo is down with audio that it will be the last word. I think we’ll have to  pay even more attention to interoperability stuff in the video cases. It will be interesting because video is a lot more complicated. 

adapter.js is a very important thing because it validates our original idea that essentially WebRTC 1.0 could be built into the JavaScript layer with ORTC.

What does the Microsoft WebRTC team look like

webrtcHacks:  Can you comment on how big the time is that’s working on ORTC in Edge? You have a lot of moving pieces in different aspects … 

Bernard: There’s the people in Edge. There’s the people in Skype. In the Windows system there’s the people on the S-channel team that worked on the DTLS. There’s people all over – for example, the VP9 work that we talked about, was not done by either Skype or the conventional Edge people. It’s the whole Windows Media team. I don’t really know how to get my hands around this, because if you look at all the code we’re using, it’s written by probably, I don’t know, hundreds and hundreds of people. 

webrtcHacks:  And you need to pull it together for purposes of WebRTC/ORTC, is that right?

Bernard: Yeah. We have to pull it together, but there’s a lot there. There’s a lot of teams. There will probably be more teams going forward. People say, “Why don’t you have the datachannel”? The dataChannel isn’t something that would be in Skype’s specific area of expertise. That’s a transfer protocol, it should be really written by people who are experts in transfer protocol, which isn’t either Edge or Skype. It’s not some decision that was made by either of our groups not to do it. We have to find somebody who proves that they can do that work, to take ownership of that. 

Feedback please

webrtcHacks:  Any final comments?

Bernard: No. I just encourage people to download the preview, run it, file bugs, and let us know what you think. You can actually can vote on the website for new features, which is cool. 

We do listen to the input. WebRTC is an expanding thing. There’s a ton of things you can do – there’s all that stuff on dev.modern.ie site and then there’s internal improvement. Getting a sense of priority is what’s most important to people, is not that easy, because there’s so much that you could possibly focus on. I’d say right now, our focus is very much on video, and trying to get that more solid, and more interoperable, at least for the moment. We can walk and chew gum at the same time. We can do more than just one thing. Conceivably, especially when you look at IE and other teams. 

webrtcHacks:  This is great and very insightful. I think it will be a big help to all the developers out there. Thanks!

{
  “Q&A”:{
    “interviewer”:“chad hart“,
    “interviewee”:“Bernard Aboba“
  }
}

Want to keep up on our latest posts? Please click here to subscribe to our mailing list if you have not already. We only email post updates. You can also follow us on twitter at @webrtcHacks for blog updates and news of technical WebRTC topics or our individual feeds @chadwallacehart, @victorpascual and @tsahil.

The post Microsoft’s ORTC Edge for WebRTC – Q&A with Bernard Aboba appeared first on webrtcHacks.

Yes.

It is a question I get from time to time, especially now, that I am a few months into the WebRTC testing venture as a co-founder with a few partners – testRTC.

The logic usually goes like this: the browsers already support WebRTC. They do their own testing, so what we end up getting is a solid solution we can use. Fippo would say that

If life was that easy… here are a few things you need to take care of when it comes to testing the most simple of WebRTC services:

#1 – Future proofing browser versions

Guess what? Things break. They also change. Especially when it comes to WebRTC.

A few interesting tidbits for you:

• Google is dropping HTTP support for GetUserMedia, so services must migrate to HTTPS. Before year end
• The echo canceller inside WebRTC? It was rewritten. From scratch. Using a new algorithm. That is now running on a billion devices. Different devices. And it works! Most times
• WebRTC’s getStats() API is changing. Breaking its previous functionality

And the list goes on.

WebRTC is a great technology, but browsers are running at breakneck speeds of 6-8 weeks between releases (for each browser) – and every new release with a potential to break a service in multitude of ways – either because of a change in the spec, deprecation of capability or just bugs.

Takeaway: Make sure your service works not only on the stable version of the browsers, but also on their beta or even dev versions as well.

#2 – Media relay

Your service might be a P2P service, but at times, you will need to relay media through TURN servers.

The word on the street is that around 15% of sessions require relay. To some it can be 50% and to others 8% (real numbers I heard from running services).

Media relay is tricky:

• You need to configure it properly (many fall at this one)
• You need to test it in front of different firewall and NAT configurations
• You need to make it close to your users (you don’t want a local session in Paris to get relayed through a server in San Francisco)
• You need to test it for scale (check the next point for more on that)

Takeaway: Don’t treat WebRTC as a browser side technology only, or something devoid of media handling. Even if the browser does most of the heavy lifting, some of the effort (and responsibility) will lie on your service.

#3 – Server scale

Can your server cater for 200 sessions in parallel to fit that contact center? What about a 1000?

What will happen if you’ll have a horde effect due to a specific event? Can you handle that number of browsers hitting your service at once? Does your website operate in the same efficiency for the 1000th person as it does for the first?

This relates to both your signaling server, which is no part of WebRTC, but is there as part of your service AND your media server from my previous point.

Takeaway: Make sure your service scales to the capacities that it needs to scale. Oh – and you won’t be able to test it manually with the people you have with you in your office…

#4 – Service uptime

You tested it all. You have the perfect release. The service is up and running.

How do you make sure it stays running?

Manually? Every morning come in to the office and run a session?

Use Pingdom to make sure your site is up? Go to the extreme of using New Relic to check the servers are up, the CPUs aren’t over loaded and the memory use seems reasonable? Great. But does that mean your service is running and people can actually connect sessions? Not necessarily.

Takeaway: End-to-end monitoring. Make sure your service works as advertised.

The ugly truth about testing

The current norm in many cases is to test manually. Or not test at all. Or rely on unit testing done by developers.

None of this can work if what you are trying to do is create a commercial service, so take it seriously. Make testing a part of your development and deployment process.

And while we’re at it…

Check us out at testRTC

If you don’t know, I am a co-founder with a few colleagues at a company called testRTC. It can help you with all of the above – and more.

Leave us a note on the contact page there if you are interested in our paid service – it can cater to your testing needs with WebRTC as well as offering end-to-end monitoring.

 

Need to test WebRTC?

 

The post Do you Need to test a WebRTC P2P Service? appeared first on BlogGeek.me.

isVisible=false; function show_hide_searchbox(w){ if(isVisible){ document.getElementById('filterBoxSelection').style.display = 'none'; w.innerText='Filter ▼'; }else{ document.getElementById('filterBoxSelection').style.display = 'block'; w.innerText='Filter ▲'; } isVisible=!isVisible; } function checkIfSelected(chk){ if(chk.checked==1) chk.parentNode.className = "selected"; else chk.parentNode.className = "notselected"; getSelectedValues(); } function getSelectedValues(){ var a=document.getElementsByClassName('selected'); var vtVal=[] , ctVal=[] , ftVal=[]; var ct=0,vt=0,ft=0; for (x = 0; x < a.length; ++x) { try{ if(a[x].getElementsByTagName('input')[0].className=='companyType'){ ctVal[ct]= a[x].getElementsByTagName('input')[0].value; ct++; } if(a[x].getElementsByTagName('input')[0].className=='vendorType'){ vtVal[vt]= a[x].getElementsByTagName('input')[0].value; vt++; } if(a[x].getElementsByTagName('input')[0].className=='focusType'){ ftVal[ft]= a[x].getElementsByTagName('input')[0].value; ft++; } }catch(err){ } } search_VType(vtVal); search_CType(ctVal); search_FType(ftVal); } function search_VType(val){ var a=document.getElementsByClassName('interview-block'); for(x=0;x=0 && val[i]!=null){ a[x].style.display='block'; } } if(val.length==0){ a[x].style.display='block'; } } } function search_CType(val){ var a=document.getElementsByClassName('interview-block'); for(x=0;x=0 && val[i]!=null && a[x].style.display=='block'){ break; } if(i==val.length-1){ a[x].style.display='none'; } } } } function search_FType(val){ var a=document.getElementsByClassName('interview-block'); for(x=0;x=0 && val[i]!=null && a[x].style.display=='block'){ break; } if(i==val.length-1){ a[x].style.display='none'; } } } } Check out all webRTC interviews >>

Fone.Do: Moshe Maeir

October 2015

SMB phone system

Telephony

Small

Voice

Disrupting the hosted PBX system with WebRTC.

[If you are new around here, then you should know I’ve been writing about WebRTC lately. You can skim through the WebRTC post series or just read what WebRTC is all about.]

 

There’s no doubt that WebRTC is disrupting many industries. One of the obvious ones is enterprise communications, and in this space, an area that has got little attention on my end (sorry) is the SMB – where a small company needs a phone system to use and wants to look big while at it.

Moshe Maeir, Founder at Fone.do, just launched the service out of Alpha. I have been aware of what they were doing for quite some time and Moshe took the time now that their service is public to answer a few of my questions.

 

What is Fone.do all about?

Fone.do is a WebRTC based phone system for small businesses that anyone can set up in 3 minutes. It replaces both legacy PBX systems that were traditionally based in your communications closet and also popular Hosted PBX systems. Businesses today are mobile and the traditional fixed office model is changing. So while you can connect a SIP based IP phone to our system, we are focused on meeting the needs of the changing business world.

 

Why do small businesses need WebRTC at all? What’s the benefit for them?

You could ask the same question about email, social networks etc. Why use web based services at all? Does anyone want to go back to the days of “computer programs” that you downloaded and installed on your computer? Unfortunately, many still see telephony and communications as a stand alone application. WebRTC changes this. Small businesses can communicate from any place and any device as long as they have a compatible platform.

 

What excites you about working in WebRTC?

Two things. Not sure which is more exciting. First of all. If I build something great – the whole world is my potential market. All they need is a browser and they are using our system in 3 minutes. The other exciting aspect is that telephony is no longer a closed network. Once you are on the web the potential is unlimited. You can easily connect your phone system to the wealth of data and services that already exist on the web and take communications to a new level. In fact, that is why we hired developers who knew nothing about telephony but were experienced in web development. The results are eye opening for traditional telecom people.

 

I know you’re a telecom guy yourself. Can you give an example how working with web developers was an eye opener to you?

There are many. The general attitude is just do it. With legacy telecom, everything has the accepted way of doing things and you don’t want to try  anything new without extended testing procedures. A small example – in the old VoIP days writing a “dial plan” was a big thing. When we came to this issue on Fone.Do, one of the programmers naturally googled the issue and found a Google service that will automatically adapt the dial plan based on the users’ mobile number. 1-2-3 done.

 

Backend. What technologies and architecture are you using there?

Our main objective was to build an architecture that will work well and easily scale in the cloud (we are currently using AWS). So while we have integrated components such as the Dialogic XMS and the open source Restcomm, we wrote our own app server which manages everything. This enables us if we need to freely change back end components.

 

Can you tell us a bit about your team? When we talked about it a little of a year ago ago, I suggested a mixture of VoIP and web developers. What did you end up doing and how did it play out?

All our developers are experienced front end and backend web programmers with no telecom experience. However, our CTO who designed the system has over 15 years of experience in Telecom, so he is there to fill in any missing pieces. There were some bumps at the beginning, but I am very happy we did it this way. You can teach a web guy about Telephony, but it is very hard to get a Telecom guy to change his way of thinking. Telecom is all about “five nines” and minimizing risk. Web development is more about innovation and new functionality. With todays’ technology it is possible to innovate and be almost as reliable as traditional telephony

 

Where do you see WebRTC going in 2-5 years?

Adoption is slower than I expected, but eventually I see it as just another group of functions in your browser that developers can access as needed.

 

If you had one piece of advice for those thinking of adopting WebRTC, what would it be?

WebRTC is here. It makes your user experience better – so what are you waiting for?

 

What’s next for Fone.do?

We recently released our alpha product and we are looking to launch an open beta in the next couple of months. Besides a web based “application”, we also have applications for Android and iOS.

–

The interviews are intended to give different viewpoints than my own – you can read more WebRTC interviews.

The post Fone.do and WebRTC: An Interview With Moshe Maeir appeared first on BlogGeek.me.

The Next Wave - KazooCon 2015 from James Solada

CTO Karl Anderson discusses the state of Kazoo. This includes integrations with FreeSWITCH, erlang, and Kamailio. Reseller milestones include the release of whitelabeling, webhooks, migration, carriers, debugging, account management and more.