Time for another WebRTC Basics: Video Codecs
I’ve been yapping about video codec more than once here on this blog. But what is it exactly?
If you’re a web developer and you are starting to use WebRTC, then there’s little reason (until now) for you to know about it. Consider this your primer to video coding.
DefinitionA video codec takes the raw video stream, which can be of different resolution, color depth, frame rate, etc. – and compress it.
This compression can be lossless, where all data is maintained (so when you decompress it you get the exact same content), BUT it is almost always going to be lossy. The notion is that we can lose data that our human eye doesn’t notice anyway. So when we compress video, we take that into account, and throw stuff out relative to the quality we wish to get. The more we throw – the less quality we end up with.
The video codec comes in two pieces:
The decoded stream will be different from the original one. It will be degraded in its quality.
The Decoder is the SpecThe thing many miss is that in order to define a video codec, the only thing we have is a specification for a decoder:
Given a compressed video stream, what actions need to take place to decompress it.
There is no encoder specification. It is assumed that if you know how the compressed result needs to look like, it is up to you to compress it as you see fit. Which brings us to the next point.
Generally speaking, decoders will differ from each other by their performance: how much CPU they take to run, how much memory they need, etc.
The Encoder is… MagicOr more like a large set of heuristics.
In a video codec, you need to decide many things. How much time and effort to invest in motion estimation, how aggressive to be when compressing each part of the current frame, etc.
You can’t really get to the ultimate compression, as that would take too long a time to achieve. So you end up with a set of heuristics – some “guidelines” or “shortcuts” that your encoder is going to take when he compresses the video image.
Oftentimes, the encoder is based on experience, a lot of trial and error and tweaking done by the codec developers. The result is as much art as it is science.
Encoders will differ from each other not only by their performance but also by how well they end up compressing (and how well can’t be summed up in a single metric value).
Hardware AccelerationA large piece of what a codec does is brute force.
As an example, most modern codecs today split an image into macroblocks, each requiring DCT. With well over 3,000 macroblocks in each frame of 720p resolution that’s a lot that need to get processed every second.
Same goes for motion estimation and other bits and pieces of the video codec.
To that end, many video codec implementations are hardware accelerated – either the codec runs completely by accelerated hardware, or the ugly pieces of it are, with “software” managing the larger picture of the codec implementation itself.
It is also why hardware support for a codec is critical for its market success and adoption.
Bandwidth ManagementA video codec doesn’t work in a void. Especially not when the purpose of it all is to send the video over a network.
Networks have different characteristics of available bandwidth, packet loss, latency, jitter, etc.
When a video encoder is running, it has to take these things into account and compensate for them – reducing the bitrate it produces when there’s network congestion, reset its encoding and send a full frame instead of partial ones, etc.
There are also different implementations for a codec on how to “invest” its bitrate. Which again brings us to the next topic.
Different Implementations for Different Content Types (and use cases)Not all video codec implementations are created equal. It is important to understand this when picking a codec to use.
When Google added VP9 to YouTube, it essentially made two compromises:
Real-tme encoding is hard. It means you can’t think twice on how to encode things. You can’t go back to fix things you’ve done. There’s just not enough time. So you use single-pass encoders. These encoders look at the incoming raw video stream only once and decide upon seeing a block of data how to compress it. They don’t have the option of waiting a few frames to decide how to compress best for example.
Your content is mostly static, coming from a Power Point presentation with mouse movements on top? That’s different from a head-shot video common in web meetings, which is in turn different than the latest James Bond Spectre trailer motion.
And in many ways – you pick your codec implementation based on the content type.
A Word about WebRTCWebRTC brings with it a huge challenge to the browser vendors.
They need to create a codec that is smart enough to deal with all these different types of contents while running on variety of hardware types and configurations.
From what we’ve seen in the past several years – it does quite well (though there’s always room for improvement).
Next time you think why use WebRTC and not build on your own – someone implementing this video codec for you is one of the reasons.
Planning on introducing WebRTC to your existing service? Schedule your free strategy session with me now.
The post WebRTC Basics: What’s a Video Codec Anyway? appeared first on BlogGeek.me.
For the first time, Chrome, Firefox and Edge can “talk” to each other via WebRTC and ORTC. Check the demo on Microsoft’s modern.ie testdrive.
tl;dr: don’t worry, audio works. codec interop issue…
Feature Interoperability Notes ICE yes Edge requires end-of-candidate signaling DTLS yes audio yes using G.722, Opus or G.711 codecs video no standard H.264 is not supported in Edge yet DataChannels no Edge does not support dataChannelsAs a reader of this blog, you probably know what WebRTC is but let me quote this:
WebRTC is a new set of technologies that brings clear crisp voice, sharp high-definition (HD) video and low-delay communication to the web browser.
In order to succeed, a web-based communications platform needs to work across browsers. Thanks to the work and participation of the W3C and IETF communities in developing the platform, Chrome and Firefox can now communicate by using standard technologies such as the Opus and VP8 codecs for audio and video, DTLS-SRTP for encryption, and ICE for networking.
This description is taken from the early-2013 Chromium blog post that announced interoperability between Chrome and Firefox. And now Edge?
Codecs…So we have interoperability – for audio calls. It is just audio. No video interoperability yet. Now this is just an issue of all vendors implementing at least one common video codec:
Audio interoperability is currently using G.722 instead of Opus because Edge still prefers Silk and G.722 over Opus.
APIsBut wait, how can those browsers talk if they do not agree on APIs?
Well, I implemented the PeerConnection API on top of ORTC. The gory details can be found here as part of a pull request for adapter.js. It has undergone a quite critical review and improved as a result of that. This process also showed some issues in the ORTC specification. While there has always been the assumption that it would be possible to implement the PeerConnection API using the lower-level ORTC API, nobody had actually done it.
The functionality provided is limited. More than a single audio and video track has not been tested and, since this is using an SDP similar to what is specified in the Unified Plan draft would likely not be interoperable with Chrome. But this is sufficient for quite a number of applications that are simple enough not to benefit from ORTC natively.
SDP!Using this Javascript implementation, Edge will generate something that is close enough to the SDP used by the PeerConnection API:
v=0 o=thisisadapterortc 8169639915646943137 2 IN IP4 127.0.0.1 s=- t=0 0 m=audio 9 UDP/TLS/RTP/SAVPF 104 9 106 0 103 8 97 13 118 101 c=IN IP4 0.0.0.0 a=rtcp:9 IN IP4 0.0.0.0 a=rtpmap:104 SILK/16000 a=rtcp-fb:104 x-message app send:dsh recv:dsh a=rtpmap:9 G722/8000 a=rtcp-fb:9 x-message app send:dsh recv:dsh a=rtpmap:106 OPUS/48000/2 a=rtcp-fb:106 x-message app send:dsh recv:dsh a=rtpmap:0 PCMU/8000 a=rtcp-fb:0 x-message app send:dsh recv:dsh a=rtpmap:103 SILK/8000 a=rtcp-fb:103 x-message app send:dsh recv:dsh a=rtpmap:8 PCMA/8000 a=rtcp-fb:8 x-message app send:dsh recv:dsh a=rtpmap:97 RED/8000 a=rtpmap:13 CN/8000 a=rtpmap:118 CN/16000 a=rtpmap:101 telephone-event/8000 a=rtcp-mux a=ice-ufrag:lMRF a=ice-pwd:NR15fT4U6wHaOKa0ivn64MtQ a=setup:actpass a=fingerprint:sha-256 6A:D8:7D:05:1A:ED:DB:BD:6A:60:1A:BC:15:70:D1:6C:A1:D9:00:79:E5:5C:56:15:73:80:E2:82:9D:B9:FB:69 a=mid:nbiwo5l60z a=sendrecv a=msid:7E4272C7-2B6C-49BD-BF7A-A3E7B8DD44F5 D2945771-D7B4-4915-AC29-CEA9EC51EC9E a=ssrc:1001 msid:7E4272C7-2B6C-49BD-BF7A-A3E7B8DD44F5 D2945771-D7B4-4915-AC29-CEA9EC51EC9E a=ssrc:1001 cname:3s6hzpz1jjCheck the anatomy of a WebRTC SDP post to find out what each of these lines mean.
This allows quite a number of the WebRTC PeerConnection samples to work in Edge, just like many of the getUserMedia samples already work.
With that working, the next big challenge was browser interoperability. Would this underspecified blob of text be good enough to be accepted by Chrome and Firefox?
It turned out to be good enough. After adding ICE candidates on both sides the ice connection and DTLS states soon changed to completed and connected. Yay. In Chrome at least.
Firefox did not work because of trivial mistakes that took a while to figure out. But then, it just worked as well.
As far as I am concerned this shows the hard part, making ICE and DTLS interoperable, is solved. The rest is something for codec folks to work out. Not my area of interest
{“author”: “Philipp Hancke“}
Want to keep up on our latest posts? Please click here to subscribe to our mailing list if you have not already. We only email post updates. You can also follow us on twitter at @webrtcHacks for blog updates and news of technical WebRTC topics or our individual feeds @chadwallacehart, @victorpascual and @tsahil.
The post Hello Chrome and Firefox, this is Edge calling appeared first on webrtcHacks.
Here’s a thought. Why not get WebRTC to the operating system level and be done with it?
Today, there are different ways to get WebRTC going:
That last option? This is the closest one to an OS level integration of WebRTC. You assume it is there and available, and use it in your app somehow.
But what if we could miraculously get the WebRTC APIs (Javascript or whatever) from he operation system itself? No compilation needed. No Cordova plugins to muck around with. Just good ol’ “system calls”?
While I don’t really expect this to happen, here’s what we’d gain from having that:
1# Smaller app sizesNot needing to get WebRTC on a device means your app takes up less space. With the average app size on the increase, this is always a good thing.
The OpenH264 codec implementation binary alone is around 300k, depending on the platform. Assume you need 3-4 more codecs (and that number will be growing every couple of years), the other media algorithms, all the network implementation, code to integrate with device drivers, WebRTC specific wrappers, … – lots and lots of size.
And less app size means more space for other app and less app data to send over the network when intsalling the app.
2# Less variabilityWhile the first one is obvious, and somewhat nagging – so it takes a second more to install an app – who cares?
This point has a lot more of a reason for it.
If there’s a single implementation of WebRTC, maintained by the OS itself, there’s a lot less hassle of dealing with the variance.
When people port WebRTC on their own and use it – they make changes and tweaks. They convince themselves (with or without any real reason) that they must make that small fix in that piece of algorithm in WebRTC – after all, they know their use case best.
But now, it is there, so you make do with what you have. And that piece of code gets updated magically and improves with time – you need not upgrade it manually and re-integrate all the changes you’ve made to it.
Less variability here is better.
3# Shorter TTMSince you don’t need to muck around with the work of porting and integration – it takes less time to implement.
I’ve been working with many vendors on how to get WebRTC to work in their use case. Oftentimes, that requires that nasty app to get a WebRTC implementation into it. There’s no straightforward solution to it. Yes – it is getting easier with every passing day, but it is still work that needs to be done and taken into account.
Back to realityThis isn’t going to happen anytime soon.
Unless… it already has to some extent and in some operating systems.
Chrome is an OS – not only Chrome OS but Chrome itself. It has WebRTC built in – in newer Android versions as well, where you can open up webviews with it.
For the rest, it is unlikely to be the path this technology will be taking.
Planning on introducing WebRTC to your existing service? Schedule your free strategy session with me now.
The post 3 Advantages of WebRTC Embedded in the OS appeared first on BlogGeek.me.
The Voxter team discuss code contributions to the Kazoo platform, discuss how they are utilizing the platform, and give an in-depth demo of WhApps
Founder and CEO Mikhail Rodionov discusses all the projects and code contributions that they have built for Kazoo over the past year.
Telnexus CEO Vernon Keenan discuss how he built the Managed Service Provider Telnexus from the ground up and the lessons he has learned in the process.
The ThinQ team discuss how to set up your routing profile, carrier selection, high volume traffic management, and LCR routing
In a competitive market, high quality voice services alone are rarely enough. Lon will speak about the customer lifecycle, back office systems from Sales to CRM to deployment, and how to drive profitable growth while delivering an excellent customer experience.
Something is brewing at Google.
Last week it was announced that Symphony just raised another $100M lead by Google. Not Google Ventures mind you – Google Inc.
Who is Symphony?The service itself is targeted at the enterprise, but a free variant of it is available. I tried logging into it, to see what is all about. It is a variant of the usual messaging app on the desktop, with bits and pieces of Facebook and Slack.
On face value, not much different than many other services.
Symphony FoundationSymphony decided to build its service on top of an open source platform of its own, which it calls Symphony Foundation. It includes all the relevant washed-out words required in a good marketing brochure, but little else for now: a mission statement, some set of values. That’s about it.
It will be open source, when the time comes. It will be licensed under the Apache license (permissive enough). And you can leave an inquiry on the site. In the name of openness… that’s as open as Apple’s FaceTime protocol is/was supposed to be. I’ll believe it when I see it.
Why Invest in Symphony?This is the bigger question here. Both for why Google put money in it, as well as others.
With a total of $166M of investment in two rounds and over 100 employees recruited in its first year of existence, there seems to be a gold rush happening. One that is hard to explain.
As a glaring reminder – Whatsapp on acquisition day had 32 developers and around 50 employees. Symphony has twice that already, but no active user base to back it up.
It might be because of its high profile. After all, this is David Gurle we’re talking about. But then again, Talko has Ray Ozzie. But they only raised $4M in the past 3 years, and have less than 10 employees (if you believe LinkedIn).
The only other reason I can see is the niche they went for.
The financial industry deals with money, so it has money. It also has regulations and laws, making it a hard nut to crack. While most other players are focused on bringing consumer technology to the SMB, Symphony is trying to start from the top and trickle to the bottom with a solution.
The feature set they are putting in place, based on their website, include:
The challenge will be keeping a simple interface while maintaining the complex feature set regulated industries need (especially ones that love customization and believe they are somehow special in how they work and communicate).
On Messaging and RegulationThe smartphone is now 8 years old, if you count it from the launch of the iPhone.
Much has changed in 8 years, and most of it is left unregulated still.
Messaging has moved from SMS to IP based messaging services like Whatsapp in many countries of the world. Businesses are trying to kill email with tools like Slack. We now face BYOD phenomena, where employees use whatever device and tools they see fit to get their work done – and enterprises find it hard to force them to use specific tools.
If Hillary Clinton can use her own private email server during the course of her workday, what should others say or do?
While regulation is slow to catch up, I think some believe the time is ripe for that to happen. And having a messaging system that is fit for duty in those industries that are sensitive today means being able to support future regulation in other/all industries later.
This trend might raise the urgency or the reason for the capital that Symphony has been able to attract.
GoogleWhy did Google invest here? Why not Google Ventures? It doesn’t look like an Alphabet investment but rather a Google one. And why invest and not acquire?
Google’s assets in messaging include today:
Jibe/RCS is about consumer and an SMS replacement in the long run. It may be targeted at Apple. Or Facebook. Or Skype. Or all of them.
None of its current assets is making a huge impact. They aren’t dominant in their markets.
And messaging may be big in the consumer, but the money is in the enterprise – it can be connectivity to enterprises, ecommerce or pure service. Google is finding it difficult there as well.
Symphony is a different approach to the same problem. Targets the enterprise directly. Focusing on highly regulated customers. Putting money into it as an investment is a no-brainer, especially if it includes down the road rights of first refusal on an acquisition proposal for example. So Google sits and waits, sees what happens with this market, and decides how to continue.
Is this a part of a bigger picture? A bigger move of Google in the messaging space? Who knows? I still can’t figure out the motivation behind this one…
Messaging and meI’ve been writing on general messaging topics on and off throughout the years on this blog.
It seems this space is becoming a lot more active recently.
Expect more articles here about this topic of messaging from various angles in the near future.
Planning on introducing WebRTC to your existing service? Schedule your free strategy session with me now.
The post Google Goes All in for Messaging, Invests in Symphony appeared first on BlogGeek.me.
Hello, again. This past week in the FreeSWITCH master branch we had 51 commits! There were some very important changes this week to the Debian packaging system. The default is now set to build packages with the upstream FS package repos. Since the system dependencies have been removed from the FS codebase the 1.6 branch is now required to use the FS public repo for dependencies. The notable feature for this week is the addition of the variable media_mix_inbound_outbound_codecs, which mixes inbound and outbound codecs, and this is a behavior change.
Join us on Wednesdays at 12:00 CT for some more FreeSWITCH fun! And head over to freeswitch.com to learn more about FreeSWITCH support.
New features that were added:
Improvements in build system, cross platform support, and packaging:
The following bugs were squashed:
And, this past week in the FreeSWITCH 1.4 branch we had 3 new commits merged in from master. And the FreeSWITCH 1.4.23 release is here! Go check it out!
The following bugs were squashed:
We have been waiting a long time for Microsoft to add WebRTC to its browser portfolio. That day finally came last month when Microsoft announced its new Windows 10 Edge browser had ORTC. This certainly does not immediately address the Internet Explorer population and ORTC is still new to many (which is why we cover it often). On the positive side, interoperability between Edge, Chrome, and Firefox on the audio side was proven within days by multiple parties. Much of ORTC is finding its way into the WebRTC 1.0 specification and browser implementations.
I was with Bernard Aboba, Microsoft’s WebRTC lead at the IIT Real Time Communications Conference (IIT-RTC) and asked him for an interview to cover the Edge implementation and where Microsoft is headed. The conversation below has been edited for readability and technical accuracy. The full, unedited audio recording is also available below if you would rather listen than read. Warning – we recorded our casual conversation in an open room off my notebook microphone, so please do not expect high production value.
https://webrtchacks.com/wp-content/uploads/2015/10/Bernard-Aboba-QA.mp3We cover what exactly is in Edge ORTC implementation, why ORTC in the first place, the roadmap, and much more.
You can view the IIT-RTC ORTC Update presentation slides given by Bernard, Robin Raymond of Hookflash, and Peter Thatcher of Google here.
{“editor”, “chad hart“}
Intro to BernardwebrtcHacks: Hi Bernard. To start out, can you please describe your role at Microsoft and the projects you’ve been working on? Can you give a little bit of background about your long time involvement in WebRTC Standards, ORTC, and also your new W3C responsibilities?
Bernard: I’m a Principal Architect at Skype within Microsoft, and I work on the Edge ORTC project primarily, but also help out other groups within the company that are interested in WebRTC. I have been involved in ORTC since the very beginning as one of the co-authors of ORTC, and very recently, signed up as an Editor of WebRTC 1.0.
webrtcHacks: That’s concurrent with some of the agreement around merging more of ORTC into WebRTC going forward. Is that accurate?
Bernard: One of the reasons I signed up was that I found that I was having to file WebRTC 1.0 API issues and follow them. Because many of the remaining bugs in ORTC related to WebRTC 1.0, and of course we wanted the object models to be synced between WebRTC 1.0 and ORTC, I had to review pull requests for WebRTC 1.0 anyway, and reflect the changes within ORTC. Since I had to be aware of WebRTC 1.0 Issues and Pull Requests to manage the ORTC and Pull Requests, I might as well be an editor of WebRTC 1.0.
What’s in EdgewebrtcHacks: Then I guess we’ll move on to Edge then. Edge and Edge Preview are out there with varying forms of WebRTC. Can you walk through a little bit of that?
Bernard: Just also to clarify for people, Edge ORTC is in what’s called Windows Insider Preview. Windows Insider Preview builds are only available to people who specifically sign up to receive them. If you sign up for the Windows Insider Preview program and install the most recent build 10547, then you will have access to the ORTC API in Edge. In terms of what is in it, the audio is relatively complete. We have:
Then on the video side, we have an implementation of H.264/SVC, which does both simulcast and scalable video coding, and as well as forward error correction (FEC), known as H.264UC. I should also mention, we support RED and forward error correction for audio as well.
That’s what’s you will find in the Edge ORTC API within Windows Insider Preview, as well as support for “half-trickle” ICE, DTLS 1.0, etc.
webrtcHacks: I’ll include the slide from your presentation for everyone to reference because there’s a lot of stuff to go through. I do have a couple of questions on a few things for follow up. One was support on the video side of things for. I think you mentioned external FEC and also talked about other aspects of robustness, such as retransmission?
Bernard: Currently in Edge ORTC Insider Preview, we do not support generic NACK or re-transmission. We do support external forward error correction (FEC), both for audio and video. Within Opus as well as SILK we do not support internal FEC, but you can configure RED with FEC externally. Also, we do not support internal Discontinuous Operation (DTX) within Opus or SILK, but you can configure Comfort Noise (CN) for use with audio codec, including Opus and SILK.
Video interoperabilitywebrtcHacks: Then could you explain H.264 UC? The majority of the people out there that aren’t familiar with the old Lync or Skype for Business as it is now called.
Bernard: Basically, H.264 UC supports spatial simulcast along with temporal scalability in H.264/SVC, handled automatically “under the covers”. These are basically the same technologies that are in Hangouts with VP8. While the ORTC API offers detailed control of things like simulcast and SVC, in many cases, the developer just basically wants the stack to do the right thing, such as figuring out how many layers it can send. That’s what H.264UC does. It can adapt to network conditions by dropping or adding simulcast streams or temporal layers, based on the bandwidth it feels is available. Currently, the H.264UC codec is only supported by Edge.
webrtcHacks: Is the base layer H.264?
Bernard: Yes, the base layer is H.264 but RFC 6190 specifies additional NAL Unit types for SVC, so that an implementation that only understands the base layer would not be able to understand extension layers. Also, our implementation of RFC 6190 sends layers using distinct SSRCs, which is known as Multiple RTP stream Single Transport (MRST). In contrast, VP8 uses Single RTP stream Single Transport (SRST).
We are going to work on an implementation of H.264/AVC in order to interoperate. As specified in RFC 6184 and RFC 6190, H.264/AVC and H.264/SVC have different codec names.
webrtcHacks: For Skype, at least, in the architecture that was published, they showed a gateway. Would you expect other people to do similar gateways?
Bernard: Once we support H.264/AVC, developers should be able to configure that codec, and use it to communicate with other browsers supporting H.264/AVC. That would be the preferred way to interoperate peer-to-peer. There might be some conferencing scenarios where it might make sense to configure H.264UC and have the SFU or mixer strip off layers to speak to H.264/AVC-only browsers, but that would require a centralized conferencing server or media relay that could handle that.
RoadmapwebrtcHacks: What can you can you say about the future roadmap? Is it basically what’s on the dev.modern.ie page?
Bernard: In general, people should look at the dev.modern.ie web page for status, because that has the most up to date. In fact, I often learn about things from the page. As I mentioned, the Screen Sharing and Media Recorder specifications are now under consideration, along with features that are in preview or are under development. The website breaks down each feature. If the feature is in Preview, then you can get access to it via the Windows Insider Preview. If it is under development, this means that it is not yet in Preview. Features that are supported have already been released, so if you have Windows 10, you should already have access to them.
In terms of our roadmap, we made a roadmap announcement in October 2014 and are still executing on things such as H.264, which we have not delivered yet. Supporting interoperable H.264 is about more than just providing an encoder/decoder, which we have already delivered as part of H.264UC. The IETF RTCWEB Video specification provides guidance on what is needed to provide interoperable H.264/AVC, but that is not all that a developer needs to implement – there are aspects that are not yet specified, such as bandwidth estimation and congestion control.
Beyond the codec bitstream, RTP transport and congestion control there are other aspects as well. For example, I mentioned robustness features such as Forward Error Correction and Retransmission. A Flexible FEC draft is under development in IETF which will handle burst loss (distances greater than one). That is important for robust operation on wireless networks, for both audio and video. Today we have internal FEC within Opus, but that does not handle burst loss well.
webrtcHacks: Do you see Edge pushing the boundaries in this area?
Bernard: One of the areas where Edge ORTC has advanced the state of the art is in external forward error (FEC) correction as well as in statistics. Enabling external FEC to handle burst loss, provides additional robustness for both audio and video. We also support additional statistics which provide information on burst loss and FEC operation. What we have found is that burst loss is a fact of life on wireless networks, so that being able to measure this and to address it is important. The end result of this work is that Edge should be more robust than existing implementations with respect to burst loss (at least with larger RTTs where retransmission would not be available). We can also provide burst loss metrics, which other implementations cannot currently do. I should also mention that there are metrics have been developed in the XRBLOCK WG to address issues of burst loss, concealment, error correction, etc.
Why ORTC?webrtcHacks: You have been a long time advocate for ORTC. Maybe you can summarize why ORTC was a good fit for Edge? Why did you start with that spec versus something else? What does it enable you to do now as a result?
Bernard: Some of the advantages of ORTC were indeed advantages, but in implementation we found there were also other advantages we didn’t think of at the time.
InteroperabilityBernard: ORTC doesn’t have SDP [like WebRTC 1.0]; the irony is ORTC allowed us to get to WebRTC 1.0 compatibility and interoperability faster than we would have otherwise. If you look at the adapter.js, it’s actually interesting to read that code- the actual code for Edge is actually smaller than for some of the other browsers. One might think that’s weird – why would it take less adaptation for Edge than for anything else? Are we really more 1.0 compatible than 1.0? The answer is, to some respects, we are, because we don’t generate SDP than somebody needs to parse and reformat. It certainly saves a lot of development to not have to write that code and have control in JavaScript, and also be easy to modify in case people find bugs in it.
The irony is ORTC allowed us to get to WebRTC 1.0 compatibility and interoperability faster than we would have otherwise
Connection State DetailsThe other thing we found about ORTC that we didn’t quite understand early on was it gives you detailed status of each of the transports- each of your ICE transports. Particularly when you’re dealing with situations like multiple interfaces, you actually get information about failure conditions that you don’t get out of WebRTC 1.0.
It’s interesting to look at 1.0 – one of the reasons that I think people will find the objects interesting in 1.0 is because you actually need that kind of diagnostic information. The current connection state [in the current WebRTC] is not really enough – it’s not even clear what it means. It says in the spec that it’s about ICE, but it really combines ICE and DTLS. With the object model, you know exactly what ICE transport went down or if DTLS is in some weird state. Actually for diagnostics, details of the connection state is actually pretty important. It’s one of the most frequently requested statistical things. That was a benefit we didn’t anticipate, that we found is pretty valuable and will be coming into 1.0.
Many simple scenariosBernard: Then there were the simple scenarios. Everyone said, “I don’t need ORTC because I don’t do scalable video coding and simulcast” Do you ever do hold? Do you ever do changing owners of codecs? All illustrations that Peter [Thatcher] showed in his WebRTC 1.0 presentation. The answer is, a lot of those things are, in fact, common, and were not possible in 1.0. There is a lot of fairly basic benefits that you get as well.
How is Edge’s Media Engine builtwebrtcHacks: In building and putting this in the Edge, you had a few different media engines you could choose from. You had the Skype media engine and a Lync media – you combine them or go and build a new one. Can you reveal the Edge media architecture and how you put that together?
Bernard: What we chose to do in Skype is move to a unified media engine. What we’ve done is, we’ve added WebRTC capabilities into that media engine. That’s a good thing because, for example, things like RTCP MUX and things like BUNDLE are now part of the Skype media engine so we can use them. The idea was to produce something that was unified and would have all the capabilities in one. It took a little bit longer to do it that way, but the benefit is that we get to produce a standardized compliant browser and we also get to use those technologies internally. Now we do not have 3 or 4 different stacks that we would have to rationalize later.
right now, our focus is very much on video, and trying to get that more solid, and more interoperable
Also, I should mention that one thing that is interesting about the way we work is we produce stacks that are both client and server capable. We don’t just produce pure client code that wouldn’t, for example, be able to handle load. Some of those things can go into back-end components as well. That is also true for DTLS and all that. Whether or not we use all those things in Skype is another issue, but it is part of the repertoire for apps.
More than EdgewebrtcHacks: Is there anything else that’s not on dev.modern.ie that is exposed that a developer would care about? Any NuGet packages with these API’s for example?
Bernard: There is a couple of things. dev.modern.ie does not cover non-browser things in Windows platform. For example, currently we support DTLS 1.0. We do want to support 1.2, because there’s additional cipher suites that are important. For example, the Elliptic Curve stuff we’re seeing going into all the browsers. I think Mozilla already has it, or Chrome has it, or if they don’t, they will very soon. That is actually very important. Elliptic Curve turned out to be more than just a cipher suite issue – the time and effort it takes to generate more secure certificates is large. For RSA-2048 you can actually block the UI thread if you thread the object. Anyway, those are very important things that we don’t cover on dev.modern.ie, but those are the things we obviously have to do.
There’s a lot of work and a lot of thinking that’s been going on in the IETF if relating to ICE and how to be better for mobile scenarios. Some of that I don’t think is converged yet, but there’s a new ICE working group. Some of that is in the ortc-lib implementation yet. Robin [Raymond] likes to be on the cutting edge so he has done basically the first implementation of a lot of those new technologies. That’s something, I think is of general interest – particularly as ORTC moves to mobile.
I should mention, by the way, that the Edge Insider Preview was only for desktop. It does not run on Windows Phone just to clarify that.
webrtcHacks: Any plans for embedding the Edge ORTC engine as a IE plugin?
Bernard: An external plugin or something?
webrtcHacks: Yeah, or a Microsoft plugin for IE that would implement ORTC.
Bernard: Basically at this point, IE is frozen technology. All the new features, if you look on the website, they all go into Edge. That’s what we’ve been developing for. I never say Microsoft will never do anything, but currently that’s not the thinking. Windows 10 for consumers is a free upgrade. Hopefully, people will take advantage of that and get all the new stuff, including Edge.
Is there an @MSEdgeDev post on the relationship between this and InPrivate? pic.twitter.com/bbu0Mdz0Yd
— Eric Lawrence (@ericlaw) September 22, 2015
A setting discovered in Internet Explorer that appears to address the IP Address Leakage issue. Validating ORTCwebrtcHacks: Is there anything you want to share?
Bernard: I do want to clarify a little bit, I think adapter.js is a very important thing because it validates our original idea that essentially WebRTC 1.0 could be built into the JavaScript layer with ORTC.
webrtcHacks: And that happened pretty quick – with Fippo‘s help. Really quick.
Bernard: Fippo has written all the pull requests. We’re paying a lot of attention to the bugs he’s finding. Obviously, he’s finding bugs in Edge, which hopefully we’ll fix, but he’s also finding spec bugs. It really helps make sure that this compatibility that we’ve promised is actually real. It’s a very interesting process to actually reduce that to code so that it’s not just a vague promise. It has to be demonstrated in software.
Of course what we’ve done is currently with audio. We know that video is more complicated, particularly as you start adding lots and lots of codecs to get that level of compatibility. I wouldn’t say that when Fippo is down with audio that it will be the last word. I think we’ll have to pay even more attention to interoperability stuff in the video cases. It will be interesting because video is a lot more complicated.
adapter.js is a very important thing because it validates our original idea that essentially WebRTC 1.0 could be built into the JavaScript layer with ORTC.
What does the Microsoft WebRTC team look likewebrtcHacks: Can you comment on how big the time is that’s working on ORTC in Edge? You have a lot of moving pieces in different aspects …
Bernard: There’s the people in Edge. There’s the people in Skype. In the Windows system there’s the people on the S-channel team that worked on the DTLS. There’s people all over – for example, the VP9 work that we talked about, was not done by either Skype or the conventional Edge people. It’s the whole Windows Media team. I don’t really know how to get my hands around this, because if you look at all the code we’re using, it’s written by probably, I don’t know, hundreds and hundreds of people.
webrtcHacks: And you need to pull it together for purposes of WebRTC/ORTC, is that right?
Bernard: Yeah. We have to pull it together, but there’s a lot there. There’s a lot of teams. There will probably be more teams going forward. People say, “Why don’t you have the datachannel”? The dataChannel isn’t something that would be in Skype’s specific area of expertise. That’s a transfer protocol, it should be really written by people who are experts in transfer protocol, which isn’t either Edge or Skype. It’s not some decision that was made by either of our groups not to do it. We have to find somebody who proves that they can do that work, to take ownership of that.
Feedback pleasewebrtcHacks: Any final comments?
Bernard: No. I just encourage people to download the preview, run it, file bugs, and let us know what you think. You can actually can vote on the website for new features, which is cool.
We do listen to the input. WebRTC is an expanding thing. There’s a ton of things you can do – there’s all that stuff on dev.modern.ie site and then there’s internal improvement. Getting a sense of priority is what’s most important to people, is not that easy, because there’s so much that you could possibly focus on. I’d say right now, our focus is very much on video, and trying to get that more solid, and more interoperable, at least for the moment. We can walk and chew gum at the same time. We can do more than just one thing. Conceivably, especially when you look at IE and other teams.
webrtcHacks: This is great and very insightful. I think it will be a big help to all the developers out there. Thanks!
{
“Q&A”:{
“interviewer”:“chad hart“,
“interviewee”:“Bernard Aboba“
}
}
Want to keep up on our latest posts? Please click here to subscribe to our mailing list if you have not already. We only email post updates. You can also follow us on twitter at @webrtcHacks for blog updates and news of technical WebRTC topics or our individual feeds @chadwallacehart, @victorpascual and @tsahil.
The post Microsoft’s ORTC Edge for WebRTC – Q&A with Bernard Aboba appeared first on webrtcHacks.
Yes.
It is a question I get from time to time, especially now, that I am a few months into the WebRTC testing venture as a co-founder with a few partners – testRTC.
The logic usually goes like this: the browsers already support WebRTC. They do their own testing, so what we end up getting is a solid solution we can use. Fippo would say that
If life was that easy… here are a few things you need to take care of when it comes to testing the most simple of WebRTC services:
#1 – Future proofing browser versionsGuess what? Things break. They also change. Especially when it comes to WebRTC.
A few interesting tidbits for you:
And the list goes on.
WebRTC is a great technology, but browsers are running at breakneck speeds of 6-8 weeks between releases (for each browser) – and every new release with a potential to break a service in multitude of ways – either because of a change in the spec, deprecation of capability or just bugs.
Takeaway: Make sure your service works not only on the stable version of the browsers, but also on their beta or even dev versions as well.
#2 – Media relayYour service might be a P2P service, but at times, you will need to relay media through TURN servers.
The word on the street is that around 15% of sessions require relay. To some it can be 50% and to others 8% (real numbers I heard from running services).
Media relay is tricky:
Takeaway: Don’t treat WebRTC as a browser side technology only, or something devoid of media handling. Even if the browser does most of the heavy lifting, some of the effort (and responsibility) will lie on your service.
#3 – Server scaleCan your server cater for 200 sessions in parallel to fit that contact center? What about a 1000?
What will happen if you’ll have a horde effect due to a specific event? Can you handle that number of browsers hitting your service at once? Does your website operate in the same efficiency for the 1000th person as it does for the first?
This relates to both your signaling server, which is no part of WebRTC, but is there as part of your service AND your media server from my previous point.
Takeaway: Make sure your service scales to the capacities that it needs to scale. Oh – and you won’t be able to test it manually with the people you have with you in your office…
#4 – Service uptimeYou tested it all. You have the perfect release. The service is up and running.
How do you make sure it stays running?
Manually? Every morning come in to the office and run a session?
Use Pingdom to make sure your site is up? Go to the extreme of using New Relic to check the servers are up, the CPUs aren’t over loaded and the memory use seems reasonable? Great. But does that mean your service is running and people can actually connect sessions? Not necessarily.
Takeaway: End-to-end monitoring. Make sure your service works as advertised.
The ugly truth about testingThe current norm in many cases is to test manually. Or not test at all. Or rely on unit testing done by developers.
None of this can work if what you are trying to do is create a commercial service, so take it seriously. Make testing a part of your development and deployment process.
And while we’re at it…
Check us out at testRTCIf you don’t know, I am a co-founder with a few colleagues at a company called testRTC. It can help you with all of the above – and more.
Leave us a note on the contact page there if you are interested in our paid service – it can cater to your testing needs with WebRTC as well as offering end-to-end monitoring.
Need to test WebRTC?
The post Do you Need to test a WebRTC P2P Service? appeared first on BlogGeek.me.
SMB phone system
Telephony
Small
Voice
Disrupting the hosted PBX system with WebRTC.
[If you are new around here, then you should know I’ve been writing about WebRTC lately. You can skim through the WebRTC post series or just read what WebRTC is all about.]
There’s no doubt that WebRTC is disrupting many industries. One of the obvious ones is enterprise communications, and in this space, an area that has got little attention on my end (sorry) is the SMB – where a small company needs a phone system to use and wants to look big while at it.
Moshe Maeir, Founder at Fone.do, just launched the service out of Alpha. I have been aware of what they were doing for quite some time and Moshe took the time now that their service is public to answer a few of my questions.
What is Fone.do all about?
Fone.do is a WebRTC based phone system for small businesses that anyone can set up in 3 minutes. It replaces both legacy PBX systems that were traditionally based in your communications closet and also popular Hosted PBX systems. Businesses today are mobile and the traditional fixed office model is changing. So while you can connect a SIP based IP phone to our system, we are focused on meeting the needs of the changing business world.
Why do small businesses need WebRTC at all? What’s the benefit for them?
You could ask the same question about email, social networks etc. Why use web based services at all? Does anyone want to go back to the days of “computer programs” that you downloaded and installed on your computer? Unfortunately, many still see telephony and communications as a stand alone application. WebRTC changes this. Small businesses can communicate from any place and any device as long as they have a compatible platform.
What excites you about working in WebRTC?
Two things. Not sure which is more exciting. First of all. If I build something great – the whole world is my potential market. All they need is a browser and they are using our system in 3 minutes. The other exciting aspect is that telephony is no longer a closed network. Once you are on the web the potential is unlimited. You can easily connect your phone system to the wealth of data and services that already exist on the web and take communications to a new level. In fact, that is why we hired developers who knew nothing about telephony but were experienced in web development. The results are eye opening for traditional telecom people.
I know you’re a telecom guy yourself. Can you give an example how working with web developers was an eye opener to you?
There are many. The general attitude is just do it. With legacy telecom, everything has the accepted way of doing things and you don’t want to try anything new without extended testing procedures. A small example – in the old VoIP days writing a “dial plan” was a big thing. When we came to this issue on Fone.Do, one of the programmers naturally googled the issue and found a Google service that will automatically adapt the dial plan based on the users’ mobile number. 1-2-3 done.
Backend. What technologies and architecture are you using there?
Our main objective was to build an architecture that will work well and easily scale in the cloud (we are currently using AWS). So while we have integrated components such as the Dialogic XMS and the open source Restcomm, we wrote our own app server which manages everything. This enables us if we need to freely change back end components.
Can you tell us a bit about your team? When we talked about it a little of a year ago ago, I suggested a mixture of VoIP and web developers. What did you end up doing and how did it play out?
All our developers are experienced front end and backend web programmers with no telecom experience. However, our CTO who designed the system has over 15 years of experience in Telecom, so he is there to fill in any missing pieces. There were some bumps at the beginning, but I am very happy we did it this way. You can teach a web guy about Telephony, but it is very hard to get a Telecom guy to change his way of thinking. Telecom is all about “five nines” and minimizing risk. Web development is more about innovation and new functionality. With todays’ technology it is possible to innovate and be almost as reliable as traditional telephony
Where do you see WebRTC going in 2-5 years?
Adoption is slower than I expected, but eventually I see it as just another group of functions in your browser that developers can access as needed.
If you had one piece of advice for those thinking of adopting WebRTC, what would it be?
WebRTC is here. It makes your user experience better – so what are you waiting for?
What’s next for Fone.do?
We recently released our alpha product and we are looking to launch an open beta in the next couple of months. Besides a web based “application”, we also have applications for Android and iOS.
–
The interviews are intended to give different viewpoints than my own – you can read more WebRTC interviews.
The post Fone.do and WebRTC: An Interview With Moshe Maeir appeared first on BlogGeek.me.
CTO Karl Anderson discusses the state of Kazoo. This includes integrations with FreeSWITCH, erlang, and Kamailio. Reseller milestones include the release of whitelabeling, webhooks, migration, carriers, debugging, account management and more.
This is an overview of VoIP fraud, different types of fraud and what telecommunication carriers are doing to combat this issue. Types of fraud include International / Premium Number Fraud, Impersonation / Social Engineering, Service Degradation / Denial of service. Presented by Mark Magnusson at KazooCon 2015.
James Aimonetti discusses VoIP Rates, Routing, and Services at KazooCon 2015.
Billing Data with Kazoo from James Solada
Product Director Aaron Gunn discusses billing options for SaaS and IaaS customers. This includes CDR API, AMPQ, and integrating VoIP billing platforms.
People love to talk about scale. Some vendors pitch that their systems easily support 100,000 simultaneous calls, or 500 calls per second, etc. The reality is, in the real world, people’s behaviors vary and the feature sets they use can cut these numbers down quickly. For example, ask that same vendor claiming 100,000 simultaneous calls if it can be done while call recording, call statistics and other features are turned on at the same time, and you’ll usually get a very different, cautious, qualified response.
In this presentation, we’ll show you how to set up your infrastructure to support 100,000 simultaneous calls.
http://www.slideshare.net/2600hzmarketing/tuning-kazoo-to-10000-handsets-kazoocon-2015
People love to talk about scale. Some vendors pitch that their systems easily support 100,000 simultaneous calls, or 500 calls per second, etc. The reality is, in the real world, people’s behaviors vary and the feature sets they use can cut these numbers down quickly. For example, ask that same vendor claiming 100,000 simultaneous calls if it can be done while call recording, call statistics and other features are turned on at the same time, and you’ll usually get a very different, cautious, qualified response.
In this presentation, we’ll show you how to set up your infrastructure to support 100,000 simultaneous calls.
Phosfluorescently utilize future-proof scenarios whereas timely leadership skills. Seamlessly administrate maintainable quality vectors whereas proactive mindshare.
Dramatically plagiarize visionary internal or "organic" sources via process-centric. Compellingly exploit worldwide communities for high standards in growth strategies.
Wow, this most certainly is a great a theme.
Donec sed odio dui. Nulla vitae elit libero, a pharetra augue. Nullam id dolor id nibh ultricies vehicula ut id elit. Integer posuere erat a ante venenatis dapibus posuere velit aliquet.
Donec sed odio dui. Nulla vitae elit libero, a pharetra augue. Nullam id dolor id nibh ultricies vehicula ut id elit. Integer posuere erat a ante venenatis dapibus posuere velit aliquet.