How CalaBoard requirements lead us to a pure Java codec

Starting up a company can be hard sometimes but luckily there are some aspects of this effort that can be really rewarding. One of this is expressing one’s self into the art of programming and this is what we actually did with the communication mechanism in CalaBoard, our video-conferencing tool based on Augmented Reality.

Video-conferencing requires a point-to-point connection between the parties involved in the conference.
At the coding level, our architecture of the system rotates around the concepts of Publishers and Readers. A Publisher is some entity that can publish data on some media (a network, a file, etc.) while the Reader is another entity capable of reading that data (and therefore capable of reading from a network connection, reading a file and so on).

Publishers and Readers are independent from the data they process as they work on a more abstract layer – the data can be a video stream or an audio stream or whatever has to be transmitted.

We currently have 3 kinds of Publisher:

  • the first implementation writes data to a file and it is useful in the following situations:
    – recording a video
    – saving messages to the hard-disk
    – playing audio/video recordings
  • the second implementation addresses data transmission over TCP networks
  • the third one addresses data transmission over UDP networks and performs NAT traversals

Data to be transmitted is encoded through a Codec layer represented by a Java class that can be plugged into the architecture through a dedicated software-slot making it possible for an on-the-fly replacement. The purpose of the Codec is to convert a video-frame or an audio-sample into a format optimized for transmission. Once data is converted, it gets passed to the Publisher for the actual transmission.

The algorithm implemented by the Codec cares about two aspects:

  • maximization of the compression rate
  • maximization of the encoding speed

The two aspects above are in conflict with each other as an high compression rate requires time and this affects negatively the overall encoding speed.
We decided to develop our own Codec logic because we wanted a pure-Java version to allow us the full control of the process. The most common Codec mechanisms are now the h264 and the mpeg. In general, available Codec technology for Java is based on wrappers around native C libraries like ffmpeg or gstreamer, sometimes not too easy to deal with.

To better understand why the encoding process is heavier than the decoding one it is sufficient to think about how movement recognition is currently dealt with: images are split into parts and only changed parts are actually transmitted. Discovering which part changed or which part is identical to another already sent requires processing time as the part has to be compared to many other older parts. The process of comparison requires many mathematical processing and involves statistics also.

The main features of our Codec technology are the following:

  • it sends only the parts of the image that actually changed between two consecutive frames (delta packets) and at regular intervals it also sends complete packets (keyframes)
  • it avoids sending blocks “near” to other blocks already sent
  • it avoids sending blocks already sent that are identical to the surroundings of the last keyframe
  • it uses a quality-less algorithm to compress the video by loosing quality but at the same time reducing the required amount of transmission band
  • it uses a configurable frame-rate

We’ll keep you posted about further updates…

Explore posts in the same categories: Augmented Reality, Calameda, Networking

Tags: ,

You can comment below, or link to this permanent URL from your own site.

2 Comments on “How CalaBoard requirements lead us to a pure Java codec”

  1. tretle Says:

    Interesting stuff. Will the source to the decoder be open? Have you thought about cortado integration?

    • calameda Says:

      It’s a bit early for us to talk about open-sourcing our encoder but it could be an option. Regarding cortado we know about it but haven’t thought of integrating our encoder. Again, it could be an option.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: