Elemental Blog

Video Processing Perspectives

Elemental Blog

Elemental Blog

Submitted by Sam on February 21, 2008

The Elemental blog seems to attract an inordinate amount of spam, but it also gets the occasional reasonable question from folks. One of the more confusing aspects of video compression seems to be inter- vs. intraframe compression. Here's a brief lesson on both. Video in it's simplest sense can be thought of as a series of still images (hence the early days of film were called "moving pictures" until some marketing genius compressed the words and came up with "movies"). When these still images are flipped past the human eye fast enough, the eye interprets the frames as motion instead of unique still images. From a compression perspective, however, they are still just a series of images displayed in a specified order. Video compression then focuses on how to take these contiguous frames of video and minimize the amount of information needed to code the picture. The natural first step is to compress each individual image. This is known as intraframe compression, and uses only information contained in the current frame to minimize the image size. As an example, JPEG (the standard file format used for images on the Internet) uses the discrete cosine transform to rid images of high-frequency components, which are generally not perceptible by the human psychovisual system; by throwing this information away, a still image can be coded with much less data. This idea has been refined over generations, with early still-image standards like GIF and JPEG laying the foundation for more complex video standards like Motion JPEG and DV, the standard widely used in MiniDV videocameras.  As demand for better picture quality at lower bit rates increased, however, the compression achievable by intra-only encoding became insufficient. Temporal compression or interframe encoding was introduced in the MPEG-1 standard, and has since been refined in the MPEG-2, VC-1 and H.264 codecs. These codecs include intraframe (I-frame) coded images as described in the preceding paragraph, but they also contain predictive-coded frames (P-frames) and bidirectionally-predictive-coded frames (B-frames). P-frames rely on images that were transmitted earlier in the sequence, and use data in these frames -- with minor changes -- create the current frame. B-frames are similar, but can use data from images earlier and later in video sequence. There can be many P- and B-frames between each I-frame, and since most video sequences have similar images for long periods of time dramatically higher compression can be achieved. The consecutive number of interframe images is referred to as Group of Picture (GOP) length. The benefits of intraframe-only compression are that it is generally less computationally expensive to process, since it doesn't require multiple frames to be stored in memory and accessed concurrently. There is also less latency in the encoding process, so compressed images are created much quicker. Hence, historically digital videocameras have captured intraframe-only formats (DV, DV50, DVCPRO HD, AVC-Intra). However, new generations of consumer camcorders with limited storage capacity are relying on interframe-encoded formats like HDV (a long-GOP version of MPEG-2) and AVCHD (a long-GOP version of H.264). These formats allow high-definition video to be stored on the same MiniDV tapes that previously could only capture standard-definition video. Editing these long-GOP formats is incredibly computationally intensive, as for each image displayed, many temporally adjacent frames need to be decoded first. Hence the need for GPU-accelerating decoding and Elemental's RapiHD™ software!

Submitted by Sam on January 14, 2008

Elemental's RapiHD™ Video Platform is powered by fast, modern graphics processing units (GPUs). This is different from most computer software, which runs on the central processing unit (CPU). We often get asked why we selected the GPU as our hardware platform. The basic answer is straightforward: it is currently the most cost-efficient architecture for high-performance computing. Furthermore, there is a confluence of three major technology trends that have come together to make the time just right to use GPUs for more than just 3D games.

  1. Unsurpassed performance, and an unlimited performance trajectory. Semiconductor manufacturers have run into painful physics limitations that have reduced their ability to gain performance through faster clock rates. For example, Intel's top of the line CPUs have run right around 3 GHz for several years now. To increase performance Intel has been forced to add multiple cores -- and Intel has the best manufacturing facilities on the planet. The major GPU makers have had to contend with this problem for longer, and thus migrated to a stream processor architecture several years ago. Today, the highest-performance NVIDIA GPUs have 128 stream processors; ATI tops out at 320 stream processors. 
  2. At long last, GPUs have gained more programmability. NVIDIA's release of the CUDA platform has created a way to take advantage of GPU horsepower with high-level C code. AMD/ATI has a similar initiative underway called CTM (Close to the Metal), although it is targeted at a significantly lower level than CUDA. Intel's Larrabee project, a high-performance discrete GPU that will go head-to-head with NVIDIA and AMD, is due out in 2009 and will reportedly contain a multitude of x86-based cores -- allowing the use of standard development environments to program the chip. All in all, the recent addition of reasonable programming models for GPUs makes them a good candidate for high-performance applications. At the same time, it doesn't make the task of finding talented developers that can efficiently implement parallel code any easier. Elemental is always hiring parallel processing superstars!
  3. Dramatically increased system bandwidth. In the "old days" of PCI and AGP, there was a very fast downlink from the CPU to the GPU. That is, the CPU could deliver data for the GPU to process and then output to the video display. However, the bandwidth for the reverse path -- from the GPU back to the CPU -- was extremely low. The development of PCI Express changed this equation. Now the bandwidth between the GPU and CPU is symmetric, allowing data to be sent to the GPU for analysis and then returned to the host system when processing is complete. This is critical for our application: frames of high-definition video are huge and require a tremendous amount of bandwidth to get schlepped back and forth quickly!


Submitted by Sam on January 3, 2008

Sam Blackman Welcome, gentle reader, to the Elemental blog. My name is Sam Blackman, CEO of Elemental Technologies, and it's a pleasure to have your virtual company. As you probably know (since you are here), Elemental aspires to write the fastest, highest quality video-processing software anywhere. That skillset does not necessarily lend itself to writing, but nonetheless my goal is to make this blog interesting and insightful. There's a fine line to tread when writing a corporate blog: if it's just hyping your latest product release, it might as well be marketing collateral and most people with better things to do won't read it; yet filling it with the latest on Britney's travails probably isn't appropriate, either, since we are going to have a tough time convincing high school kids to buy the RapiHD SDK. Given these constraints, I will try to keep the blog interesting and relevant as well as technically accurate. A good way to start, then, is to highlight other entrepreneurial blogs that I think have figured out this balance. These two are well worth adding to your RSS feed if you have ever thought about starting a company. Two of my favorites:

  • Redfin logoGlenn Kelman's blog over at Redfin. One of the most detailed yet still humorous blogs on starting a company. He also is good buddies with Guy Kawasaki and has some great postings on Guy's site, including "Financial Models for Underachievers: Two Years of the Real Numbers of a Startup."
  • gDiapers LogoJason Graham-Nye at gDiapers has a great product (biodegradeable, compostable disposal diapers), and an equally entertaining blog. Jason moved to Portland from Australia a couple years ago and started gDiapers with his wife. They now run the business out of (and live in) a large old home in Northwest Portland ... every day on the way to work I pass the Prius in the parking strip with a giant sticker that says "CHANGE gDIAPERS". Both Redfin and gDiapers have really mastered the art of marketing and public relations!



View by Tags