Streaming Video: An Overview
The delivery of video through the web seems to have taken on a life of its own. About two years ago I discovered just how cool it was and wrote an article about it at Community MX and at the MX Developers Journal. The premise was that "QuickTime" , the video format, was a dead technology when it came to adding video to a web page. At the time I privately thought the Flash Video technology would gain broad acceptance and achieve critical mass by now. Boy was I wrong. It seems like half the web came to the realization that video is now a staple of the web at the same time I did. In fact, I knew critical mass had been reached when Amazon ran a series of Flash Videos on its home page in December of 2004.
That's the good news. The bad news is there are still a lot of people out there who think adding video to their pages or those of their clients is dead simple. It is...if you know what you are doing and pay attention to the pipe. There are a lot of tutorials out there that show you how to get a video playing in Flash. Then an odd thing happens. The video works... sort of ... but the playback, the video images or the sound sucks.
This article walks you through many of the decisions and technologies used to get a video to the point where it is handed to you with the instruction, "See what you can do to get this on our site." As you will discover if you read this article, many of the problems you will encounter in following that direction were in place well before you were given the QuickTime or AVI file.
At its most basic video is nothing more than a series of still images with a soundtrack. If you have a TV, the video you watch there plays back at a rate of 29.97 FPS. All this means is each second of video contains 29.97 still images. These images playback in a straight line and, as they are shown, the corresponding sound is reproduced. If you have video editing software, you see this when you add a video clip. For example, in Adobe Premiere Pro 2, adding a video clip will automatically put the video on one track and the audio on another.
Image 1: When a video clip is added to video editing software, such as Adobe Premiere (shown), the clip is automatically separated into a video track and an audio track.
The two most important aspects of digital video
Everything you do with digital video revolves around two words: "Time" and "Compression".
If you "got" the first concept I presented- "Video is nothing more than a series of images synchronized to an audio signal"- you will understand the concept of time. When video plays, its only loyalty is to time. If it is just short of the 1-second mark, a video will jump to that mark by dropping frames to get to that point on the timeline. This explains why, especially during the early days of web video, the videos we watched were so "choppy". As computers processed the video, time was lost and it simply went to the point in the video where it was supposed to be.
You can also see the time concept in Image 1. The audio and the video tracks follow a timeline. The data that makes this happen is laid out in contiguous chunks of information. When the video plays, it is these chunks that either light up the pixels in the video track or represent the audio.
Where video really gets weird is when you concentrate on compression.
Those contiguous chunks of information require enormous amounts of computing power. How enormous? The North American standards body for video is the NTSC (National Television Standards Council) and their standard uses 480 of 525 lines on a non-HDTV machine to display a TV image. If we move that 4:3 ratio to our computers, you will be able to display, say, 640 pixels in a row used to display a 24-bit image. Rather than go through the complex calculations, the number of bytes required by your computer to display that row of pixels would require about .9 mb of memory.
Knowing that each frame of a video is a picture, we can easily calculate the hit on the processor. The NTSC standard is to display 29.97 images per second. Thus your TV is processing about 26.9 mb of information per second (29.97 *.9) and this calculation doesn't include the audio track.
Obviously, pushing that much information through an internet connection is simply not going to work. Think of that information/data stream as water running through a common garden hose. Connect that hose to a fire hydrant and the amount of water (data) will inevitably "clog" the hose. Connect that same hose to the faucet for your garden and the water (data) will cleanly flow. If all you have available to you is a fire hydrant, you will need to attach a device that controls the amount of water shot down the hose. What that device does, therefore, is to reduce the flow of data to a manageable level.
Image 2. A stream is like connecting a water hose to a fire hydrant.
The goal of compression is to reduce the data rate while maintaining image quality. Video destined for the web might need to be compressed at a ratio of 50:1, 60:1 or higher. Lossless compression, where no essential data is lost, can compress by no more than 30:1,. Thankfully we deal with humans and human perception is not only based upon what we actually see and hear but also on what we “think” we see and hear. This means video can use lossy compression and still have acceptable quality. Though you can apply increasing amounts of lossy compression, the higher the value the greater the sacrifice in quality.
The easiest way to reduce the file size of a video is to reduce its physical dimensions and/or its frame rate. For example, reduce the frame rate of a 30 fps video to 15fps and the data is reduced by 50%. Even though Flash can handle a frame rate of up to 30 fps , consider 15 fps as being a good rate for video destined for the web. Even so, there is still a huge issue to face.
Reducing the physical size of the video and its frame rate still does not reduce the bit rate flow of the video. Digital video uses compression to control the flow, and there are two types of compression: "Temporal" and "Spatial".