13.1: Formats/Compression

Codecs

A "codec" is made up from 2 different words:

COmpress and DECompress = CODEC

Codecs translate the analog video and audio signals into numerical formulas that are read and used by computer editing applications, to numerically process these light and sound signals, (as well as still images), into digital information.

For example, a couple of codecs that are optimized for camera recording, are H.264 and ProRes.

Some image codecs you’re familiar with are JPEG, and PNG.

It’s important to understand how these codecs influence the edit resource files in terms of quality and size, and how they affect the speed and storage of the hardware used. In the best of all possible worlds, we want to have:

• the highest quality files
• the smallest size files to keep our storage needs in check
• the fastest processing speeds for editing.

Unfortunately, the way our systems are built, we can only ever have 2 of these 3 situations at a time. As we’re set up, if you want quality and speed, the files generated are large. If you need speed and smaller files, then the results are of poorer quality. So, understand, that you have to settle for 2 best choices, while compromising the third. You are always playing the balancing game...

You should be familiar with these many codecs, in order to make the best choices in your role as an editor, since you’ll be seeing different codecs used for filming, different codecs for editing, and still different ones for exporting and distribution.

When you begin to edit, you’ll want to change the camera codecs to ones more suitable for editing. You'll want to choose an editing codec that will give you a good, clear picture, yet be small enough in size, to edit quickly and efficiently. This process of codec conversion is called "Transcoding," and most current editing applications create these transcoded files quite easily. This process is referred to as "optimizing" for the edit.

Compression

When you compress files, you’re making them smaller, so that they take up less space and can be processed faster. Unlike a corset that squeezes in a waist to make it smaller, you can’t squeeze a digital image. The only way you can reduce its size is to take away some of the number of pixels used to display that image or frame, while using the appropriate codec.

This "removal of pixels" will affect your image, making it less sharp, or even blurry if overdone. So be careful when editing, not to magnify an image more than a very few percentage points over 100% – because this will result in fewer pixels in the image – translating into an unfocused look.

Types of Compression

With professional editing applications, it’s possible to "optimize" your working files, by transcoding them for faster working speeds, and after the edit is done, transcode the edited files back into their original quality. The different codec formats used can be divided into 3 main categories:

Native codecs usually refer to camera recording codecs. These get the recording done in the most efficient way. Most editing apps will be able to use these native, camera codecs directly for editing, but they’re not set up to do a speedy, high quality job. It’s therefore best to transcode these native camera codecs into ones better suited for editing.

Examples of camera, native codecs are:

• H.264
• H.265
• JPEGs

These native codecs compress files using a system called INTERFRAME or GOP (stands for "group of pictures") compression. This method works by eliminating and re-using information from neighboring frames, when there is no change from frame to frame in the image, and thus is able to save a great deal of storage space and record very efficiently.

However, in doing so, bits of information are often thrown out for the sake of file size, giving these files a "lossy" connotation. Lossy files have irretrievable information lost or thrown away during compression, where the intent is that this lost information is either not noticeable, or less important than the reduction in file size.

Optimized codecs are those especially designed for editing. They deal with the editing operations, from basic editing to special effects, compositing, distribution etc., in a speedy and visually high quality way.

A good example of optimized codecs is the Pro Res family of codecs from Apple. They come in many flavors, from the lowest to the highest resolutions, and are used by all professional editing applications. They start from Pro Res Proxy (lowest) to different Pro Res HQs with the ability to edit in the newest 10- to 12-bit resolutions and deal with multi-streaming performance as well.

They do the most efficient job, because they work with a form of compression called INTRAFRAME compression, where the information in each individual frame is compressed individually, and not dependent on any other frames in the file. This results in the highest quality resolution files for editing and efficiency in all post production tasks. The cost of the high quality imagery is of course, its heavy file size - all those extra pixels require enormous amounts of storage.

This form of compression is also referred to as "lossless" compression, where images are compressed without losing information.

Latest production recording now includes RAW footage, where everything about the image is now recorded. There is actually no compression. This abundance of information about the camera subject, allows for the highest quality amount of image manipulation. But again, this calls for unlimited amounts of storage space, to the point where even terabytes (TB) are insufficient, resulting in formidable production challenges and cost.

Going from one extreme to another, we come to PROXY resolutions. These are codecs which shave down the amount of information in an image, resulting in files that require the least amount of storage space. This makes them efficient, speedy tools for editing, requiring the lowest amounts of storage. Yet, proxies are visually high enough quality to maintain a good idea of what the final, high resolution, transcoded edit will look like. These lower resolution files are designed specifically for efficient editing, with optimum system performance. Also, most professional editing apps will allow you to easily and quickly switch back and forth between proxy and higher resolutions, in order to check on your final output and distribution look.

An example of a proxy codec is Pro Res Proxy.