Working with Multimedia

What is Sound?

Pressure waves of air - bells, clapping, speakers - push the air to create waves

How is Sound recorded?

Typically microphones have a membrane that vibrates to capture the sound either as analogue or digital

Digtial sound recording takes snap shots of the sound - a technique known as sampling

Sample Rates

Sample rate is the number of samples per second, measured in hertz (Hz) or kilohertz (kHz)

A rate of 44,100 samples per second is more commonly referred to as 44.1 kHz

Amplitude and Frequency

Amplitude - the height of the wave (the volume)

Frequency - the periodic vibration (the pitch)

Human ear 20 to 20,000 Hz - lower frequency 'felt' rather than heard

44,100 Hz is a common sampling frequency - ie double Human range

Audio File Formats

WAV

Waveform Audio File Format - uncompressed audio format

MP3

Popular lossy audio compression format

AAC

Advanced Audio Coding format, designed to be the successor format to MP3, with generally better sound quality

Ogg

Vorbis free and open source audio format

MIDI (Not Digital Audio)

Symbolic representations of notes/duration/velocity not a recording of the sound

Software

Free Software: Audacity

Adobe Software: Audition

<audio>

src

Specifies the URL of the audio to play. The browser will use the first recognized format

controls

Specifies that controls will be displayed, such as a play button.

autoplay

Specifies that the audio will start playing as soon as it is ready.

loop

Specifies that the audio will start playing again (looping) when it reaches the end

preload

Specifies that the audio will be loaded at page load, and ready to run. Ignored if autoplay is present.

Syntax of <audio>

<audio controls="controls">

	<source src="jquerysong.mp3" type="audio/mpeg" />
	<source src="jquerysong.ogg" type="audio/ogg" />
	<p>Sorry your browser does not support audio playback</p>
</audio>

The Persistence of Vision

The persistence of vision theory states that the human eye holds a still image for a fraction of a second, remaining on the retina long enough to blend with the next image.

Film displays 24 still images every second

This is called 24 frames per second or FPS

Eadweard Muybridge

Analog and Digital

Analog system directly records the variations of the video and audio signals - has a tendency to deteriorate when copies made

Digital system regularly samples the waveforms and converts them into numerical (binary data). Copies can be made without loss of quality and easily stored electronically

Progression from Film > Video > DVD > BlueRay > 4K

In the Olden Days

Cathode Ray Tubes (CRT)

Electron Gun and a Screen

Screen covered in phospourous dots called pixels

Colour CRT use 3 electron guns - red, green and blue

Signal received causes gun to fire electron beam at the screen to light it up

Width of the screen aspect ratio of 4:3

Plasma Screens

Screen made up of pixels that each contain a red, green and blue phospourous cells

Rather than a beam, an 'address electrode' applies electric charge to each cell based on the video signal

Interlaced Scanning

Beams scans odd lines first

Displays half a frame (called a field)

Beams scans even lines next

Two fields create the completed image

Progressive Scanning

Scans or paints each line successively (no fields)

HDTV (High Definition TV) use more lines either 720 or 1080

Ultra HD, 3840 × 2160

Aspect Ratios

The width to height ratio of the frame is called the Aspect Ratio

HDTV use aspect ratio of 16:9 compared to Standard 4:3

16:9
Standard aspect ratio for high definition television

4:3
Standard aspect ratio for standard definition television (NTSC/PAL) and Digital Television (DTV)

'Old School' Example in 4:3

Vimeo Example in 16:9

Letterbox

When viewing 16:9 content on 4:3 screen black bars at the top and bottom of the picture - known as Letterboxing

Anamorphic Video

Horizontally squeezed video 16:9 video to fit 4:3 screen - images look tall and thin

High Definition TV

Files are Big!

Uncompressed 1080i high-definition video recorded at 60 frames per second uses 410 gigabytes per hour of video

74 Minutes of CD quality audio requires 680MB

Files need to be smaller for digital delivery - thus compression

Video Containers

Lots of different formats - AVI, FLV, F4V,MOV, MP4, OGG, WebM

In reality these are 'containers' / 'ZIP' files that define how the data is stored not what kind of data stored

Container can hold video, audio, caption and meta data

Codecs

Codec is short for "compressor-decompressor" or, more commonly, "coder-decoder".

A file format is a container that holds the data ie MOV.

A codec describes the format of the media ie H.264.

Video Codecs

The Compression algorithm used to store the array of pixels

Audio Codecs

The Compression algorithm used to store the waveforms of the recorded audio

Lossy vs Loseless

Lossy:

A lossy codec will reduce the quality of the original input to reduce file size.  JPGs and MP3s are lossy.  In video the H.264 codec is lossy.

Lossless:

A lossless codec will retain the quality of the original and as a result will be a larger file size.  In video the JPG-2000 codec is lossless and therefore popular for archiving files as their best quality. Also AVI and Quicktime using PNG

Encoding Terminology

Bitrate - the rate at which bits (data) are transferred

Frame Rate - fps - frames per second

Keyframe

keyframe interval

Constant Bit Rate CBR

Variable Bit Rate VBR

Two Pass Encoding

Bitrate

File size = bitrate (kilobits per second) x duration

Bitrate is a measurement of the number of bits that are transmitted over a set length of time

Overall bitrate is a combination of video and audio streams with the majority coming from your video stream.

How to reduce file size

Reduce the bitrate

Reduce the length

Reduce the resolution

Reduce the frame rate

... whilst all the time trying to maintain quality

Frame Rate

Lower the frame rate in even increments (24 to 12, 30 to 15) to avoid de-syncing of the video and audio streams.

Variable Bit Rate VBR
vs
Constant Bit Rate CBR

Constant Bit Rate:
Choose a target bit rate and the codec uses whatever amount of compression is necessary to achieve it
- strives to maintain the bit rate of the encoded media

Variable Bit Rate:
Efficiently compress the data dipping into the minimum bitrate when there is little to no motion/change on screen
- strives to achieve the best possible quality of the encoded media

Two Pass Encoding

Single Pass Encoding

File is encoded by one pass through.

Two Pass Encoding

File is analyzed in the first pass, encoding takes place in second pass.

Two pass can give you smaller files with better quality at lower bitrates.

At high bitrates, two pass or single pass can have the same quality.

Encoding Software

Premier Pro and Adobe Media Encoder

Render Queue in AfterEffects - Export -> Add to Render Queue

Miro Video Converter

Firefogg for Firefox - OGG, WEBM

Handbrake

File Size vs Quality

Direct correlation

Things to try

There is no magic formula. Experiment.

Original 1280 x 720 or 1920 x 1080

Encore for 'medium' web page
"stats for nerds" in youtube - 640x360

Video File Formats

MPEG-4/H.264

Commonly used video compression format - mp4, mpeg

Ogg/Theora video format

Vorbis free and open source lossy format - Firefox, Chrome support

WebM

Royalty-free format for use with HTML5 video - no Safari Support

H.265

Emerging successor to H.264

Video File Types

Issues around browser support of different codecs mainly resolved with MPEG-4/H.264 dominant over Ogg/Theora and webm.

Non HTML5 fallback required for Internet Explorer 8

Youtube

Upload and Embed

Embed options

* Anamorphic Frame Size 720 x 480 3:2 to 16:9 square pixels of 853 x 480

Embeds with an iframe ie

<iframe width="560" height="315" src="https://www.youtube.com/embed/br7sxRPqMZU" frameborder="0" allowfullscreen></iframe>
    

<video>

src

The URL of the video to play

controls

If present, controls will be displayed, such as a play button.

autoplay

If present, then the video will start playing as soon as it is ready

loop

If present, the video will start over again, every time it is finished.

poster

Placeholder image that is displayed when the video is downloading or before the user hits plays the video

preload

If present, the video will be loaded at page load, and ready to run. Ignored if autoplay is present.

width / height

Sets the width and height of the video player

Syntax of <video>

<video width="640" height="360" controls>
    <source src="myvid.mp4" type='video/mp4'>
    <source src="myvid.ogv" type='video/ogg"'>
    	<p>Sorry your browser does not support video playback</p>
    	<!-- Optionally add Flash fallback -->
</video>

Video Poster Images

First frame of the video not always the most flattering or has no content

Poster attribute allows for a selected image to appear at the start

Video editing software can be used to export a suitable JPG