P&S Theory Part #1


Let me challenge you with something - there is absolutely no difference between picture and sound. Picture and sound are exactly the same.

Hear me out. Even though we may perceive picture and sound in drastically different ways, imagery and audio are recorded in exactly the same way. This blog post is the first in a series of two. This blog post will start at the beginning, exploring the theories and concepts behind light and sound. Photons, sound waves, anatomy diagrams - this blog post should feel like a compressed high school science class. Take pride in the fact that by reading this you will become a master of general knowledge and be admired by your local pub quiz team. It is also a great primer for the onslaught of technical information you will succumb to in the next post.

The second blog post in this series will move away from the acoustic domain and will dig into the realm of analogue and digital - covering camera and sound acquisition, camera sensors and a bit of colour science.

Personally, I have struggled to fully understand these theories and concepts for a long time, so I hope that these two blog posts will be a convenient place to find all the relevant information grouped together in an easy-to-read, digestible format.

This is an opportunity to focus on the raw basics, so let's get into it.

WHAT IS SOUND? (3)

Put simply, sound is the collision of molecules. These molecular collisions travel outwards from an audio source and are often audible to the human ear.


For example, the bell ring above is disturbing the molecules around it in a set pattern, varying in air pressure as it ripples outwards. Because of this, it can be represented as a (simplified) sound wave:


The amplitude, or how tall the sound wave is, reflects the perceived loudness of the sound. The frequency, or wavelength, is the rate at which the air pressure changes. The rate at which this changes determines the pitch of the sound. The higher the frequency (shorter the wavelength), the higher the pitch. The lower the frequency (longer the wavelength), the lower the pitch.


This handy Flat 12 mix'n'match smorgasbord will test your knowledge. Who's voice is higher - Mitchell's or mine?

We measure the frequency/wavelength in cycles per second, in a unit of measurement called Hertzs. The frequency range that a young, healthy human can hear starts at around 20Hz, the lower pitched sounds, to higher-pitched sounds of around 20kHz (20,000Hz). We measure the amplitude (height of the wave) in Decibels, or dB. I’m not going to describe decibels in great detail, because frankly, the measurement is too damn complicated (it relates to a logarithmic relationship with pressure).


The human ear.

Finishing up this section - how do these sound waves actually interact with our weirdly shaped ears?

Passing through the outer ear, sound waves enter the ear canal and vibrate the ear drum. The drum passes these vibrations on to the three tiny bones in the middle ear (malleus, incus and stapes) that act as little amplifiers - increasing the signal. This amplified signal goes to the snail-like object in the inner ear that is presumably pronounced Coachella. The cochlea is split-in-half by a thin elastic membrane, the bottom half filled with liquid. The incoming sound vibrations cause this liquid to roll and move, causing tiny hairs sitting on the top side of the membrane to bend and sway, hitting a surface above. This hair cell collision releases chemicals and sends electrical signals to the brain via the auditory nerve. (1) NB: High-frequency sensitive hair cells live near the wider side of the cochlea, while the low-frequency sensitive hairs cells can be found closer to the centre. If these cells are damaged, they cannot regrow and will result in a permanent loss of hearing. (1)

WHAT IS IMAGERY? (2)

Audio, check. So, changing tack for a minute - what happens when we open our eyes each morning? When we're driving to work, watching Blade Runner 2049 or painting a picture? The answer is we’re seeing a giant magic trick, orchestrated by our brain. So let's pull the rabbit out of the hat.


Light is the smallest unit of energy that can be transported: a photon. Photons are very small, indivisible particles that travel (understandably) at the speed of light.

Let me hit you with a slightly confusing science fact: scientists define a photon as a particle and a wave at the same time, labeling it as the wave particle duality theory. Confusing, eh?

The most important thing to take away here is that photons can be described as a wave - just like sound.

So let's clarify something else. When we talk about light, we are normally discussing visible light - a small section of the electromagnetic spectrum that humans can perceive with the human eye.


Visible light is only a small sliver of the electromagnetic spectrum.

The spectrum encompasses low frequency waves such as radio waves, microwaves and infrared while also extending to higher frequency waves such as ultra-violet, x-rays and gamma waves. These lower and higher frequencies/wavelengths can be extraordinarily extreme. Radio waves can range up to 100 kilometres in length, while gamma rays are usually under 10 picometres (which is smaller than a Hydrogen atom).

So where does visible light come into play?


Light's colour can be defined by its frequency/wavelength.

The myriad of rainbow colours that we see everyday sit in the middle of the spectrum, with frequencies/wavelengths measuring from between 400nm-700nm (nanometres) in length. This is roughly around the size of bacteria.

NB: Why can't we see any other wavelengths of the electromagnetic spectrum? Well, visible light is the only slice of the electromagnetic radiation pie that transmits well through water. The sea is where most organism's eyes originally developed from millions of years ago. That's just how our eyes happened to evolve!

So, we have gauged that the frequency/wavelength of light defines the colour. As you may be able to see, this is where light and sound are starting to come together. Light and sound can both be measured through the characteristics of their wave.


The human eye

So lastly - how do we see these wavelengths of light? Firstly, photons flood into our eyes - passing through a protective outer-layer called the cornea (a). The muscles of the iris (b) control and limit the amount of light that passes through the lens (c). Light passes through the pupil (e), through the lens (c) to be focused and directed onto the thin wall of light-sensitive receptors lining the back of the eye, the retina (g). From there, the light information is translated into electrical signals and sent down the optic nerve (h) to the brain. (4)

ARE YOU READY FOR MORE?


Part two of this blog post is out now and it digs deeper into the relationship between picture and sound in the analogue/digital domain, covering topics such as resolution, quantization, sampling rate and bit-depth.

REFERENCES How Do We Hear?

www.nidcd.nih.gov/health/how-do-we-hear (1)

What is Light?

www.youtube.com/watch?v=IXxZRZxafEQ (2)

Digital Audio Foundations

www.lynda.com/Acoustics-tutorials/Welcome/383529/486990-4.html?autoplay=true (3)

The Science of Vision: How Do Our Eyes See?

www.independent.co.uk/life-style/health-and-families/features/the-science-of-vision-how-do-our-eyes-see-10513902.html (4)

This is the first half of a two-part series on Picture vs.

Sound. If you have any questions you'd like to ask, flick me an email at lukerosspost@gmail.com