1
The Ultimate Audio Plugin Developer Checklist
© Jan Wilczek
Table of Contents
Introduction............................................................................................ 3
Mathematics ........................................................................................... 4
Digital Signal Processing ........................................................................... 5
Programming .......................................................................................... 6
C++ ...................................................................................................... 7
Psychoacoustics ...................................................................................... 9
Music Theory ........................................................................................ 10
Basic Audio Concepts and Algorithms ........................................................ 11
Plugin SDK or the JUCE Framework .......................................................... 13
Appreciation of Music! ............................................................................ 14
BONUS: Where to learn this stuff? ............................................................ 15
2
The Ultimate Audio Plugin Developer Checklist
© Jan Wilczek
Introduction
Congratulations on downloading The Ultimate Audio Plugin Developer Checklist!
This list contains all the “knowledge dots” that one should know in order to write
audio-processing code in a form of digital audio workstation plugins.
What to do with this list?
You can print it out or keep as a PDF file.
Begin by ticking the checkboxes corresponding to each concept you are already
familiar with.
Then, look at the remaining points and pick one that you will learn next week.
Next week, pick another one… and so on, until you have checked them all!
Remember that this is a comprehensive list. That means that you do not have to
know everything from this list by heart to develop plugins. But this list can give
you a direction of self-development so that you can become the ultimate audio
plugin developer!
Finally, knowing the “dots” is not enough, no matter how many you collect. It’s
connecting the dots that really matters. This list helps you to discover the dots. I
am here, to help you understand and connect them. But the connections must be
made by you.
Good luck on your audio programming journey and enjoy ticking off the points on
the list!
Best regards,
Jan Wilczek
The founder of [Link]
3
The Ultimate Audio Plugin Developer Checklist
© Jan Wilczek
Mathematics
Fortunately, the mathematics behind audio processing are very well defined and
thoroughly studied. There are lots of materials to learn from! And there is not so
much one needs to learn.
What do you need to know from mathematics to be the ultimate audio plugin
developer?
☐ Complex numbers: complex numbers are used primarily for dealing with the
frequency-domain representation of sound. Real and imaginary parts, complex
roots, and complex exponentials are parts of the most important audio algorithms,
like filters, phase vocoders, and audio effects.
☐ Graphs: if you think about this, the elements of any audio chain and their
interconnections can be modeled as graphs. A hardware example: vocalist-
microphone-preamp-audio interface. A software example: audio file-compressor
plugin-EQ plugin-reverb plugin.
☐ Series: often when we analyze the mathematical form of filters, we use
numerical series. For example, the convolution-an operation that happens in every
filter-is defined as an infinite sum.
4
The Ultimate Audio Plugin Developer Checklist
© Jan Wilczek
Digital Signal Processing
Audio signal processing is a subfield of digital signal processing. Therefore, it’s
beneficial to know and understand the “older brother”.
☐ Signal: in our case audio, be it sound in the air or its representation stored in
a file.
☐ Sampling: the process of storing values of a continuous signal at certain points
in time.
☐ Sample rate (sampling rate): how many samples represent 1 second.
☐ Latency: the time delay between specific events, like recording the sound and
playing it back.
☐ Analog-to-digital conversion: the process of converting an analog signal (like
a sound wave) to a digital representation, usually done via sampling.
☐ Digital-to-analog conversion: the process of converting a digital signal to an
analog one; every playback device does this to play music.
☐ Aliasing: distortion introduced when sampling a signal at too low sampling rate.
☐ Nyquist frequency: maximum frequency we can sample without aliasing at
the given sample rate, i.e., half of the sample rate.
☐ Amplitude spectrum: the amplitudes of sine tones in an audio signal.
☐ Discrete Fourier Transform, Fast Fourier Transform: means to convert an
audio signal from the time domain into the frequency domain.
☐ Phase delay, group delay: how any processing delays particular frequencies.
☐ Digital FIR and IIR filters: means to alter the frequency content of a signal.
☐ Transfer function: a frequency-domain representation of a filter.
☐ z-transform: a mathematical transformation to obtain the transfer function of
a digital filter.
☐ Audio representation: floating-point (float) or fixed-point (integer) and
its trade-offs: there are many ways to represent samples of audio; it’s important
to know their properties.
☐ Phase cancellation: when two tones at the same frequency but mirrored with
respect to the time axis are added they result in silence.
5
The Ultimate Audio Plugin Developer Checklist
© Jan Wilczek
Programming
Developing audio plugins requires knowledge of some Computer Science concepts.
☐ Threading, locks, mutexes, thread-safe queues, atomic variables, race
conditions: these threading-related concepts should be familiar to you because
audio plugins are inherently multithreaded: one thread deals only with audio and
at least one other is responsible for the graphical user interface. These two
domains must interact smoothly and not cause any deadlocks or interruptions.
☐ Operating systems API differences: you’d be surprised how much operating
system-specific most programs are. Even if you use purely C++ features, you may
still run into trouble, trying to target various platforms. Knowing the differences
between them is important to avoid surprises.
☐ Debugging: debugging audio applications is a little bit more challenging than
debugging typical desktop applications. Learn to leverage the tools your integrated
development environment gives you and supply it with logging and writing
intermediate values to files.
☐ User account or license handling: depending on how you want to distribute
your plugin, you may need to learn how to approach users’ access to it. A lot of
companies require product keys and internet connection to run their plugins.
☐ Filesystem access: being familiar with access permissions and OS-specific
differences comes in handy when we want to output files.
☐ State machine: if you think about it, every plugin has at least 2 states (playing
or idle). The concept of the state machine can help you express formal
requirements in code.
☐ Design patterns: I consider the following design patterns important in audio:
Adapter, Observer, Singleton, Flyweight, Facade, Command, State, Prototype,
Abstract Factory, Factory Method, Template Method, Composite.
☐ Pointer-to-implementation pattern: to hide the implementation details from
your plugin’s API.
☐ XML files: to store the presets of your plugin.
☐ Integrated development environment (IDE): to develop, test, and debug
your code. Popular IDEs for plugin development are Visual Studio, XCode, Visual
Studio Code, or CLion.
6
The Ultimate Audio Plugin Developer Checklist
© Jan Wilczek
C++
C++ is the go-to language when it comes to audio programming. Even most
Python libraries that I’ve seen are actually just wrappers around C++ libraries!
Not to mention the JUCE C++ framework, which enforces the usage of C++.
Although often criticized as “out-of-date”, C++ stood the test of time and
continues to be the prevalent language for audio development. As such, it is
important to learn for audio programmers. However, not all of its elements are
equally important for them. The ones that are important are listed below.
☐ std::vector and std::array: standard containers of flexible and compile-time-
fixed size respectively.
☐ Standard library algorithms or where to find them: [Link]
should be your friend.
☐ Memory handling with pointers, new, delete, and smart pointers: audio
buffers come typically in a form of (possibly multi-dimensional) dynamically-
allocated memory arrays. Additionally, using classes and class hierarchies favors
the usage of dynamically allocated objects. Learning how to manage the lifetime
of these objects is crucial to avoiding bugs and memory leaks in the
implementation. You should know std::shared_ptr<> and std::unique_ptr<> as
the back of your hand.
☐ Classes and inheritance: object-oriented programming is the backbone of
almost every piece of commercial software. As the audio code typically runs on a
single dedicated thread, the usage of pure functional programming is discouraged
and, thus, the need for objects arises. If you know the object-oriented concepts in
any programming language, learning those from C++ shouldn’t be a problem.
☐ Basics of templates: templates are a very complex metaprogramming tool.
Fortunately, for audio development, you need only the basics: what they are, how
to write them, and how to specialize them. As audio code must be highly efficient,
templates can provide compile-time polymorphism that can be faster than run-
time polymorphism shipped with class hierarchies.
☐ Threading: C++ 11 introduced threading to the standard library. Learning the
standard components for thread manipulation is crucial for thread-safe and
efficient audio code.
☐ Filesystem access: C++ 17 added filesystem functions to the standard library.
In audio plugins, you need to access the preset files, for example.
☐ Boost library (signals, threading): the boost library provides a plethora of
extensions to the C++ standard library. Signals and threading utilities are ones
that should be known to any audio developer. Do you must use them? No, you can
implement these utilities on your own.
7
The Ultimate Audio Plugin Developer Checklist
© Jan Wilczek
☐ Testing frameworks: there are a few C++ testing frameworks, so you have
plenty to choose from. Personally, I favor GoogleTest because it’s incredibly easy
to integrate and use.
8
The Ultimate Audio Plugin Developer Checklist
© Jan Wilczek
Psychoacoustics
Psychoacoustics is the science of human sound perception. If you’ve never
explored this field, you’d be surprised of how much we actually hear of what’s not
present in the sound or not hear of what is. A fascinating subject on its own, it has
a few important concepts that audio programmers should be familiar with.
☐ Precedence effect (Haas effect, law of the first wavefront): the
impression that sound is coming from the direction we first heard it coming from.
☐ Masking in time and in frequency: under certain conditions, certain sounds
can hide other sounds from our perception. It’s important to understand when and
why that happens.
☐ Hearing threshold: the quietest sound we can hear at the given frequency.
☐ Loudness perception: how we perceive loudness is frequency-, age-, subject-
and condition-dependent. It’s a complex subject matter but well worth
understanding.
☐ Equal-loudness contours: the sound levels of tones at particular frequencies
that are perceived as equally loud.
☐ Pitch perception: how humans perceive the pitch of a given sound. The
perceived pitch is not equivalent to the fundamental frequency although the
fundamental is the major factor contributing to it.
☐ Combination tones: additional tones we can hear while listening to two
separate tones.
☐ Critical bandwidths: “frequency bins” in which multiple tones are perceived
as one by our brain.
☐ Just noticeable difference: a value that tells us how much a physical quantity
must change to be noticed by a person. For example, how much louder a sound in
decibels must be, until we perceive it as actually being louder.
9
The Ultimate Audio Plugin Developer Checklist
© Jan Wilczek
Music Theory
Although formal music theory is beyond the scope of most audio programmers,
some basic concepts are crucial for music processing as we know it from plugins.
For example, what would you auto-tune to if not for the equal-tempered scale?
☐ Equal-tempered scale: how are the notes spaced on the piano keyboard and
how to calculate the difference between frequencies in cents, semitones, tones,
and octaves.
☐ Fundamental frequency: the difference between harmonics that highly
contributes to the perception of pitch.
☐ Harmonics, overtones, and partials: harmonic tones in the spectrum of a
sound, harmonics without the fundamental frequency, and all tones within a
sound, respectively.
☐ The MIDI protocol basics: what are the note numbers and which control
information can be passed with MIDI messages. On the API level, one does not
have to go into the implementation details; the basic meaning of MIDI messages
suffices.
☐ Types of noise: above all, white and pink. These can be used as either musical
effects or test signals for testing and debugging your system.
10
The Ultimate Audio Plugin Developer Checklist
© Jan Wilczek
Basic Audio Concepts and Algorithms
If you jumped here right from the table of contents, I get you! I also wanted to
start writing audio algorithms right away!
All previous sections lay the foundation for this one. Of course, you could code the
concepts below without understanding them but then you wouldn’t be able to
extend and tweak them.
☐ Audio buffer: audio plugins process samples in a block-wise fashion. It’s
important to understand how audio buffers behave and what are the implications
of their size.
☐ Clipping: exceeding the [-1, 1] range, why it happens, how to avoid it.
☐ Musically-useful filters: whatever your plugin does, it probably uses a filter
somewhere. Understand basic filter types and how to implement them.
☐ Automatic leveling: when mixing multiple channels of audio, we must be sure
we don’t clip them and that we present a coherent behavior to the user.
☐ Parametric EQ filters: because of its ubiquity, I consider it beneficial to
understand how EQ works.
☐ Envelopes: the pattern of change of some parameter (amplitude, cutoff
frequency) over time. ADSR and ADBDR are the most popular types.
☐ Generators and basic waveforms: be it for sound synthesis or modulation,
it’s important to know what are the basic waveforms and how to generate them.
This includes sine, triangle, sawtooth, square, and pulse.
☐ Sound synthesis algorithms: wavetable, subtractive, additive, amplitude
modulation, ring modulation, frequency modulation.
☐ Source-filter model: a lot of sound generators can be modeled as consisting
of a source and a modifier (filter). For example, that’s how human voice is often
modeled.
☐ Formants: peaks in the amplitude response of a filter.
☐ Comb filtering: the effect of summing a signal with its slightly delayed version.
Can be used as an effect but can also be dangerous when downmixing from stereo
to mono.
☐ Reverberation: the effect of decaying echoes of the initial (direct) sound that
give an impression of space.
☐ Compression: changing the dynamic (amplitude) range of the audio signal.
☐ Filter resonance: the peak in the filter’s amplitude response around the cutoff
frequency and how to control it.
11
The Ultimate Audio Plugin Developer Checklist
© Jan Wilczek
☐ Distortion: a surprisingly useful musical tool that not only can make the sound
more harsh but also more lively.
☐ Phaser: summing the output of time-varying allpass filters with the direct path
creating varying notches in the output spectrum.
☐ Flanger: time-varying comb filter creating equally-spaced varying notches in
the output spectrum.
☐ Amplitude panning and time-delay panning: apart from track panning, it is
important for stereophonic effects (like stereo ping-pong).
☐ Head-related transfer functions: ear-specific transfer functions of the
incoming signal; can help create realistic vertical panning effect.
☐ Audio formats: audio files come in a variety of formats. Learn what are the
basic differences between them. The most important to understand is the .wav
format. Writing .wav writer and reader utilities is a good audio programming
exercise!
☐ Specific algorithm you want to implement: the core algorithm you are
implementing in your plugin. What will it be?
12
The Ultimate Audio Plugin Developer Checklist
© Jan Wilczek
Plugin SDK or the JUCE Framework
The final piece of the puzzle is the interconnection between your audio code and a
digital audio workstation. That’s what this section is about.
☐ The concept of a plugin and a plugin host: what is a plugin? How can we
interact with it? What is a plugin host?
☐ How a digital audio workstation works: you should be familiar with the
track layout, routing, mixing, and the FX chain.
☐ Plugin formats: their types, where are they used, what are the superficial
differences between them, and their inner classification into instruments, effects,
etc. The most popular formats are VST, AAX, and AU.
☐ Specific plugin format API: only if you plan to integrate your plugin with a
specific format manually.
☐ JUCE framework: the core assumptions behind JUCE, its classes, API, utility
functions, and the flexibility of abstraction it provides. JUCE is a huge topic on its
own but you don’t have to know it all; to start, just a few classes suffice, for
example, the plugin processor, the plugin editor and the
AudioProcessorValueTreeState.
13
The Ultimate Audio Plugin Developer Checklist
© Jan Wilczek
Appreciation of Music!
It’s hard to imagine working with audio and disliking the domain at the same time…
As one of my mentors, professor Meinard Müller, said: „Music processing is one of
the rare domains where researchers can actually like their datasets.”
14
The Ultimate Audio Plugin Developer Checklist
© Jan Wilczek
BONUS: Where to learn this stuff?
Writing audio-related code is challenging and exciting at the same time.
What are the plugin projects that you want to undertake?
Do you want to build a juicy synthesizer?
Or maybe you have an idea for a powerful reverb that will have some unique twist
in comparison to other reverbs?
Either way, the list above will be of great help to you in discovering the pieces of
the puzzle, the “knowledge dots”.
And how to learn these knowledge dots?
You can google around, scrambling for pieces of information on forums and general
blogs.
You can buy expensive books and browse through thousands of pages to look for
the information you need, unsure whether it’s there or not.
There are also online code documentation pages and 2-year-long university
degrees…
Or you can simply… stay on my email list.
Every week you will be able to understand one dot of the above and tick one
checkbox, all that effortlessly, easily, and in much shorter time than with the help
of textbooks.
Thanks to my weekly content, you will be able to develop your plugins faster,
cheaper, and focus on what matters the most to you: the core of your plugin
processing.
As a subscriber of my newsletter, you will also improve your coding skills because
I often share helpful insights from the field of audio development that you can’t
get anywhere else.
If you have any questions, you can always reply to my emails. That will also guide
me as to which articles and videos would be most useful to you.
Do you want to connect the dots and become an audio programmer?
Let me help you get there.
15
The Ultimate Audio Plugin Developer Checklist
© Jan Wilczek
