First, the test that passed with the feature deleted
PULSE ships with tests/dsp_test.cpp. Its last check, as first written, was meant to prove that the parameter smoothing works: jump Depth from 0 to 1 halfway through a second of audio and make sure no single sample jumps by more than 0.01. Here’s the output from the real build:
$ c++ -std=c++17 -O2 -ISource tests/dsp_test.cpp -o dsp_test && ./dsp_test
PASS no NaN/Inf (0.0000)
PASS peak never exceeds input level 0.5 (0.4998)
PASS trough matches 0.5*(1-depth) (0.0115)
PASS no NaN/Inf (0.0000)
PASS peak never exceeds input level 0.5 (0.4971)
PASS trough matches 0.5*(1-depth) (0.2699)
PASS no NaN/Inf (0.0000)
PASS peak never exceeds input level 0.5 (0.5000)
PASS trough matches 0.5*(1-depth) (0.5000)
PASS no click when depth jumps (0.0000)
RESULT: ALL PASSThen we deleted the smoothing. One line in Tremolo.h, smoothCoeff = 1.0, so Depth snaps to its new value in a single sample. That should click loudly. The test still printed PASS no click when depth jumps (0.0000).
The reason is worth knowing, because AI-written tests fail this way all the time. The test ran the LFO at 0.5 Hz, so at exactly 0.5 seconds the LFO sat at its peak. At the peak the tremolo’s gain is 1.0 whatever Depth is set to, so the jump happened at the one moment it couldn’t be heard. The fix moves the LFO to 1.5 Hz, where 0.5 seconds lands on the trough, and the same jump drops the gain from 1.0 to 0.0:
with smoothing: PASS no click when depth jumps (0.0005)
smoothing deleted: FAIL no click when depth jumps (0.5000)That is the rule for every test an assistant writes for you: break the feature on purpose and make sure the test goes red. If it stays green, the test is decoration. It takes two minutes, and it’s the only way to know a check is actually checking.
Check 1: a DSP test with no JUCE in it
Keep the sound-making code in a plain C++ header that knows nothing about JUCE or the host. PULSE’s Tremolo.h is 61 lines: prepare(), setParameters() and nextGain(). Because it has no dependencies, the test builds with a bare compiler in about a second, with no CMake, no plugin format and no DAW involved.
What to test is the same for most effects:
- No NaN or infinity, ever, on any setting. One NaN in a plugin can silence the whole track until the user reloads it.
- Level stays where you expect. PULSE never boosts, so the peak must stay at or under the 0.5 input.
- The math matches the spec. At Depth 100 % the trough should be 0.5 × (1 − 1) = 0, and we measured 0.0115 (the 10 ms window catches a sliver either side of the dip). At Depth 50 % it should be 0.25, and we measured 0.2699.
- No clicks on parameter jumps, tested at a moment where the jump changes the sound.
If you started in the browser, write this test to repeat the browser’s offline test on the same inputs. When both versions give the same numbers, you know the port is right.
A second deliberate break shows what the NaN check is for. We made prepare() ignore the host’s sample rate, the way a forgotten prepareToPlay call behaves, and the test went red in six places, with a peak of inf. In a DAW, a bug like that usually means silence or a burst of noise.
Check 2: pluginval
pluginval is a free tool from Tracktion that loads your plugin the way a host does and then pushes it around: it opens and closes it cold and warm, processes audio at different sample rates and block sizes, saves and restores state, automates every parameter from different threads, and finally fuzzes the parameters with random values. It also runs auval for Audio Units and Steinberg’s VST3 validator where those are available. Most crashes that users would otherwise find for you, it finds first.
This is the exact command and result from the PULSE build in the kit (pluginval 1.0.4, JUCE 8.0.15, Linux, run under a virtual display):
$ pluginval --strictness-level 10 --validate-in-process \
--validate build/Tremolo_artefacts/Release/VST3/Tremolo.vst3
pluginval / Open plugin (cold)...
pluginval / Open plugin (warm)...
pluginval / Plugin info...
pluginval / Plugin programs...
pluginval / Editor...
pluginval / Open editor whilst processing...
pluginval / Audio processing...
pluginval / Non-releasing audio processing...
pluginval / Plugin state...
pluginval / Plugin state restoration...
pluginval / Automation...
pluginval / Editor Automation...
pluginval / Automatable Parameters...
pluginval / Parameters...
pluginval / Background thread state...
pluginval / Parameter thread safety...
pluginval / auval...
pluginval / vst3 validator...
pluginval / Basic bus...
pluginval / Listing available buses...
pluginval / Enabling all buses...
pluginval / Disabling non-main busses...
pluginval / Restoring default layout...
pluginval / Fuzz parameters...
SUCCESSHow to read it when something fails, grouped by what usually caused it in AI-written code:
| Failing stage | Usual cause in vibe-coded plugins | Where to look |
|---|---|---|
| Open plugin (cold / warm) | Work done in the constructor that should wait for prepareToPlay, or a file or network load at startup | Processor constructor |
| Editor, Open editor whilst processing | The UI reads audio-thread buffers directly, or a WebView page path that only exists on your machine | Editor constructor, resource paths |
| Audio processing, Non-releasing audio processing | Buffers sized once for 512 samples, state not reset when the sample rate changes, NaN from a divide by zero | prepareToPlay, processBlock |
| Plugin state, Plugin state restoration | Empty getStateInformation, or parameters renamed between builds | State save and load |
| Automation, Parameter thread safety | Parameters read with locks or strings on the audio thread, or the DSP keeping its own copy that never updates | How processBlock reads parameters |
| Fuzz parameters | A setting combination nobody tried: zero delay time, feedback at 1.0, Q at its extreme | Parameter ranges and DSP edge cases |
Start at strictness 5, which pluginval’s own documentation treats as the baseline for working in most hosts, and work up. The PULSE build passes at 10, the maximum. Useful flags when you’re debugging: --skip-gui-tests to rule the editor in or out, --timeout-ms if a slow test gets killed, and --repeat with --randomise to shake out failures that only happen sometimes. pluginval exits with a non-zero code when a test fails, so you can drop it into a build script and stop shipping broken builds by accident.
Check 3: auval, if you build Audio Units
Logic Pro and GarageBand only load AUs that pass Apple’s validator. pluginval runs it for you on a Mac, but it’s worth knowing the direct command, because Logic’s error messages say very little. auval takes three four-character codes: the type (aufx for an effect, aumu for an instrument), your PLUGIN_CODE and your PLUGIN_MANUFACTURER_CODE from CMakeLists.txt. With the placeholder codes the PULSE project ships with:
auval -v aufx Trm1 McmpChange both codes to your own before you share a build. Two plugins with the same codes will fight, and one of them won’t show up. If you change the codes and macOS keeps validating the old build, it is using a cached copy. killall -9 AudioComponentRegistrar clears it, and Logic’s Plugin Manager can then rescan. A note on what we checked: PULSE was built and validated on Linux, which has no auval, so on a Mac you run this step yourself.
Check 4: ten minutes in a real DAW
Validators prove the plugin won’t crash. They don’t prove it sounds right in a session. These are the checks that catch what the tools can’t, in the order we run them:
- Save, close, reopen. Set every knob somewhere unusual, save the project, quit the DAW and reopen it. Every value should come back exactly.
- Change the buffer size from 64 to 1024 samples while audio plays. Rate-based effects that count in blocks rather than samples change speed. Delays with buffers sized at startup crackle.
- Change the sample rate from 44.1 to 96 kHz. An LFO that doubles in speed, or a filter that shifts pitch, means a hard-coded 44100 somewhere.
- Bounce offline and compare it to real-time playback. If they differ, the plugin is reading the clock or doing work in the UI that the audio depends on.
- Automate a knob with a fast, steep curve. Listen for zipper noise. That’s the smoothing check from step one, now with your real parameter path.
- Load three instances on three tracks. Shared static variables show up as instances changing each other’s settings.
- Bypass it and put it on a mono track. The level shouldn’t jump on bypass, and mono shouldn’t crash or come out of one side only.
When a check fails, what to tell the AI
“It’s broken, fix it” gets you a rewrite and new bugs. Give it the failing stage and the constraint instead:
pluginval fails at "Plugin state restoration" (strictness 5, output below).
Fix only getStateInformation / setStateInformation.
Don't rename parameter IDs, don't touch processBlock or the DSP header.
After the fix, tell me which line caused the failure and why.
[paste the pluginval output]Then run the same check again, and run the DSP test, so you know the fix didn’t break the sound. The troubleshooting guide has the symptom-by-symptom version for clicks, silence and runaway feedback.
Before you send it to anyone
- DSP test passes, and you’ve seen it fail by breaking the feature on purpose.
- pluginval passes at strictness 5 or higher, for every format you ship.
- auval passes, if you ship AU.
- Session save and reload, buffer size, sample rate, offline bounce and three instances all check out in a DAW.
- Your own company name, bundle ID and four-character codes, not the template’s.
Keep going
- Learn hub, the full path map
- Web Audio to JUCE parameter mapping
- From browser prototype to VST/AU
- When your vibe-coded plugin breaks
How we tested: the DSP outputs come from tests/dsp_test.cpp built with the system C++ compiler (C++17, -O2) against the PULSE Tremolo.h, unchanged and with the two deliberate breaks described above. The pluginval log is the one included in the kit, from pluginval 1.0.4 against a JUCE 8.0.15 VST3 build on Linux.