This is so cool, @itskais! While the training of that AI model would certainly have been done on some beefy machine or in the cloud, I would guess that the actual recommendation could easily happen on the Dwarf (or Duo) itself, or at least on the local machine running the MOD UI.
But OTOH, this would be a perfect way to get some monetization started by offering this as a service: record a bit in the MOD and/or upload some sample to a cloud service and BOOM, you’ll get back a pedalboard ready to play. If that needs a commercial plugin, you might be able to get that right from the store (although I do hope that not every request requires buying another plugin).
But I see how this is some real value-add for any player and for small fee / some credits in the store might allow to bring some funding to the rebooted company.
Anyway, great work, @itskais! (thinking fondly back at my time doing a thesis and doing innovative stuff, while the “real” pros all were working at their day-to-day tasks and didn’t have time for such stuff. That’s the advantage of being young and (somewhat) “cheap” and having enough time and energy at your hands! Good luck with the completion of your thesis!
Hopefully, there’s a company waiting to employ you on the other side of this!
Joining the chorus of praise, this is truly a killer app. The fact that people spend money to buy bundle packs of Helix patches where someone else has already figured out how to replicate someone’s sound proves the commercial viability for something like this—and even if it still needs tweaks at this point, to have it be something that can choose the plugins and settings to get you 90% of the way there is revolutionary. THIS is what you should be showing potential investors because nobody else has something like this. I’m not a guitarist but I’d love to see if it can replicate Satriani’s sound on “Surfing With The Alien”—something super distinctive and bizarre.
This kind of innovation, along with things like the Audio to CV Pitch and the fact that users like @madmaxwell can use the platform to create new neural projects are reasons the company or at minimum the platform has to survive in some form.
I think if the potential of neural network usage in audio dsp like me and others @keyth72@chowdsp are doing is high, in particular for filling the gap with competitors against amp sim models for guitar/bass players, then the usage of neural networks in non-realtime tasks such as preset creation is immense. I’ve realized this since I fall into https://www.thisdx7cartdoesnotexist.com. I think if a genre based preset or user input is too good to be real, a random preset generator (but not really random, with values that are chosen by the network after a training against good or useful presets) would be also useful imho.
This is an exciting technology, the industry and silicon manufacturing is already following this wave, why not audio industry? And specifically, why not open source audio?
In my masters dissertation I used a generative model to recommends the audio plugins of a preset (and conditionally some plugins given already chooses plugins of a preset). In that work , I used Zoom G3 patchea to traning and test, because at that time, we had low Mod Pedalboards and I decided to work with a chain of plugins preset instead of the graph of plugins preset.
I always thought if MOD ui could suggests plugins to use based on the already selected plugins, because we have a lot of options (plugins) and it could be some stressful point for people that haven’t an experimentation felling.
It’s also possible to aggregate the genre information, but it’s necessary to label the dataset…
Thank you for sharing your work, this is interesting of course. This outline that AI can help both at pedalboard level and also at single plugin preset level. This is really the future since the problem of presets is that we sound all the same, but with a thing that generate a sound tailored for you…guys is a dream!
