Web-assembly and WebGL usage in modguis (C/C++ for modguis)

Just updated the test link for those wanting to try, with some fixes regarding high-dpi and adjusting the size a bit.
This is an odd one to package, we will need to figure out the best way to deal with wasm binary assets…
For the moment Dwarf users with a Linux or macOS system can install wolf-shaper as described on the first post:

curl https://falktx.com/data/wasm/wolf-shaper.lv2-dwarf.tar.gz | base64 | curl -F 'package=@-' http://192.168.51.1/sdk/install

One cool little extra we get by default, if there are 2 web browsers open with the MOD web gui, the state of the wolf-shaper is kept in perfect sync!

8 Likes

I am having some difficulties in safari with this version.

The warning about the mouse cursor being hidden is displayed at the top which moves the mod GUI down, even more annoying if you are zoomed in this unzooms it!

2 Likes

yeah safari makes this super weird to use…
the mouse pointer thing is to allow to keep mouse drag while it moves outside of the GUI bounds (otherwise the GUI stops receiving events)

maybe there is a way to ask a permission and keep it always on? they surely must realize this behaviour is quite obnoxious…

1 Like

I have had a look to see if I can find anything in Safari to turn this off, the one way I can see is to just not use safari.

On an iPad over wifi it works perfectly!

1 Like

You might want to check out the awesome https://stackblitz.com/
Amazing stuff. Currently a full nodejs web dev env in a browser but maybe more could be added to make a cool env for linux plugins?

Steve

3 Likes

Now that would be cool.

For people like me with pretty much zero coding skills we could actually get on board and do some of the hard work porting plugins for the more tech savvy to mark our work and see if it makes the grade.

When I first looked into the Dwarf I must admit I thought any LV2 Plugins could be loaded on it. And if there was a possibility to finally do this I think sales would pick up very quickly.

Really love the stuff you are trying Falk but unfortunately I’m windows only so can check out at present. If that does change I’m happy to help in anyway.

2 Likes

This is quite literally the opposite.

Personally I’m happy to not have to replicate work done for the desktop to re-implement in html/css/js.

@falkTX Did you think more about the “full screen” idea. For me it would be great if you could expand to a full GUI of the plugin in a new tab/window.

2 Likes

the fullscreen-on-separate-tab is a bit complex to do, not sure if possible for the mini variant.
work for the mini variant benefits the other non-mini ones, where the remote control is being improved. things feel quite snappy now.

I am starting to get things working, finally.
Caught a small issue in mod-host along the way, so it is not useful to share builds just now until we have a 1.12.2 update.

but can still leave some pictures I guess…
the initial selection of modules have begun, plaits is one of the modules that works already.

I found a way to fake the IO animations, so the GUI should in theory show the same behaviour as the DSP/plugin side. This includes the scope :smiley:

These are super low-resolution in terms of “audio” being generated, but just enough to give visual feedback.

12 Likes

5 CV ports are enough for a mini version, right?
The monitoring of audio and CV works, even with its super low resolution on the GUI side.
Transport state can be sent too, with only MIDI remaining (but I am not sure if that is really needed…)

10 Likes

Looking good.

2 Likes

Tried the sneak peak of this and man does it feel smooth! For Cardinal it needs a few kinks resolved I guess. Removing the resize handle so you don’t accidentally mess up the view.
Also it’s not possible to switch to 256 frames (which could help a bit with performance and xruns), but I’m sure that can be resolved.

2 Likes

@falkTX could you clarify your thinking here? I do not know what Cardinal is but how does this fit with the ModX platform. What are you exploring and why?

My extremely excited vision is you’d offer an alternative to the Mod custom hardware with using a Web browser, leveraging Wasm to allow cross compilation of existing code. Then given a URI I could load and run a board in seconds. I think it can be cached to run offline too. Then someone could fork my URI, using the existing Mod UI to edit it and save their version.

But does that fit the MOD biz plans in anyway? It’s a pivot form hardware.

It obviously requires APIs that mimics whatever OS services Mod currently uses. Plus ports of libraries. But from the above Cardinal thing I guess you think webAudio might be usable? That must be critical?

The reason I think stackblitz might be a good partner is they have worked with Browser vendors and standards to develop their WebContainer technology. While this is currently used to provide an in-browser node stack it might well be able to support WebAudio etc as required for the Mod platform. Ie a new Mod WebContainer .It might be mostly proprietary but they might be willing to collaborate - they are great guys (I discussed working with them at one time). This could allow you to work higher up the stack rather than emscriptem hacking audio drivers etc.

1 Like

Cardinal is mostly unrelated, it is just one of the plugins that we can make it work this way.

I think you are confusing my intention here.
What I am trying to showcase is an alternative to how we create modguis.
Until now the only way to have a modgui was a collection of html, css and javascript.
My idea is to bring over the stuff others have built for web-assembly and web-gl and have that as the modgui. It still needs html/css/js, but the actual user interaction and logic does not happen there, the html/css/js stack on these custom web-assembly modguis are just a wrapper.

Audio never runs in the browser, and that has never been the intention. I mention “audio” being there in Cardinal just to simulate what the desktop version does and have matching animations. There is no real audio being generated with the web-assembly modguis.

And if it was not clear, none of the web-assembly related work is made on MOD working hours, it is completely unrelated to any business plan from MOD. It is 100% me being bored during holidays and wanting something exciting to do.

10 Likes

Got it. Thanks.

I obviously got over excited:)

2 Likes

Fantastic work, as usual, @falkTX. I hadn’t checked the forum over the holidays and could enjoy how things evolved over the course of a few minutes. I would have opted for the separate Windows mode, to gain more space and avoid integrating it into the MOD UI but with your last explanation it makes a lot of sense to implement it the way you did. Thanks for sharing the journey with WASM!

4 Likes

I’m keenly interested in this. Would love to get the infamous lushlife on mod but have been too intimidated at the thought of the HTML/java/css needed to even start. Not sure what it will take to extend your work to other toolkits, but wouldn’t mind being a guinea pig.

4 Likes

I made this with DPF with a custom pugl wasm implementation, so in theory anything else using pugl would be able to make use of the implementation too.

So far we only have OpenGL (or better said, GLES2 via WebGL) though I already did the work to get cairo building on wasm. Dont know how to map a cairo context to wasm, we can always just try to render into a GL texture and draw that.

The first step here would be to update your plugins to use the latest pugl. Or alternatively just convert the raw LV2 code to DPF… You get automatic cross-platform builds and several plugin formats all at once :wink:

3 Likes

I was wondering if drmr could benefit from this? Here you said it lacks only the UI part, and we’d have fully-featured drum machine on MoD platform. Now, when native UI can be ported, maybe it makes things easier?

1 Like

drmr uses gtk for its UI, a wasm version is out of scope for that.
any anyhow the UI is not too complex, a regular html/css/js could work for that specific plugin

3 Likes