Combining OffscreenCanvas with SharedWorkers

The secret of successfully using multi window WebGL Canvas

Rendering a Canvas node with a WebGL content and moving it into different windows without reloading dependencies like D3, keeping the current state and without creating new component JS instances.

Content

1. Introduction video
2. Definitions: OffscreenCanvas & SharedWorker
3. Benefits & potential use cases
4. Content of the demo
5. Code
6. Feature request for SharedWorkers
7. Online demos
8. The neo.mjs project

1. Introduction video

2. Definitions: OffscreenCanvas & SharedWorker

The OffscreenCanvas interface provides a canvas that can be rendered off screen, decoupling the DOM and the Canvas API so that the <canvas> element is no longer entirely dependent on the DOM. Rendering operations can also be run inside a worker context, allowing you to run some tasks in a separate thread and avoid heavy work on the main thread.

OffscreenCanvas - Web APIs | MDN

The SharedWorker interface represents a specific kind of worker that can be accessed from several browsing contexts, such as several windows, iframes or even workers. They implement an interface different than dedicated workers and have a different global scope, SharedWorkerGlobalScope.

SharedWorker - Web APIs | MDN

3. Benefits & potential use cases

The whole concept of OffscreenCanvas is centered around transferring the ownership of a canvas node into a worker. This makes a lot of sense to ensure that expensive calculations can happen inside a separate CPU core which can compute them in parallel without having bad side effects on our rendering thread.

Obviously the first thing that comes to mind is to move every single canvas node into its own dedicated worker, as long as there are still free CPU cores available, to maximise the performance gain.

So, why should we even think about using a SharedWorker instead, which allows us to connect to multiple browser windows?

Benefits:

1. In case you want to use bigger libraries like D3, you only need to load them once into your shared scope. Not even a cached reload is needed.
2. In case you have an expensive and / or time sensitive logic to manipulate the state of your canvas, you can apply it to multiple canvas nodes inside multiple windows.

Use cases:

1. Think about creating a multi screen game where you want to show a fancy animated clock on each of them and want to ensure they are absolutely in sync.
2. I recently discussed with a big company about an engineering workplace, where each engineer works on multiple screens. A bit like a complex web-based IDE. They are using heavy 3d models and would love the ability to just move these in real-time on a different screen if needed.

4. Content of the demo

Creating a decent workers-setup on our own can take a significant amount of time. Luckily neo.mjs provides us with exactly what we need:

I already created the demo a while ago inside a dedicated worker. Feel free to explore the content here:

Rendering 3d offscreen: Getting max performance using canvas workers

Since this part is covered, we can purely focus on the multi-window aspects inside this article :)

5. Code

You can find the repository which contains the full working demo (MIT licensed) here:

GitHub - neomjs/shared-offscreen: test case to move an existing canvas node into a new browser…

Differences to the previous demo:

Inside our main app, we need to switch to using SharedWorkers apps/mainapp/neo-config.json

For moving our canvas node into a different browser window, we do need a 2nd app: apps/childapp

This one pretty much only contains an empty viewport:

Inside our demo, we do have the Move Canvas button at the bottom right:

So, let us take a look into what this one → moveCanvas()does:
apps/mainapp/view/MainContainerController.mjs (full code)

1. moveCanvas() creates a new popup window which contains our ChildApp
2. The amazing point is that both of our apps live within the same shared application worker, so creating the popup will trigger onAppConnect()
3. The best part is that we can simply remove our existing webgl-component JavaScript instance and add it into the viewport of our ChildApp Although this one happens to live within a different browser window, we can even keep the same JS instance
4. In case we close the popup window, we will trigger onAppDisconnect() which will move our canvas component instance back into our MainApp

But … can we really transfer the ownership of a canvas node multiple times?

In theory it might work for an empty canvas node, but as soon as we set a context like WebGL , trying to move the node again causes a JS error.

So, how does it work?

The magic happens inside our canvas component itself:
src/component/Canvas.mjs

Every time we (re)mount our component, it will drop a new empty canvas node into the DOM. Afterwards it will get transferred into our (shared) canvas worker and gets added into our id map there.

Since it does have the same id as before, the next animation tick will apply our existing state to the canvas node inside a different browser window.

6. Feature request for SharedWorkers

Browser support:

1. OffscreenCanvas is available inside Chromium and Firefox
2. SharedWorkers are available inside Chromium, Firefox and Safari Technology Preview

For multi screen apps we will most likely use a native shell (e.g. Electron), so it is not too bad if not all browsers are supported yet. It does make me happy to see some effort though!

You can also use requestAnimationFrame in workers

OffscreenCanvas - Web APIs | MDN

While requestAnimationFrame() is indeed available inside dedicated workers, this is sadly not the case for SharedWorkers .

A bit obvious since it would not be clear which main thread we want to use.

However, inside a shared worker we have connecting ports.

In case a new main thread connects, the port should include requestAnimationFrame() and we would be good to go.

7. Online demos

Side note: sometimes we need to reload the app once to see the animations. Need to debug this at some point.

Dev mode (running the code as it is → Chromium):
https://neomjs.github.io/pages2/workspace/neo-shared-offscreen-canvas-demo/apps/mainapp/index.html

dist/production (webpack based → Chromium & Firefox):
https://neomjs.github.io/pages2/workspace/neo-shared-offscreen-canvas-demo/dist/production/apps/mainapp/index.html

The canvas animations run a lot smoother inside the dev mode. This is most likely related to a scoping issue inside the d3fc lib.

Still, seeing this work inside Firefox is awesome:

8. The neo.mjs project

“An application worker being the main actor” or “off the main thread” are no longer fiction, but work very stable and extremely fast:

GitHub - neomjs/neo: The application worker driven frontend framework

neo has already hit v4.2.1 with more than 15.000 commits inside the ecosystem.

In case you have not looked into the benefits of performance, scalability and extensibility, you might want to catch up with the next gen multithreaded frontends topic.

Best regards & happy coding,
Tobias