Here is a small start, for now, only for writing code for the vvvv gamma ecosystem:
far from perfect, and far from complete, but a starting point:
Install only the mcp via cli (without pulling the repo):
dotnet tool install -g vvvv-mcp
Configure your MCP client (Claude Desktop, VS Code, Cursor) automatically
vvvv-mcp --setup
the mcp knows the graybook and @tonfilm’s skills, so it does an ok job at explaining simple patches and can also explain vvvv’s concepts in general. Nothing really a chatbot which you point at the graybook couldnt do, but it’s a start…
The repo also pulls all nugets available in the package manager and runs an analyzer to collect info about available packs and their nodes, which is then shipped with the mcp…but this will need more love to be fully useable.
not working yet: creating/editing patches. The mcp hasn’t “seen” any patch other than a simple box and a renderer, so it’s knowledge is strictly theoretical atm. the goal here currently is to edit vl files directly and possibly read back console and/or images from renderers.
what might never work (unless @devs take over here): see running instances, patches, open/close patches, manage nuget packages, export apps, etc.; so no real remote control of gamma for now.
100% vibecoded with free credits, since I’m spending too much for work already :D that’s why I have set up a sponsor button on github for the first time. Further development will not depend on donations, just bring them to you earlier 🙏
Update: I stand corrected, the HDE extensions give us a pretty good “attack vector” to control things like open/close patches, undo/redo, bring tab to front, read available packages, etc…I wasn’t aware of that, but the mcp itself already knew and the llm found out the rest…works here but need to ship a vl package for that too…coming…
Also thanks to @motzi and @readme for pointing out the mcp @kopffarben is working on, which seems way more advanced than what i have whipped up here in 1 day ;) i will still bring this into a working state, but I guess it’s fair to call this the simple one, hehe… @kopffarben i’ll hit you up in the chat!
Hey @digitalwannabe! super cool to see what you have built. I did not get a chance to run it yet but by the shape of the repo it looks quite feature rich already, especially the knowledge and vvvv-context endpoints available to the agent. I think that direction is really promising in reducing the time agents spend theorizing about VL patching tasks.
I may have a useful thread for the write functionality: a few weeks ago I have built a similar prototype with read and write tools for vl files. Works quite well already, although it is still very slow.
Editing a patch works by placing operation requests (placeNode, placePad, connectPins, etc) using a defined json schema into an inbox. These operations are then picked up by an execution thread inside an HDE Extension and applied to the patch using the public editor APIs.
I did not get around to polishing it yet so it is in rough shape, but there may be some helpful ideas or insights in there. Feel free to poke around in the repo, if you find anything useful I am happy to contribute it :)
We discussed this prototype at Link and compared it to @kopffarben’s MCP, seems like we all landed on similar implementation shapes. Seeing that your project now joins the list of work-in-progress MCP connectors, it might be worth consolidating our efforts into one really capable package at some point.
Let me know what you think of this, cheers!
Hi @prt,
thanks for sharing your repo, and thanks for the feedback! Tbh, I didn’t expect to come this far this fast, and if I had known about your initiatives I probably would have never started this. Still not sure if I should regret it 😛
Your approach to creating patches sounds very neat, I’ll have a look! Would be cool if we could create some mcp bench to test these different approaches to the same goals.
I’m currently writing to the xml only (except for 1 api test path for setting pin values), but this needs a ui refresh to show up, which I assume your approach won’t need….. Doing it via xml also has other issues, like having to compute/load node sizes, etc…@kopffarben‘s approach seems to be even more checked out from what I understand.
I think it’s a good idea to exchange these approaches of how this kind of mcp tooling is solved, learn about this together and eventually converge on a best design, since atm there is still a lot to be gained from well-designed tools; simple example: at first, I had the mcp place one node after the other, first searching for the node, then placing it, min 2 llm calls per node, now I create a full subgraph in one go, try to validate it and only then drop it into the patch with only 1-2 llm calls total (but more mcp tool code running locally).
In the long run however, looking at how powerful the latest frontier models have become, the important thing here I believe will be the condensed knowledge database, since rather sooner than later a llm will be able to read all of it pre-prompt and construct the necessary tools on the go. Not a prediction mcps will vanish, but in case of patching in vvvv, knowledge about vvvv is where the juice is imho (…downloading the forum and chat rooms as I’m writing this 😇).
The rest is xml, c#, .net, hlsl, http…stuff the llms already know too well - at some point there’ll be no need for us to tell it what to do there exactly…
To get knowledge right is a huge task however, and it will be continuous work; also, this shouldn’t be a lone endeavour, because at some point you are asked opinions and have to decide on best style, common practices, „ideal“ solutions etc….also the amount of work is obviously a lot. I currently have 227 ocr-analyzed image descriptions from the gray book waiting to be checked by a human (won’t do that before @kopffarben is back, maybe he’s done that already…..)
So I’m generally up for pooling efforts, I think it will be necessary; but since there are a number of vvvv library initiatives depending on community funding/contribution, which are not thriving as much as they probably should, I also (changed my mind on this and) think this should be made commercially viable from the get-go - for everybody involved - since it will not only consume time to curate/update the knowledge and improve the mcp, but also incur additional expenses through llms needed to do this well.
I’d happily be corrected here, but atm it seems like it’s more feasible to charge pros/studios than relying solely on community support. Ideally, some of that revenue could also be re-distributed to pro users who contributed indirectly to the knowledge base via packs, help files, forum answers, etc.
(Note that for this reason I have changed the license on the repo for now- it’s still free to use, there‘s no commercial license since you shouldn’t use it commercially yet, and I’m definitely not saying my approach is the way to go, but like this it leaves all options on the table.)