Some thoughts on vvvv

vvvv beta question
1

I lovvvve vvvv. It has become my main tool for all the work I do. Spreading them data, seeing my images alter realtime while patching, having these vast landscape of nodes with loads of functions in there, animating graphics in light speed.

But truth is: I am pissed about some of the core principles of vvvv for some time now.
And so i want to write about two of the major problems I see about vvvv and want to outline my own ideas how to deal with them.

My major gripe always has been the use of Windows only technology like DirectX and until recently I just thought: Hell yeah okay, with the option to install Windows on my Mac and since the Windows UI improved in Vista I can live with that. Plus I always unterstood that in the beginning there must have been a big advantage in sticking to a Windows-only world. Time changes and now I am not just unsatisfied but deeply concerned about the relevance of vvvv if the single platform approach won’t be left behind and vvvv will be multi platform.
The PC era just ended, so don’t think multiplatform as Windows / Mac OS X / Ubuntu. Think about the major shift to touch technology that has started two years ago. Under the hood of these shiny new devices there runs a UNIX or Linux based System. And I want to develop on these new devices, because, f.e. creating user interfaces is part of the work i do and we’re all gonna use more and more touch technology tomorrow. I can do that, but I can’t do that with vvvv. I have to change my entire work process from working node based to writing lines of codes. If I would have liked to write code to generate images, I wouldn’t have come this vvvvorld in the first place and would have started Processing or whatever (now OpenFrameworks). The iPad is so cheap it could be easily used in museums and other exhibition areas. I am note saying that it is a must for vvvv to run on the iPad, but it should. And we can be sure that a lot of similar devices from other companies will be released soon, as we have seen that after the iPhone.

This problem leads to the following conclusions:

-1- only the core of vvvv should run on these devices
-2- the user interface must change

ad 1
On systems like the iPad and its future sisters and on other smaller multitouch devices it doesn’t make sense to do the patch work, or it isn’t technically possible (sandboxed apps etc.). So why not abstract the user interace away from a kernel that can run everywhere? Only the Renderer opens (in a single window i.e.). VVVV would even work as a standalone app on Apple Devices. See pd, the UI just communicates with the server. Years after pd has been started on a Linux machine, it can be used in iPhone apps like RjDj. Another advantage would be to make native user interface controls possible. Using vvvv on a Mac with all the gorgeous Quartz3D UI elements, how great would that be!

ad 2
Back in the days, when the vvvv user interaction model was drafted, the mouse was the only logical way to patch. Not anymore. You already have a hard time patching on systems where the mouse wheel isn’t working or where you don’t have a second mouse button at all.
So the three-button-mouse interaction model should be abandoned completely (which I would prefer) or a second model should be integrated and optionally made available. This new interaction model should work with just a single touch point and without having to drag pins or other elements to change values. There is a simple reason for that: This way patching would be possible not just with a mouse but also on a touch screen, or by using a Wacom tablet etc. Optionally multi touch gestures could be integrated into this model. I just had a quick look at the new Max 5 interface, they found some nice approaches there. Why doing a double-right-click? Double tab and choose IOBoxes and other nodes from a Popover. You can tell your grandchilds about using something like a right mouse button, now stop using it.

I have always been a user of vvvv, not a developer. I don’t write nodes or shaders. As mentioned above, I use vvvv as a designer because I can work node based and see my results immediately. This is the strongest thing about vvvv and I don’t want to give this up anytime soon just because vvvv only runs on PCs with a three buttons mouse attached.
vvvv’s core technology is too old and too narrowed down. vvvv’s interaction model is outdated. I wrote this thoughts because I really hope there will be fundamental changes, so i can continue using my belovvvved tool.

rainer.

2

touchscreens:

patching on an ipad ? patching with a touchscreen? fingers are far less precise than an indirect control via mouse pointer. not to mention the ergonomical problems involved with touchscreens. try using a vertical reasonably sized touchscreen for 6 hours, check your arm. try using a horizontal touchscreen for 6 hours, check your neck.

touchscreens will become more and more important but i don’t see them replacing the mouse keyboard combination for quite some time. unless someone manages the haptical problem with current slick touchscreen surfaces…and the ergonomics.

one button mouse:

getting rid of a three-button mouse interface ? maybe i’m oldfashioned but i like using at least two fingers of my right hand and not just one. it doesn’t make an interface easier by switching functions from a mouse button to a keyboard button.

windows:
i still don’t mind the windows only approach. i would if direct-x is not being developed, if all the cool hardware wouldn’t work with this system because there is no driver. if there aren’t new things coming up like direct physics. and don’t forget the games industry who is still using pc’s and windows to push realtime graphics (see crysis 2 etc.)

if development on microsoft side (+ dev community) would stall then i would be with you.

and patching for all the neat touch devices which run some kind of unix or proprietary OS would be cool but i guess that’s too much to ask. you want an ‘eierlegende wollmichsau’ which vvvv to some extend already is but not as much as you want…

anyway, although i don’t agree with you in any point, it’s interesting to start such a discussion.

3

hello rainer,

i think gregsn, joreg an me are totally aware of the evolution of the computer world and we know the position of vvvv in it. there was just no time/funding to do major steps so far. as you know, the only income vvvv has comes from the users who buy a license. now, after years, there is an amount of money which will be used to give the developers (including elias) a small payment during this year to work on such things.

we will see what comes out…

btw. i like the vvvv interaction design. i assume vvvv users are rather intelligent and can handle such advanced interfaces like a 3-button mouse… i cannot imagine, how to do things faster with only one button or even a touchscreen with gestures. for me patching is about possibility and speed, not simplicity.

4

good point form wirmachenbunt about the DirectX/Microsoft/Gaming thing, DX11 is far more advanced than any other 3D API.

5

very good to hear that during this year you plan to have some developers working on such things

i’ve always been trustful in vvvv evolution, and i will always be, cause i know there is really clever people behind it

and if things go slow sometimes, it’s just because of time and money and people

about this, i still think that the opensource step would really speed up the evolution process

my main concern would be :

-the multi-core step
-dx10/11 and proper shaders (things that are in videogames since many years already, but still seems hard/impossible in vvvv)
-the user interface, just look at max5, there are some things that works better there (mainly about patch organization, colors, comments), but for some other i still prefer vvvv

i know that’s a lot of work for developers so i’ll just keep waiting the time it takes

6

oh and vvvv identity on the web should be improved (website and social networks)…

7

I love windows and directX and there will be plenty of windows based iPad type things on there way that will be much better than the iPad. Yeah it’s a shame that probally half the world will buy an iPad and it won’t run vvvv, but I don’t really see vvvv being a development language for applications that need to run on anything other than bespoke build systems. Visual systems for nightclubs, multi touch units, art installations etc.

I can understand from a designers/graphics point of view that you want to get you’re vvvv graphics/apps etc out there in the form of an application that runs on anything but something like Flash is far more suited to this and pretty easy to learn. From the vvvv.org home page, ‘vvvv is designed to facilitate the handling of large media environments with physical interfaces, real-time motion graphics, audio and video that can interact with many users simultaneously.’ I know vvvv it is more than this to many but like the devs say, they haven’t got unlimited resources and time. If vvvv was a multi platform project then it wouldn’t be anywhere near what it is was now. Vvvv development seems to me to move soooooo fast it’s great! Keep the focus!

8

@rrrr
Apple don’t even have quartz composer working on iphones/ipad, so I don’t expect vvvv to work on it!
Flash doesn’t even work on it…
But flash will compile apps for it, so for development for the ipad, look to flash…
And TBH anyway I would expect the ipad too send osc to a control pc to render data on projectors/ large screens in those museum installs anyway…

What I do want to see is multi core (even if this isn’t a fully thread app, at least nodes should be threaded like the plugins can be, I suppose ideally the slices would be spread between threads?)
The other thing of course is that Dx is lagging a bit, we used to be streets ahead of openGl, but even that seems to be past dx 10, slimDx does claim to support dx 10 tho?

But once you get to vista/win7 theres the whole multi-monitor problem and texture allocation. I did wonder whether a solution could be an extension of the dxtexture / video texture nodes that when enabled copied the textures to memory on both heads, presumably there would be some performance issues, but if it was an option they would hopefully only hit when you absolutely needed it to work! Of course currently the only option is to patch twice one for each head, and that is a lot more resource hungry!
Definitely good to hear the devs are getting some play time! :)

9

I was forced to use Mindoms (I was a Slackware defender ;^) just because of vvvv. By now it’s ok to have just one platform, consider that porting vvvv to other platforms would require a huge work !! But well, it could be cross language, it would be nice to have a C/C++ Template plugin (we know vux can do that magic), or even a Perl or Python Template plugin :P
Another almost “free” option should be hoping on Wine, if you read how it works they explain that you can call a .dll from everyOS so no port is needed, only a lot of
fixme:
fixme:
fixme:
…

10

@catweasel: “And TBH anyway I would expect the ipad too send osc to a control pc to render data on projectors/ large screens in those museum installs anyway…”
my thoughts exactly! the only thing we must wait for (or build ourselves) is apps to create custom interfaces and OSC mappings. the hardware of iPad is pretty weak - intended for long battery life, compared to regular laptops with dedicated graphics cards. I know they demoed some games, but I bet those were specially optimized to look good on a small resolution etc. But then again, it might not be so weak. I can’t claim that since I don’t know much details about the iPad. It’s just a hunch.

@rrrr: I understand what you want, I must admit daydreaming of similar things. However, it occurred to me that vvvv does not need to be ported completely to other platforms. As many have said, to make vvvv multi-platform now, too much time, money, energy would be lost on something non-essential. A much better road would be to take only the user interface and make that for other platforms, such as iPad. That could perhaps be an open source project. The output of such an application could be a v4p file, the format of which is already XML, which in theory should help make the collaboration of systems easier. The running vvvv patch would be on a remote computer, where all the rendering would take place. A video stream could be delivered back to iPad to simulate Renderer windows? But as many have said, a touch pad screen device has many challenges when one wants a precise and complex UI with lots of details and control. Until such technology advances some more, perhaps building simple OSC control GUIs will be enough, at least to simulate a LEMUR device! :)

On the other hand, I’m just a database driven business style web app developer (ASP.NET, SQL Server). What do I know? :)

11

@rrrr: I just remembered you already mentioned a similar idea when you mentioned pd. But you meant the other way around, right? You suggested the render process executed on the iPad device, right?

12

Starting this topic had a lot more to do with thinking outside the box than sticking with all the old pros and cons. Alas, the replies so far stick more with the latter thinking. I am totally aware of the resources of our vvvv dev gods and i am sure they very well know their possibilites and where they see vvvv in the future. It’s their domain to decide this.
But i wanted to be a little loud because I know their will be some new developments, so now is the time to think how could be such a situation we have right now (winonly) avoided in 10 years. The devide won’t be between three operating systems but between two different worlds. And betting on Microsoft technology here? Come on? Is their DirectX running on the Zune? Windows 7 Mobile will be a completely new OS. Running Windows with all the same UI on a slate device? That’s ridiculous.

The world out there currently changes from something that will be considered a little odd by our descendants to something beautiful and you are talking about speed? VVVV already supports multithreading in the plugin architecture. If you want speed you want OpenCL (or, hurra - Cuda is superior right now, so we could narrow everything down, so it runs on machines with a Nvidia graphics card installed).

Common sense, the iPad is precisely aimed at consumers. But i am not so sure if their won’t also be a lot of Pro software coming out for it. And there will be a lot of apps that will merge different apsects of the Art, Music and Design worlds together – something we already see in a lot of projects done with vvvv (and bros/sis). So in my opinion, vvvv sould be perfect to make a lot of stuff I am interested in making in the touch world. I have written this post, because I wanted to think about what would be necessary to make that possible. Otherwise I’ll have to learn some C++ and start using OpenFramworks which would be a bummer.

The second topic that fit into this thinking was the critisism about user interface controls. I haven’t been precisely here, because I think the vvvv UI is the best node based UI out there and you can patch very fast. It’s brilliant, but it’s not perfect. Because you have to use three button mice, which is a problem right now, not just in the future.
About a year ago, I tried using vvvv with a wacom tablet, because my arm gets tired when using the mouse all day and I heared from tablet users that this goes easier on their arms. It’s absolutely impossible to use such a tablet in vvvv because you have to drag your mouse pointer to change a value.
90% of the computers sold in the US that cost more than $1000 are macs. A big segment of them are Macbooks. You can’t use vvvv on them without attaching a mouse, although it comes with a great multitouchpad. I know that the current way to interact won’t be left behind (although I think it should because a toggle should be switched in a way it is done since the invention of the GUI, with the left mouse button), but I think there should be a second interaction model for now and for later.
It’s nonesense that you’re patching faster because you’re using three buttons on a mouse. You’re patching fast because you can create a nodes really fastly or lookup a name if you have forgotten the naming. Dragging with the right mouse button isn’t very effective and makes a lot of problems in Windows because a lot of times vvvv is losing focus or something and then i suddenly drag files around in Explorer, or stuff like that. And if you have different monitor resolutions this feature gets broken completely.

I have written this postings from a designer’s perspective. I want to use beautiful things like Macbooks, tablets, etc. in combination with vvvv. Why do you think Processing is so big now? It also has been started by just two guys. It should be the other way round, where every year there are new books released where people talk about vvvv. But Processing got something, that vvvv doesn’t. You don’t have to use awkward things like Windows or a three-button-mouse for example.

13

Starting this topic had a lot more to do with thinking outside the box than sticking with all the old pros and cons. Alas, the replies so far stick more with the latter thinking. I am totally aware of the resources of our vvvv dev gods and i am sure they very well know their possibilites and where they see vvvv in the future. It’s their domain to decide this.
But i wanted to be a little loud because I know their will be some new developments, so now is the time to think how could be such a situation we have right now (winonly) avoided in 10 years. The devide won’t be between three operating systems but between two different worlds. And betting on Microsoft technology here? Come on? Is their DirectX running on the Zune? Windows 7 Mobile will be a completely new OS. Running Windows with all the same UI on a slate device? That’s ridiculous.

The world out there currently changes from something that will be considered a little odd by our descendants to something beautiful and you are talking about speed? VVVV already supports multithreading in the plugin architecture. If you want speed you want OpenCL (or, hurra - Cuda is superior right now, so we could narrow everything down, so it runs on machines with a Nvidia graphics card installed).

Common sense, the iPad is precisely aimed at consumers. But i am not so sure if their won’t also be a lot of Pro software coming out for it. And there will be a lot of apps that will merge different apsects of the Art, Music and Design worlds together – something we already see in a lot of projects done with vvvv (and bros/sis). So in my opinion, vvvv sould be perfect to make a lot of stuff I am interested in making in the touch world. I have written this post, because I wanted to think about what would be necessary to make that possible. Otherwise I’ll have to learn some C++ and start using OpenFramworks which would be a bummer.

The second topic that fit into this thinking was the critisism about user interface controls. I haven’t been precisely here, because I think the vvvv UI is the best node based UI out there and you can patch very fast. It’s brilliant, but it’s not perfect. Because you have to use three button mice, which is a problem right now, not just in the future.
About a year ago, I tried using vvvv with a wacom tablet, because my arm gets tired when using the mouse all day and I heared from tablet users that this goes easier on their arms. It’s absolutely impossible to use such a tablet in vvvv because you have to drag your mouse pointer to change a value.
90% of the computers sold in the US that cost more than $1000 are macs. A big segment of them are Macbooks. You can’t use vvvv on them without attaching a mouse, although it comes with a great multitouchpad. I know that the current way to interact won’t be left behind (although I think it should because a toggle should be switched in a way it is done since the invention of the GUI, with the left mouse button), but I think there should be a second interaction model for now and for later.
It’s nonesense that you’re patching faster because you’re using three buttons on a mouse. You’re patching fast because you can create a nodes really fastly or lookup a name if you have forgotten the naming. Dragging with the right mouse button isn’t very effective and makes a lot of problems in Windows because a lot of times vvvv is losing focus or something and then i suddenly drag files around in Explorer, or stuff like that. And if you have different monitor resolutions this feature gets broken completely.

I have written this postings from a designer’s perspective. I want to use beautiful things like Macbooks, tablets, etc. in combination with vvvv. Why do you think Processing is so big now? It also has been started by just two guys. It should be the other way round, where every year there are new books released where people talk about vvvv. But Processing got something, that vvvv doesn’t. You don’t have to use awkward things like Windows or a three-button-mouse for example.

14

not sure what you mean with “from something that will be considered a little odd by our descendants to something beautiful”, can you clarify that? what would “something beautiful” be?

honestly, i don’t give a **** about apple, macbooks, imac, ipad, iphone, ipod etc (they don’t look so beautiful to me) and i really hope that vvvv won’t become a consumer thing, cause it would go against its own essence imho

talkin about speed, from what i can see, nowadays technical landmarks are still to be found in videogames, which is mainly windows/directx

about the UI, three button mice works good for me, i just would like to have the possibility to use colors in my patches.

patching with a tablet, well i’m not sure about that, i don’t think it’s the right interface for this purpose, but thats just my opinion.

why processing is so big now… well that’s a good question, do you think it’s because processing runs on mac? i don’t think so, or at least thats not the main reason
it’s more about their policy, first of all it’s opensource, it has a lot of focus on getting new people into it (publishing books, good use of web etc), and you can export standalone apps, and it runs on the web too.

you have to remember though that processing users still envy vvvv’s interface, speed and technical potential, cause we can do things very easily which for them are quite hard.

when it comes to realtime visuals, i think vvvv has no competitors at all

anyway this is not about vvvv vs processing, i think they’re different and don’t directly compete

of course vvvv needs to evolve, we all know that, let’s wait and see what devs have in the pipes…

15

written from a designers perspective ? i’m being devils advocate right now …does this mean designer only use mac’s and one button mice ? or windows machines are only tools for office applications ? that’s thinking inside some box. :)

and you are putting mobile applications and large scale media installations in one kettle. what’s wrong learning a second language for mobile applications to make use of their special possibilites and take in account their special needs (low hardware specs). don’t blame one development environment for not covering every single system which you are interested in.

a second interaction model
well, that’s some interesting idea.

16
  • can you clarify that? what would “something beautiful” be?

PCs are starting to vanish. we’re going from the universal machine back to dedicated devices. nobody will care what blend of operating system is under the hood, and that’s a good thing. i clearly didn’t wanted to start the old mac/windows discussion, simply because this isn’t about mac vs win.

  • and you are putting mobile applications and large scale media installations in one kettle

hallelujah! that was the main reason why i’ve started this post. yes, i want to use vvvv for mobile devices too. yes, our society currently likes big ass media installations. but someday that might just be as boring as watching tv nowadays. and the interesting content might be somewhere else then. i don’t watch tv.

17

Been looking at lot of comments, so will make a few as well.

User interface wise, vvvv is definitely by far the best, try nodebox (now node based), reaktor, eyesweb, and you’ll see the difference.

The processing/vvvv doesn’t apply really, they both serve different purposes, some stuff in processing is really easy to do, some is not (just like linearspread+cross+transform+quad is a 30 seconds job in vvvv, takes you easy 5 mins on processing). For things like particles, outlines, processing can be more handy.

About the rendering engine, i’m happy with directx, but it’s been microsoft policy to decide dx10/11 would not be available on xp, which is a shame really. A lot of cool stuff can be done with Shader Model 4.0, but we stuck to 3.0.
About OpenGL, it’s pretty powerful too, but much more low level, many things which are trivial in dx are not at all in gl.
A gl rendering engine could be done as plugin, but that’s really long as a job.

About multi platform, you’d rather start a new project from scratch rather than making vvvv portable. That would take too much time, bug fix to check.
You need to reprogram the rendering engine, access to all devices, and a lot of other functions.

About performances and Multithreading, did some tests on that, you can write your plugins using pointers, sse2 optimizations and thread the nodes. You can have good performance boost. Some nodes are trivial, some REALLY hard (like randomspreads for example). But that’s also a pretty long job (specially you need to keep a non sse2 version in case processor doesn’t support it), and having a lot of node duplicates is not great.
On OpenCL/Cuda, you’ll have the problem that vvvv uses doubles, whereas for the moment Cuda supports only floats (that will change in near future), and if you don’t have a complex operation the time to copy stuff in graphics card and get it back, you’d have done the job in cpu already.

But I know vvvv will evolve, and as you never have unlimited resources/time/money, the devvvvs will have to make choices on what they want to prioritize, and I’m sure they’ll make the right ones :)

18

@rrrr: there’s quite a bit of what you say I personally disagree with but I haven’t got the time, energy, need, or bother to address them all. Some notes on a couple of more interesting points though;

Running Windows with all the same UI on a slate device?

I don’t think anyone would think this to be the case.

Interestingly, have you seen the stand, external keyboard and mouse for the iPad? They clearly have two forms of interaction with its apps, touch and traditional.

So in my opinion, vvvv sould be perfect to make a lot of stuff I am interested in making in the touch world.

There is a touch world way beyond Apple that is far more flexible and powerful and is far more interesting for me. And what about gesture?

Otherwise I’ll have to learn some C++ and start using OpenFramworks which would be a bummer.

Why a bummer and why C++ and OpenFrameworks? I use a whole host of different software and programming languages to achieve so many different things from art to 3D to visuals to websites to bespoke touch/gesture apps. Never have I found an app that does all I want it to do. So maybe expanding you’re software base isn’t such a bad idea and from experience it may even improve you’re work. I’d say Flash is the best way forward, incredible software from a designers and coders point of view, works on everything, and of course easy to learn and easy to integrate back into vvvv.

Fair shout for bringing up you’re issues with vvvv, however slightly misguided I feel they are. However, I won’t eliminate the fact that I maybe wrong and you maybe right.

19

@rrrr: there’s quite a bit of what you say I personally disagree with but I haven’t got the time, energy, need, or bother to address them all. Some notes on a couple of more interesting points though;

Running Windows with all the same UI on a slate device?

I don’t think anyone would think this to be the case.

Interestingly, have you seen the stand, external keyboard and mouse for the iPad? They clearly have two forms of interaction with its apps, touch and traditional.

So in my opinion, vvvv sould be perfect to make a lot of stuff I am interested in making in the touch world.

There is a touch world way beyond Apple that is far more flexible and powerful and is far more interesting for me. And what about gesture?

Otherwise I’ll have to learn some C++ and start using OpenFramworks which would be a bummer.

Why a bummer and why C++ and OpenFrameworks? I use a whole host of different software and programming languages to achieve so many different things from art to 3D to visuals to websites to bespoke touch/gesture apps. Never have I found an app that does all I want it to do. So maybe expanding you’re software base isn’t such a bad idea and from experience it may even improve you’re work. I’d say Flash is the best way forward, incredible software from a designers and coders point of view, works on everything, and of course easy to learn and easy to integrate back into vvvv.

Fair shout for bringing up you’re issues with vvvv, however slightly misguided I feel they are. However, I won’t eliminate the fact that I maybe wrong and you maybe right.

20

@rrrr a tip for using vvvv on a macbook, I use input rempper, which uses the fn key + click to right click and get on fine with it, my hands there for the ctl and alt keys anyway, so it doesn’t take long to get used to.