[FP10] GPU-accelerated sprite engine

http://www.birchlabs.co.uk/Tower.html

Was wondering what Tower Defense would be like with a hundred times more enemies, so I started work on a more flexible sprite-pusher. Invasion's one required manually pre-made sprite sheets, and all coordinate adjustments had to be done by hand.

In my new engine, however, all I have to do is punch in a movie clip and it does all the work for me - it makes the assumption that I want it to rotate around the centre, then fills arrays with all permutations of angle (choosing to render every 5º), frame, and symbol. It does all of this rendering in parallel rather than putting them into a queue, and temporarily (and sneakily) bumps the framerate up to 120 to help the pre-rendering along on faster computers.

It's a bit slower than Invasion's engine (this hits 30 fps at about 3,000 sprites, whereas Invasion - with troops off - could keep up 30 fps with at least 7,000 arrows). I'm attributing this to the fact that the manually-handled spritesheets meant I was able to forego a few thousand array lookups every frame.

However, the special thing about this is that... _in theory_, it's using GPU acceleration ("gpu" wmode). Doesn't appear to exist on my iMac (figures ¬_¬), but do any of you get a green box and a nice speed boost on that page?

That's really nice mate.

Does this mean you're not going to be pulling out the invasion demo at every opportunity any more ? ;)

As to the green square / gpu thing, I've got a feeling that was just for the pre-release demo and that the current final FP10 doesn't display that.

I'm sure I've seen some javascript which can feedback on if the swf is running under the gpu, although without actually having touched any F10 code at all yet I wouldn't be able to tell you anything more than that I'm afraid.

Squize.

gpu in flash is a myth, designed to stop developers moving to more powerful platforms.

I was able to ask a few questions about it to a french Adobe guy (Thibault Imbert).
GPU acceleration will be activated ONLY for FULL SCREEN rendering.
So yep you can forget GPU acceleration for most the kind of flash games we are generally making here.

Quote:

---

Originally Posted by Sietjp

I was able to ask a few questions about it to a french Adobe guy (Thibault Imbert).
GPU acceleration will be activated ONLY for FULL SCREEN rendering.
So yep you can forget GPU acceleration for most the kind of flash games we are generally making here.

---

Are you sure you're aren't mixing that up with "Hardware Scaling"?

Hardware Scaling was added sometime ago, and was only available in Fullscreen.

the demo is nice,but yeah, of course there´s no gpu acceleration going on at all (if there would be it should be able to handle more than 1000 times the particle count at high fps counts).
Sietjp is right and even in fullscreen mode it basically only uses the gpu for drawing the image, not for all the graphical operations required prior to actually generate the image in first place, so yeah, even then no big performance boost there.
(Besides that point many portals don´t like having a fullscreen mode option and next to that one still can´t use lots of keys in fullscreen mode due to Adobe´s "wicked" security concepts).
Adobe touted more gpu support functionality than just the fullscreen mode scaling stuff but yeah,besides a nicer filtering i still have to see any of that in action to believe it.

I think its really bad mistake that your game has no way to know if gpu acceleration is used or not. The examples I have seen only return the value of wmode set by developer but it only says that gpu acceleration should be used by Flash Player when available, nowhere can you check if it is currently being used or not.

yeah,that´s really quite a problem and i´ve brought it up at several places before, so far no change there though.

Quote:

---

Originally Posted by tomsamson

even in fullscreen mode it basically only uses the gpu for drawing the image, not for all the graphical operations required prior to actually generate the image in first place, so yeah, even then no big performance boost there.

---

Yes that's exactly the kind of things I was told by Thibault Imbert.

"I think its really bad mistake that your game has no way to know if gpu acceleration is used or not"

Totally agree, to me it's as important if not more so than being able to check the [ FP ] version.
All you need is a true / false and you can then tailor the game to suit ( 300 particles or 3000 ).

In saying that, if hardware acceleration is limited to full screen mode ( I didn't know that either, I knew they used it for mip-mapping on full screen mode, but I thought the new GPU mode worked, well, like you'd expect it to, in normal windowed mode ) then it's moot anyway.

Squize.

Quote:

---

Originally Posted by Squize

That's really nice mate.

Does this mean you're not going to be pulling out the invasion demo at every opportunity any more ? ;)

As to the green square / gpu thing, I've got a feeling that was just for the pre-release demo and that the current final FP10 doesn't display that.

I'm sure I've seen some javascript which can feedback on if the swf is running under the gpu, although without actually having touched any F10 code at all yet I wouldn't be able to tell you anything more than that I'm afraid.

Squize.

---

Thanks. :) But regrettably, since the Invasion demo's faster, I'm probably gonna have to keep hold of it.

It would certainly be nice to know for sure whether or not this GPU acceleration exists. I'm surprised nobody's made a bigger deal of it.

I think the JavaScript things only detect wmode...

http://www.bytearray.org/?p=245

Hardware acceleration isn't limited to full-screen. I saw a screenshot (I think from Tinic Uro's blog) of "gpu" mode video rendering in a window, which had The Green Square.

But if tomsamson is right, and that the GPU would only ever be used to plot draw the graphics, rather than to build them... then it's useless for a copyPixels() engine, which only draws one BitmapData to the screen every frame. Having said that, it'd probably be pretty useful for vectors, right?

"But regrettably, since the Invasion demo's faster, I'm probably gonna have to keep hold of it."

Ok, just perhaps not link to it in every post ;)

I think the whole GPU thing has slipped by largely unnoticed because people who make games for the most part know that it's going to be sitting on a site where they have no control of the wmode so why torture yourself doing something that rocks in that mode only for 1% of sites to support it ( And out of that 1%, what % of end users can use it ? ).

In theory my machine can support the gpu mode, if you have the aero effect running in Vista than it can, but I've never noticed anything different. I tried it with 651 ( Which got a huge speed-up going from FP9 to 10 ) and I didn't notice a thing.

It's a bit of a damp squib right now, to the point that we're not even sure if it's only in full screen mode or not ( I'm sure it's not, but I'm willing to stand corrected ).

It's great that you're playing with it, someone needs to, but I wouldn't waste too much time on it mate. F11 is the next holy grail for GPU speed ups.
Pixel Blender and alchemy are the things that are making my pants like a tent atm as they have loads of potential for doing really cool things, a lot lot more than getting the gpu to plot some pixels to the screen.

Squize.

GPU Acceleration isn't something that can be turned on and off though. If the flash player naively did a texture change for each movieclip it had to render you would be getting performance far worse than what the cpu offers.

Flash's entire concept doesn't provide the rigidity that is needed to fully take advantage of the GPU. When you limit yourself to drawing many copies of an identical sprite to a single bitmap you can start to take some performance gains, but reading back the texture will take longer than just drawing it yourself.

In the languages you are comparing flash to such as Silverlight and Unity you need to write the shaders and logic yourself. An efficient GPU particle system on these platforms would involve writing several shaders, knowledge of HLSL/CG and some rather complicated logic involving exchanging an absolute minimum of information back and forth.

The more I learn about writing games using the GPU the more it seems impossible for flash to take advantage of it without either a complete rewrite or seperation. You will notice the workflow in plugins with GPU acceleration is very different.

Quote:

---

Originally Posted by Sietjp

I was able to ask a few questions about it to a french Adobe guy (Thibault Imbert).
GPU acceleration will be activated ONLY for FULL SCREEN rendering.
So yep you can forget GPU acceleration for most the kind of flash games we are generally making here.

---

I still dont understand it, quote from here:
http://www.adobe.com/products/flashp...res/#section-1

Quote:

---

Hardware acceleration Enhanced

Use the hardware processing power of the graphics card to paint SWF files into the browser and accelerate compositing calculations of bitmaps, filters, blend modes, and video overlays faster than would be performed in software.

---

It clearly says swf files in browser, not in fullscreen.

Quote:

---

Originally Posted by 691175002

GPU Acceleration isn't something that can be turned on and off though. If the flash player naively did a texture change for each movieclip it had to render you would be getting performance far worse than what the cpu offers.

---

Isn't using the GPU totally transparent to the developer in flash? If we could just check for GPU acceleration when the game launched I'd think you could use that in a simple way by doing like Squize said where for example, some particle effect just uses 300 or 3000 particles based on that check.

Quote:

---

Originally Posted by Draxus

Isn't using the GPU totally transparent to the developer in flash? If we could just check for GPU acceleration when the game launched I'd think you could use that in a simple way by doing like Squize said where for example, some particle effect just uses 300 or 3000 particles based on that check.

---

The GPU just doesn't work that way. The GPU is fine if you want to render 500,000 triangles with the same texture and render state. Rendering a hundred quads each with a different texture would render most graphics cards completely useless. Every time you change the renderstate (such as using a new texture among hundreds of other possibilities) you have to completely reset a large portion of the pipeline.

Also, many parts of flash are simply not possible or incredibly complicated to implement on the GPU. Something like shape tweening would be impossible and drawing vector shapes on the GPU is also nearly impossible. For the average use case the GPU would have a hard time even matching the speed of only the CPU.

For the GPU to be useful, the objects you want to draw have to be sorted in very specific orders and if possible fall into a hierarchy of as few states as possible. There is a lot of research in the field and many complex solutions that must be present in even the simplest general engines.

I have yet to see a single GPU accelerated graphics framework/engine either 2D or 3D that was not bound by a very specific set of restrictions. Considering the backlash Adobe has gotten from the DisplayList, I would hate to see the carnage if they implemented a SceneGraph.

@tony, I've sent an email to Thibault to ask where is the truth.

Quote:

---

Originally Posted by 691175002

I have yet to see a single GPU accelerated graphics framework/engine either 2D or 3D that was not bound by a very specific set of restrictions. Considering the backlash Adobe has gotten from the DisplayList, I would hate to see the carnage if they implemented a SceneGraph.

---

webgeek told me that AS3 did use a SceneGraph? At least, he often pointed out I didn't need arrays to keep track of children on stage, because of the 'scene graph'. Might've been a terminology thing, though.

While the flash DisplayList could be considered a SceneGraph, it is far more primitive. To begin with, the DisplayList is a tree, not a graph. In a SceneGraph, nodes can have multiple parents and other oddities.

Secondly, a SceneGraph gains is flexibility and power from the fact that everything is a seperate node. If I want to draw two boxes, I might have a RenderStateNode with two child TransformNodes that both are parents of a single DrawableNode. For example, here is an example graph of a scene consisting of a truck, a road and two boxes (Omitting render states).
http://img377.imageshack.us/img377/5130/capturepx5.jpg

In general, a SceneGraph implies management over a large portion of the scene which generally includes state management and maintaining a bounding volume hierarchy as well.

Sorry it was a misunderstanding, FP10 is able to handle GPU acceleration in the browser.

Here are a few points Thibault told me :

* Flash Player 9 : Hardware acceleration only when using de fullScreenSourceRect in full screen mode.
* Flash Player 10 : Hardware acceleration GPU or Direct in fullscreen mode or not.
* The FP10 detects the drivers, and the features of the graphic cards and decides if the use of the GPU mode or Direct mode will improve the performances.
* Avoid to embed 2 hardware accelerated swf on the same html page as it could cause memory problems (hardware acceleration uses a lot of memory).
* The hardware acceleration in GPU mode, manages the video compositing (RVB YUV), bitmap transformations (roation, resizing etc...), filters. And many other things.
* The hardware acceleration renders differently (colors, anti-alias) depending on the machines.

I hope it can helps.

Interesting. Still not convinced I see the advantage of a more complicated scene graph structure, though; I can do everything I need to in Flash's current setup. Is this a structure that you have to use for the GPU to handle it quickly, or would it be possible to accelerate Flash's primitive one?

Clears a few things up, but did he give any indication of how we would know that it's on? And is it any good for blitting?

Hi Sietjp,

A little subtlety about the third point :

* The FP10 detects the drivers, and the features of the graphic cards and decides if the use of the GPU mode or Direct mode can be used.

Even if the GPU or Direct mode is used, it is not sure at all that performance will be improved. For instance if a user is has a good graphic card and a slow CPU there is a good chance he will see performance improvements. On the other hand, a user having a top notch CPU and great graphic card will not see performance improvements that much. So keep in mind that the Flash Player only test if the graphic card meets some requirements to process GPU or direct acceleration, content played and current configuration (CPU, graphic card) will then leads to better or no performance improvements.

Hi Vengeance MX,

Sadly we cannot detect at runtime if hardware acceleration is currently running.

Yes, when hardware compositing is used (GPU mode), GPU power is used to process blitting, filtering, and blending tasks.

Oh, wow. Thanks for replying. :)

So copyPixels() operations would be accelerated? What would be a better choice for this to run well, GPU or direct? I'm not entirely sure that I understand the difference.

Quote:

---

Originally Posted by VENGEANCE MX

Is this a structure that you have to use for the GPU to handle it quickly, or would it be possible to accelerate Flash's primitive one?

---

That is the real issue. Here is a simplified rundown of how the GPU works:

You set the current states (This will essentially consist of what texture you want to draw among other properties such as alpha blending). This is essentially the slowest (As in really, really slow) common operation you can do on the GPU.

You perform a Draw call which will draw hundreds-thousands of triangles all with the same state.

You repeat the steps for each different texture you have.

This works fine for 3D models where thousands of triangles may share the same texture (And perhaps 30-50 textures total in a complex scene), but in flash you could have a hundred or so textures split up over only 200 objects which will completely drown the card in state changes without really doing anything. As well, you need to group objects by their renderStates so you are not switching back and forth between two renderStates (AKA you want to render AAABBB not ABABAB). This would require a traversal and sorting of the DisplayList. (That is why things like the SceneGraph are used, if stages are nodes then no sorting is necessary)

Basically, GPU acceleration is designed to pump hundreds of thousands of the exact same thing to the screen (Which is why it is massively parallel). This is rarely the case in flash (Although it does happen sometimes).

Oh, I see. But surely there's a limit to how many textures you can re-use? There are games with randomly-generated or user-designed content, right? Also, I'm surprised that in a detailed and varied game like Crysis there'd be enough re-usage of textures to make acceleration feasible. Why can't CPUs be optimized like this?

Quote:

---

Originally Posted by VENGEANCE MX

Oh, I see. But surely there's a limit to how many textures you can re-use? There are games with randomly-generated or user-designed content, right? Also, I'm surprised that in a detailed and varied game like Crysis there'd be enough re-usage of textures to make acceleration feasible. Why can't CPUs be optimized like this?

---

It is hard to explain the techniques that a game like Crysis would use to render scenes just because it takes advantage of a lot of features of the GPU that have no CPU equivalent. Crysis takes advantage of texture splatting, detail maps, instancing and pretty much every trick in the book and still hammers the computer you run it on. Also, keep in mind that its not the total number of textures but only the number that you are rendering (You don't need to switch to a car texture if you arent rendering any cars at the moment).

User generated content is unrelated to render state switches and could be implemented in a variety of ways. Both user generated content and random content could potentially be generated/calculated completely on the GPU anyways.

The main problem is just that data in a 3d game is treated and kept in a completely different from than in flash on almost every level, and even if one bypassed all of the other issues, some parts of flash, such as tweening and vector rendering are just fundamentally not going to work very well on the GPU. So, the best solution one could put together would be performing some operations (such as bitmap operations) on the GPU and then reading it back to the CPU. The problem there is that transferring data to and from the GPU is extremely slow (The GPU is intended to push pixels to the monitor, not back to the CPU). Reading back data is almost a taboo of graphics programming. And even then, for ideal performance, the user will have to order the draw calls by texture (and hope the texture has already been uploaded to the GPU).

GPU acceleration will only be beneficial in edge cases, and under specific circumstances. Even then, when flash is standing between you and the video card I will be incredibly surprised if any major performance gains will be had (For example, you wont be able to use instancing which would probbably be the most useful technique for rendering particles and similar in flash (http://en.wikipedia.org/wiki/Geometry_instancing)). Making a separate draw call for each quad will kill performance like nothing ever seen. (http://http.developer.nvidia.com/GPU...chapter03.html ) And the sad part is that there is no way for flash to avoid making a separate draw call because it just doesn't have the right information.

Ah. Yes, Geometry instancing sounds like it'd REALLY help in a game like this. Presumably it's why Left 4 Dead can handle so many enemies; there are only a handful of different types, and they keep getting re-used.