Multithreading and 64bit

[TheScruffyBandit 382]

We're all very much aware that the engine dayZ is based on is rather dated and while it's certainly capable of producing a very pretty game has some issues getting the most out of modern hardware. Obviously, nearly everyone is aware of the current object related performance issues causing poor framerates and Dean has aleady made it clear that the team is working on them, however, in a world where the vast majority of PCs being used for gaming have 4 or more CPU cores and more than 4096mb of addressable memory, I think a it should also be a key concern to improve multithreading and also converting the games code to utilize 64 bit systems.

 

While I'm aware how time consuming both of these issues are to address. Partically the conversion of code from 32 to 64bit for something as complex as a videogame engine however I feel it's extremely important given the amount of ideas and concepts that we've been told are to be implimented including things like every item having physics and increased numbers of zombies, I find it hard to believe that DayZ standalone will be able to provide reliable, smooth gameplay on even high end systems while only fully utilizing one CPU core and 4096Mb of memory, especially when you consider that most modern GPUs themselves have 2048Mb of memory, if not more.

 

I would be very suprised if with all those additional fuctions and concepts if DayZ will ever truely run WELL in 32bit, simply due to lack of usable memory.

 

I would asume improved multithreading on the client side is already on the cards at somepoint in the future, given how CPU bound the engine can It wouldn't make much sense to barely use more than 1 core, give that this would likely limit the potential customer base to those users with heavily overclocked i5/i7 cpus and even then, provide somewhat limited framerates, but I would like to see this addressed sooner rather than later if possible as currently, even brand new CPUs are stock speeds are not providing what I'd consider comfortable framerates.

Edited March 4, 2014 by TheScruffyBandit

[Lady Kyrah 1109]

"Dated"... I really wish people would stop spreading this kind of FUD, there is no such thing as dated code, it's called stable.

 

Last time i checked the RV engine IS multi-threaded.

 

Eh the x64 fad again, most games won't bother until x86 is definitely gone. All you really need to get the memory benefits is the DayZ executable to be Large Address Aware.

Edited March 4, 2014 by Lady Kyrah

[TheScruffyBandit 382]

Last time i checked the RV engine IS multi-threaded.

There is some multithreading, but it's rather limited and not particually effective. It's fairly obvious if you watch your CPU usage across all cores while playing.

[Lady Kyrah 1109]

There is some multithreading, but it's rather limited and not particually effective. It's fairly obvious if you watch your CPU usage across all cores while playing.

At the same time, what do you expect them to put in separate threads, i think they do disk IO on a thread, model prepping on another and AI on another, or something like that.

 

Multithreading isn't like flicking a switch, you need to figure out which parts of your application can run asynchronously on separate threads. When it comes to games it gets even harder considering that there isn't a lot that you can effectively do "while doing something else". Most of the cpu side of rendering is a process that goes from start to finish and where each step usually depend on the previous one, you can't multithread that.

[damagefilter 34]

So true.

That multithread-mania must be wiped off peoples minds.

Multithreading is not the magic solution to all the performance problems either.

 

And x64...

I don't think it's feasible nor required at all.

But mostly not feasible.

It's not an easy task to introduce 64bit proof code to a project and at the same time don't break the 32bit mode.

Which is what they would probably want to do because, although rapidly getting less and less,

there are still people out there with x86 processors. I presume it's still a significant margin.

 

Yes, the game does need a bunch of performance tweaks, that's obvious.

But those proposed here are not the solution.

The solution lies elsewhere, in things like interest management (or improved IM, if they already got it) and improved visibility culling etc.

You said yourself (kinda sorta, you get the point, I hope), the engine uses barely all the resources available.

That's likely not because it doesn't want to, it simply doesn't need to. Your memory?

Not full and still getting lag? That's not because RV doesn't want to put more stuff into memory, it's because there is nothing left to put in.

 

My 5 €cents.

Edited March 4, 2014 by damagefilter

[jamie721 3]

finaly some one slightly compatent

[TheScruffyBandit 382]

there are still people out there with x86 processors. I presume it's still a significant margin.

There aren't any x86 cpus that meet the minimum requirement for this game, I don't see legacy support as an issue here.

[DarkwaveDomina 1099]

Eh the x64 fad again, most games won't bother until x86 is definitely gone. All you really need to get the memory benefits is the DayZ executable to be Large Address Aware.

Either/or ASAP please.

[KarmaCoin 72]

Games like this don't hit the CPU as hard as you are thinking they do. They primarily hit the GPU which is why your frame rate is lower in cities than it is in the wilderness. The CPU is not working much harder in a city versus in the wilderness but the GPU has to render a lot more objects.

 

I would guess that some combination of the following things are why frame rates are low in DayZ:

Objects' polygons count are too high,

Object density is too high (especially is cities),

Draw distance is too high,

Rendering algorithms may be slightly dated (I'm assuming DayZ uses DirectX but which version I don't know)

[TheScruffyBandit 382]

Games like this don't hit the CPU as hard as you are thinking they do. They primarily hit the GPU which is why your frame rate is lower in cities than it is in the wilderness. The CPU is not working much harder in a city versus in the wilderness but the GPU has to render a lot more objects.

 

I would guess that some combination of the following things are why frame rates are low in DayZ:

Objects' polygons count are too high,

Object density is too high (especially is cities),

Draw distance is too high,

Rendering algorithms may be slightly dated (I'm assuming DayZ uses DirectX but which version I don't know)

https://www.youtube.com/watch?v=9ssjr36NKTE

 

GPU usage drops in cities. Granted, it shouldn't be and I believe they're trying to fix that... but it still does.

[damagefilter 34]

Have you ever thought why it does that? Dropping the GPU usage I mean?

That is because foliage needs far more time to render than any old house.

Why?

Because it's transparent.

The renderer needs to do a lot more things on trees / transparent objects than it needs to do on the average house, which is solid for the most part.

 

Funny enough, RV is pretty good with rendering foliage of all sorts.

But yeah, it would boil down to object density and polycount.

Although (according to steam, if I remember that right) RV seems to utilise DX10 so polycount shouldn't be all that much of a problem after all.

[KarmaCoin 72]

https://www.youtube.com/watch?v=9ssjr36NKTE

 

GPU usage drops in cities. Granted, it shouldn't be and I believe they're trying to fix that... but it still does.

 

Very interesting video. Clearly something is messed up because GPU usage should not decrease when rendering objects. If anything it should increase.

[KarmaCoin 72]

Have you ever thought why it does that? Dropping the GPU usage I mean?

That is because foliage needs far more time to render than any old house.

Why?

Because it's transparent.

The renderer needs to do a lot more things on trees / transparent objects than it needs to do on the average house, which is solid for the most part.

 

Funny enough, RV is pretty good with rendering foliage of all sorts.

But yeah, it would boil down to object density and polycount.

Although (according to steam, if I remember that right) RV seems to utilise DX10 so polycount shouldn't be all that much of a problem after all.

 

I kind of follow what you are saying but I don't understand why the frame rate also decreases when rendering the objects since the GPU is clearly not under stress.

 

Also the GPU was at full utilization when just looking at the ocean which has no foliage

Edited March 4, 2014 by KarmaCoin

[TheScruffyBandit 382]

Have you ever thought why it does that? Dropping the GPU usage I mean?

That is because foliage needs far more time to render than any old house.

Why?

Because it's transparent.

The renderer needs to do a lot more things on trees / transparent objects than it needs to do on the average house, which is solid for the most part.

Your logic doesn't add up. You may also note the system never gets close to 100% CPU usage in that video, if the towns are causing GPU usage drop due to requiring less rendering performance that still wouldn't explain the drop in framerates without an increase in CPU usage.

 

:rolleyes:

Edited March 4, 2014 by TheScruffyBandit

[damagefilter 34]

I shall explain!

 

In digital gaming, there exists a thing that is called a "draw call".

There is a rather easy fomula to draw calls. The less, the better.

Thise thigns are extremely expensive to have.

 

A draw call is a thing that happens on your GPU. It happens the moment the GPU needs to swap a texture with a different texture,

in order to put it on a mesh (for instance a house).

Not swapping it in/out of the VRAM. Within that there is plenty of space for a lot of textures.

Talking about the place where stuff gets drawn so you can see it.

Simply speaking. We're not the tech course at the university so I suppose that'll do just fine.

(For the records: I am not a student)

TLDR:

Draw Calls are super expensive tasks but neccessary and unavoidable that happen on your GPU.

 

 

Then, there is a thing called draw call batching.

That is, when the renderer is somewhat intelligent.

It looks at the data it needs to render and then pushes all the things towards the GPU that share the same texture.

In the case of DayZ that means: If you're out in the forest, there's a lot of demand on the GPU, due to all the transparent things, depth testing, possibly some FXAA etc.

But the amount of draw calls will be low, because all the sorts of trees share the same texture. In fact, most of them are even the same mesh.

 

So instead of 100 draw calls you get 10. For each sort of tree there is one because GPU needs to swap the textures once, for each.

TLDR:

Draw calls can be batched with objects that share the same material so we can save a couple of them to gain performance.

 

 

And this very draw call batching, this cannot happen in a city where there are so many things that need rendering and do not share a texture.

Like all the items, the terrain, some trees, each and every house, house interiour ...

 

Now, draw call batching is not expensive in the sense of "using a lot of your GPU power". It just takes a moderately long time to swap textures.

It's like when you draw a picture and then put colours on it. Changing the colour on your brush isn't really an exhausting task, but it takes time.

 

Right.

Now, I'm not the engine developer at bohemia so ofc I wouldn't know to what extend this all applies (especially draw call batching, not every renderer does that).

But I hope, this sufficiently explains to you how I came to think of things like I do.

 

Edit (because it's not enough text yet xD)

 

Also the GPU was at full utilization when just looking at the ocean which has no foliage

That's because rendering a water shader is even more expensive. I kid you not. Feel free to look it up :)

 

Edit 2:

That whole thing is based on the fact that your FPS are directly influenced by how long a single frame needs to get painted.

It takes longer when there are more draw calls as they take some time. (As mentioned)

And if frames take so long to render that they cannot get up to 60 per second, well your FPS drop down ofc.

Edited March 5, 2014 by damagefilter

[KarmaCoin 72]

I kinda see what you're saying but I still don't follow why GPU usage drops. I understand how a city may take longer to draw than a forest of identical trees but I don't understand how this results in lower GPU usage.

 

At any given time the GPU is either executing code or not. To calculate GPU usage you ask the GPU if it is executing something 1000 times a second. Then take the number of times it was executing something and divide by the total times sampled. So if it was executing something 580 times out of the 1000 you sampled the GPU usage would be 58%.

 

So in the video when he is looking at the houses the GPU usage is like 40-50% which means that the GPU was not doing anything most of the time. To me this suggests some sort of bottleneck in getting to the draw calls. They seem to not be getting called as frequently as they should be considering the GPU is mostly idle.

 

Seems like something needs to be better optimized because the game is not fully utilizing the resources available during harder frames.

[damagefilter 34]

I'll try to explain further ^^

 

A draw call, no matter how much of a beast your graphcis card is, takes a very very long time to execute, however, in itself does not utilise any of the graphics processing resources.

All it does is swapping texture in and out of the buffer it uses to paint things.

See, it doesn't need to process anything at this very time in regards of GPU usage.

The GPU usage comes into play after the textures were swapped, when it starts painting.

It cannot paint however, when it is instructed to swap the texture first.

 

Now, if you have, lets say, 200 draw calls (entirely possible number) per frame in a scene, than we need to swap a texture in and out of the buffer 200 times per frame. 

To get 60 FPS that means your gfx card must swap a texture 200 times and draw a thing 200 times in the 60th of a second; Then do it again for the next frame.

Physically, due to the draw calls taking so much time(because there are so many of them), you get FPS drops because it can't swap that fast.

 

At the same time, due to what exactly is rendered (solid objects, not much transparency, simple shaders in cities), the GPU usage drops because the stuff it needs to paint,

isn't that complex at all, compared to, as mentioned, a water shader, where there is vertex manipulation, refraction, transparency ... or trees, where there is loads of transparency

and whatever else they put into their shader to get the effects they have.

 

So, GPU usage in itself is independent of draw calls. It can be high, while there are maybe 5 drawcalls in a scene, it can aswell be low, depending on what needs to be painted.

 

Cities: Trivial painting process, low GPU usage because of that but loads of texture swaps, resulting in frames taking longer to paint because lots of drawcalls.

Forest: More complex painting process, resulting in higher demand on the GPU, not much texture swaps, faster frame paintingbecause less draw calls.

 

But then again.

That's only theory. Only Bohemia knows how exactly their engine works, I sure as hell don't.

But, I like to think I know a bit about that kind of stuff so this makes sense to me (at least that xD).

Edited March 5, 2014 by damagefilter

[KarmaCoin 72]

Memory can't move itself. Swapping textures in and out of memory should still cause utilization of either the GPU or the CPU because a processor must execute code to move memory. In the video, the GPU usage drops and the CPU usage does not increase. So what you're saying does not make sense to me. The system is doing less during these scenes not more which is not how it should be. 

 

Also the author of the video states that he is using an Nvidia GTX 780 which has 3GB memory. This should be more than enough to keep these textures in memory the whole time and so swapping should be pretty minimal.

 

I'm not sure if the issue is engine related or something that the DayZ team is doing wrong, but if you run the same test in other games you do not see the same strange behavior.

[BEWARE-OF-ME 2]

I've got an AMD Radeon HD 7850, runs BF4 pefectly, runs World of Tanks perfectly, runs War Thunder perfectly, runs just about anything, at a good rate. With only about 75-80% CPU being used.

 

Barely is able to run DayZ Standalone.

 

BF4 is one of the most graphically advanced games yet, I can reach a solid 50FPS on high settings. (it's one of the best looking & playing). Hell my old HD6670 even ran it at 25fps.

 

DayZ on the other hand does not have the same graphical advancement *yet* (basically dayz doesnt look as good as bf4).

 

Textures and high poly models fuck your cpu and gpu over sometimes, yet even running DayZ SA on the lowest possible settings outputs an irregular FPS rate for my card. (In towns -> 14-19 fps) (countyside/unpopulated areas/coast -> 30-35fps)

 

Why? Because DayZ Standalone is horribly written and is archaic as far as technology advancements go. It's only utilizing 4gb of RAM, when some PC's even have 32gb! It's got limited multithreading support, but if they fix it as a priority, more people may be persuaded to buy the game with great performance. But the reviews on performance are doubting what the game specifications show.

 

And I shouldn't need to go waste money on an i7 just to play 1 game properly.

Edited March 6, 2014 by BEWARE-OF-ME

[damagefilter 34]

I was not talking about texture swapping in the VRAM.
And I was not saying that (well I didn't mean to say) that swapping textures in a buffer doesn't need any form of processing.
Ofc it does, as you already mentioned.
I was talking about the buffer that contains the texture that is about to be drawn.

This is not about thr RV engine, but here's a link to the unity forums where user pete explains that muchos better and much shorter than I ever could ^^

http://forum.unity3d.com/threads/27416-What-are-Draw-calls

 

But ofc, that thing does act quite weird. No doubt about it.

Only, the point of the whole thing, lets not forget the topic here, is that I think that slapping on 64bit support and more threading is just not the solution.

And the problem, in my opinion likely being too much and too slow draw calls (as CPU usage seems to be unaffected for the most part), needs to be tackled differently.

 

If, in the end, this is all about draw calls or not - they'll see, sooner or later.

Although I think I have a point with draw calls (I wouldn't have said it otherwise ^^)

[BEWARE-OF-ME 2]

@damagefilter thanks for telling us about that, but that doesn't explain why a game such as Bf4 which probably has to swap a texture lots of times (idk how much, but more than dayz) runs better than a game that most likely swaps less textures.

 

And if water is more complex why would I be getting 75fps+ when looking at it lol

[Lady Kyrah 1109]

Water is a very complex shader but it's only used once in a scene so it doesn't really matter.

 

My best guess is that CPU transform skyrocket in cities due to the vast amount of unique buildings and varied objects. Add to this a boatload of textures, building interiors (remember that arma2 didn't have many interiors for a very good reason), and all the loot objects that can potentially be seen. (automated object to object culling being awfully slow it's prolly not used)

 

Forests are cheap when it come to draw calls and transforms because they are just many instances of the same handful of already very optimized (draw call wise) objects. But that's still a crapload of triangles to render, so while you're saving up on draw calls, you have a massive amount of semi transparent surfaces to light, render and SHADOW (i'm stressing this part).

[KarmaCoin 72]

Only, the point of the whole thing, lets not forget the topic here, is that I think that slapping on 64bit support and more threading is just not the solution.

 

Totally agree, the CPU clearly is not bottle necking as it stays at a good 70% usage throughout the entire video.

 

This is not about thr RV engine, but here's a link to the unity forums where user pete explains that muchos better and much shorter than I ever could ^^

http://forum.unity3d.com/threads/27416-What-are-Draw-calls

 

I get how cities require more draws and how that could reduce frame rate if the GPU bottle necked, but the GPU is not bottle necked in the video. The GPU usage actually drops which is the reverse of what I'd expect. Maybe it's just an inability on my part to understand but I feel that something else is going on here.

[Pokerguy12 218]

 

I would asume improved multithreading on the client side is already on the cards at somepoint in the future, given how CPU bound the engine can It wouldn't make much sense to barely use more than 1 core, give that this would likely limit the potential customer base to those users with heavily overclocked i5/i7 cpus and even then, provide somewhat limited framerates, but I would like to see this addressed sooner rather than later if possible as currently, even brand new CPUs are stock speeds are not providing what I'd consider comfortable framerates.

 

YOU SEE THIS DEV TEAM!! YOU LOSE PROFITS $$$$$$$$$$$$$$$

[wirher 8]

Water is a very complex shader but it's only used once in a scene so it doesn't really matter.

 

My best guess is that CPU transform skyrocket in cities due to the vast amount of unique buildings and varied objects. Add to this a boatload of textures, building interiors (remember that arma2 didn't have many interiors for a very good reason), and all the loot objects that can potentially be seen. (automated object to object culling being awfully slow it's prolly not used)

 

Forests are cheap when it come to draw calls and transforms because they are just many instances of the same handful of already very optimized (draw call wise) objects. But that's still a crapload of triangles to render, so while you're saving up on draw calls, you have a massive amount of semi transparent surfaces to light, render and SHADOW (i'm stressing this part).

It seems to be a lot of triangles but performance shouldn't be bad there because of batching technique. This way single draw call can render a lot of instances of the same object. It may be confusing, because polycount of the scene stays the same but batching can save a lot of time between frames. Forests shouldn't EVER drop below 60fps with vsync on $300 cpu and gpu but it does.

DayZ SA looks decent today. It is huge open world, however today with occlusion culling, LoD and other tricks we can have almost infinite game world and keep great graphics. If anyone says "dayz has bad performance because the game world is big" remember that's bullshit - it's just not optimized well enough.

Edited March 7, 2014 by wirher