How to fight the abuse of the gamma settings

[tommekk 26]

This is a suggestion for a post-processing shader to improve the gameplay at and the look of the night in DayZ. It makes the abuse of the gamma and brightness settings almost useless (especially for PvP) without restricting these settings.

---Update 2016-06-12---

• Updated description
• Created thread in suggestions section of the forum

---Update 2015-12-18---

• NEW: Uploaded source to pastebin.com for public testing
• NEW: Added controls to toggle the shader and the gamma correction independently at runtime
• Updated and tested the shader for ReShade with ReShade 1.1.0
• Performance optimization, code clean-up and updated documentation/comments
• Further optimization for low visual impact while playing with normal gamma settings
• New images

---Update 2015-02-23---

• NEW: Shader for ReShade (HLSL)
• NEW: Shader for SweetFX (HLSL)
• NEW: Shader & Sketch for Processing (GLSL)
• Optimized performance
• Optimized for low visual impact while playing with normal gamma settings
• New images

---Submitted 2013-12-18---

0) INDEX:

0) Index
1) Theory
2) Implementation
3) Results
3.1) Images
3.2) Performance
4) Summary
5) Conclusion
6) Source
6.1) Installation and use

1) THEORY:

If you aren't interested in the theory, skip to the "RESULTS" section to see the screenshots.

One important problem in DayZ on night time servers is the abuse of the gamma settings as it is easy to change the in-game gamma-/brightness settings, the systems graphic settings or even the display settings accordingly. And since it is that easy and literally undetectable many players seem to use it for their advantage today. This forces each player to decide to either play with normal gamma settings and having a disadvantage but a nice looking game or to increase gamma to have a greater chance to survive but everything looks awful. Aside from its aesthetic appeal the night doesn't make much sense if the main aspect, the restricted vision, can be circumvented so easily like at the moment.

 
Fig. 1: Gamma
Comparison of normal gamma settings (left) and abusive gamma settings (right)

To make things worse the nights in DayZ (and ArmA 2 & 3) seem to be either to dark or to bright:
On the one hand we have the very dark nights around new moon which are quite realistic in terms of lighting: After all it's the apocalypse; the power grid seems broken down, so large scale electric light sources shouldn't work. But then it is nearly impossible to even navigate, not to mention looting, fighting, etc. and this could lead even players not really interested in PvP to exploit the gamma settings. On the other hand we have the nights around full moon which are so bright that you can see and recognize players several hundred meters away. But this is neither realistic nor authentic.
Opposed to the big difference between full and new moon nights the difference between winter and summer nights seems quite small: While the winter nights are OK, the summer nights seem too dark and you can see much less than in the real world.

Fig. 2: Lighting
On the left an example of the relatively dark new moon, summer nights at 2035-06-12 20:20 in-game time. At least in Central Europe this darkness isn't reached until around 23:30 at this date.
On the right an example of the relatively bright full moon nights.
(The white object in the background is a radome)

But only changing the ambient light wouldn't be a solution as the problem is not just the lighting, but that the maximum "useful" view distance, up to which you can recognize important details, doesn't correlate with the actual brightness of the ambient light. If the brightness exceeds the level at which you are able to recognize your close surroundings, you can also recognize details hundreds meters away. But below that level you almost instantly can't see your hand in front of your face. What is missing is a continuous transition from details being recognizable to details being not recognizable for the transition from day to night and back.

 
Fig. 3: Details
While on the left there are some details visible both nearby and a few hundred meters away too, there are no significant details visible on the right, neither close nor at a distance.

In the real world at some point everything looks more and more blurred the darker it gets and even in a full moon night you are barely able to recognize an idle person >100m away in a field of tall grass or >30m in the woods. But you are able to see and recognize large objects like trees, large rocks or bushes at great distance but way less detailed compared to daytime.

To understand why this happens we have to look at the physiological characteristics of the human eye. The human eye contains mainly two different types of photoreceptor cells; the cones and the rods:

Fig. 4: Retina
Simplified representation of the retina showing the photo receptive rods (gray) and cones (colored) and some of the neurons (brown) responsible for the summation of the values of several rod cells and for connecting all photo receptors to the next layers of the retina.

The cone cells are responsible for the vision in daylight. They provide color vision and are connected to the brain more or less one by one to achieve a high resolution, but they need relatively bright light to work.
At night the rod cells take over. They are about a hundred times more sensitive to light compared to the cones but can not differentiate between colors. Additionally they form clusters of several rod cells to accumulate and amplify the light, leading to even more sensitivity. But because each of these clusters is connected to the brain as a whole, instead of every single cell on its own, they provide a lower resolution. That leads to a blurrier vision at night compared to the vision in daylight.

To simulate this behavior the shader needs to blur each pixel depending on the brightness of the area around it. The darker an area is, the more it gets blurred: An object below a bright street light isn't modified at all, while a dark tree line gets heavily blurred.

In addition to the more natural look of the night the shader significantly reduces the benefits of abusing the gamma settings: If you increase the gamma value the dark areas may be brighter but stay blurred and the removed details can not be recovered. While the shader is optimized for low visual impact when playing with normal gamma settings, you still can't reveal significant additional details by increasing gamma.

 
Fig. 5: Effectiveness
Comparison of abusive gamma settings without (left) and with the shader (right)

This leads to a much wider range of brightness of the ambient light usable for balancing:
You neither need pitch black nights to restrict vision nor you need to reveal details hundreds meters away if you just want to allow player to travel the map at night without additional light sources. Besides that you wouldn't have to restrict the gamma correction itself as simply restricting the gamma correction is unfair to those players who really need it.

While the blurriness is barely visible it could induce gaming sickness for some sensitive players. This is most likely to happen at some point during the in-game (nautical) twilight. At this time the ambient light reaches a level at which most of the surroundings are already a bit blurred but it is not yet dark enough to "cover it up". While this is the same in the real world, it is rather unusual in computer games. This discrepancy between expectation and reality could cause negative symptoms for some players.
But it is also possible that the shader reduces the symptoms for other players if these symptoms are caused by small, noisy, barely visible details in the dark surroundings. As these details are hard to see, the tracking and focusing of them could draw much concentration or be tiring for the eyes and lead to headache or dizziness. Since the shader removes exactly these details it could reduce or even prevent this sort of gaming sickness.
To evaluate these assumptions more players need to test the shader and report back. I know from my own experience that it takes a while to get used to the slightly changed look and feel of the game. When you test the shader for the very first time or the first time after a bigger change of some parameters it can take up to two hours until you have adapted to it. That means you shouldn't give up too early and keep on testing for a while even if you aren't pleased by the results right from the start.
As described above; the shader is configured to cause as little visual blur as possible. This leads to even less blurriness than in the real world and should be kept in mind when comparing the results to real world examples.

2) IMPLEMENTATION:

The following descriptions of two possible approaches are simplified as the detailed ones would go beyond the scope of this overview.

Fig. 6: First approach
rectangle - data
ellipse - processing step

The first approach uses adaptive blurring of the image:

1. The luminance of the neighborhood of each pixel is determined by calculating the luminance of each pixel, save the result in a separate buffer and then blur it with a fixed radius.
2. The result of step 1 is used to calculate the radii for the actual blurring of the frame. For this a function similar to 1.0/luminance is used. That means the brighter the neighborhood of a pixel is, the less blurred it gets.
3. The frame gets blurred according to the result of step 2.

 

Fig. 7: Second approach
rectangle - data
ellipse - processing step

The second approach uses the blending of the original and a blurred version of the image:

1. To get the luminance of the neighborhood of each pixel the image is blurred first and the result is saved in a separate buffer. Then the luminance is calculated and saved to another buffer.
2. Almost the same function like in the first approach is used to calculate the proportion of the original image and its blurred version for each pixel.
3. The original image and its blurred version are blended pixel-wise according to the result of step 2.

If we want to compare the two approaches we just need to examine the third step as the first two steps are almost the same for both approaches.
It is easy to see that the second approach is faster because a simple interpolation is faster than a convolution with a kernel of reasonable size.
When it comes to the efficiency of the gamma exploitation prevention the first approach is superior. This becomes clear when we look at the frequency spectra of the results of both approaches:

We assume that all frequencies of the input are equally distributed so its spectrum looks like this:

Fig. 8: Input spectrum
Homogeneous input spectrum

Then the output of a shader using the first approach would have the following spectra:

Fig. 9: First approach output
The output spectra of the first approach for different light intensities.
From left to right: Day, early twilight, late twilight, midnight.

As you can see the darker the input is the lower is the cut-off frequency. All frequencies significantly higher than the cut-off are suppressed enough to be technically unrecoverable as their amplitude reaches virtually zero. So more and more details are removed beginning with the smallest(highest frequency) ones.

For the second approach we assume the same input and the following spectrum of the blurred version of the input calculated by the first step:

Fig. 10: Fixed radius blur spectrum
The low-pass filtered intermediate result calculated during the first step.

Then a shader using the second approach would have the following output spectra:

Fig. 11: Second approach output
The output spectra of the second approach for different light intensities.
From left to right: Day, early twilight, late twilight, midnight.

In contrast to the first, the second approach lowers the amplitudes of all higher frequencies altogether. That means that unless in complete darkness all higher frequency details have significant non-zero amplitudes and could be recovered by unsharp masking. This renders the second approach ineffective against gamma exploitation as even a simple sharpen effect of the graphics card or the display could recover significant details.
As the effectiveness against gamma exploitation is one important requirement the shader has to  meet, it has to use the first approach.

In addition to that you need to consider the following:

• The shader should be applied prior to the tone mapping to guarantee that it only takes effect in low light environments like at night and not on dark surfaces at day
• If you aim for physiological correctness you would need to configure the 2. step that the effect starts to fade in at the same luminance at which the colors start to fade out as at this time the transition between day- and night vision starts
• To reduce the abuse of the gamma settings it has to be applied prior to the gamma correction.
• If you plan to allow post-processing injectors like ReShade and shader suits like SweetFX (I hope you will!) then you need to decide whether you want to make the depth map accessible or not. Because if you do, you need to blur it too; otherwise it could be exploited as well to get something like "bat-/sonar-vision".

If you follow these recommendations then you should have a fair solution, because everything has exactly the same level of detail for everybody regardless of their gamma- and brightness settings.

The shader could be controlled server side, as it might lower performance on very old systems or could induce gaming sickness, but would really improve the fairness. Then it would be up to the player to choose between servers with and servers without the shader, but if they choose one with it enabled, they can be certain everybody uses it.

3) RESULTS:

3.1) Images:
The screenshots were taken with ArmA 2 & 3, because it is easier to get a greater variety of scenes while ArmA and DayZ are comparable in terms of brightness/gamma. The previous screenshots (2015-02-23) were taken with ArmA 2 CO in Chernarus, the new ones (2015-12-18) with ArmA 3 in Chernarus and Altis.

Fig. 12: Example 1
NWAF

 

  
Fig. 13: Example 2
Hills north of Chernogorsk

 

  
Fig 14: Example 3
Country house on Altis

More examples in the same order like above for every row:

New screenshots (2015-12-18):
http://imgur.com/a/7G0b7/all (Chernarus)
http://imgur.com/a/kuhsl/all (Altis)

Previous screenshots (2015-02-23):
(These are outdated, but I left them for comparison)
https://imgur.com/a/1wWjW/ (Chernarus)
https://imgur.com/a/S7Db4/ (Chernarus)

3.2) Performance:

Tested on:

• AMD Phenom II X4 965 Black Edition @ 3,4GHz
• NVIDIA GeForce GTX 560 Ti
• ASUS M4A89GTD-PRO/USB3
• 12GB RAM @ 1333MHz

With ReShade 1.1.0 and the recent version of the shader.

Test of ArmA 2 CO:
The Shader takes 0.1-0.2ms to process one frame.
No reduction of the frame rate observed.

Test of ArmA 3:
The Shader takes 0.01-0.02ms to process one frame.
No reduction of the frame rate observed.

The difference of one magnitude in the processing time between ArmA 2 and ArmA 3 might be a result of the different DirectX versions used by both engines.
The shader should be at least as fast as in the test with ArmA 3 if it is included directly in the post processing of the new renderer (DirectX 11).

4) SUMMARY:

Pro:

• Combats the abuse of the gamma correction without restricting the gamma correction itself
• Improves realism and authenticity
• Gives the night a more natural look
• Good performance

Con:

• Might reduce the performance on very old or minimalist systems without dedicated graphics
• Could cause some problems for players who are prone to gaming sickness

5) CONCLUSION:

The shader improves the authenticity and gameplay without changing the rest of the game or forcing the players to certain gamma and brightness settings.

6) SOURCE:

The source can be found at:
http://pastebin.com/Cm4y12Qq

The attached source files are outdated and for the record only.

6.1) Installation and use:

IMPORTANT NOTE: The shader hooks into the renderer and thus it might modify the memory of the running game. This could be classified as a sign of cheating/hacking and because of this it is recommended to test the shader in singleplayer/editor with disconnected networking. After testing and before reconnecting you should uninstall it by deleting the two files you copied during the installation. That might sound a bit paranoid, but better safe than sorry... :)

During the installation you need to rename two files and change their file extensions too. For this you need to activate/show the file extensions in the windows explorer, if you haven't done it already:
http://windows.microsoft.com/en-us/windows/show-hide-file-name-extensions

1. Download ReShade
   During the testing Reshade is needed for hooking into the renderer and loading/starting/toggeling the shader. It is available at http://reshade.me/
2. Copy ReShade32.dll from the downloaded archive into ArmAs main directory (same directory like the arma2.exe/ArmA2OA.exe/arma3.exe)
3. For ArmA 2 (OA/CO) rename the copied ReShade32.dll to d3d9.dll, for ArmA 3 rename it to dxgi.dll
4. Download the shader file
   http://pastebin.com/Cm4y12Qq
5. Rename it to ReShade.fx
6. Copy this ReShade.fx into ArmAs main directory too.
7. That's it. If you now start ArmA ReShade should load the shader.

Press [Insert] to toggle the shader on/off
Press [Home] to toggle gamma 1.0/2.0

ReShade is not compatible with software like screen capturing or overlays and you need to deactivate them in order to test the shader. To take screenshots ReShade has its own screenshot functionality (but sometimes it seems to miss some keystrokes):
Press [Print] to take a screenshot (they are saved to the same directory where you put the shader files)

Edited June 12, 2016 by tommekk
formatting

[Troll_Hunter 54]

Wow what a big post tommekk!

Great idea of obscuring the dark(er) parts.

However I don't like the rounded blurry blue and red blob like edges.

Look at what happens to camera's that are underexposed, with applied high gain (ISO). With the increasing grain in the dark parts of the image (not in the light parts!) the details in the dark are obscured by the grain / noise.

With eyes, in low light, the detail is also greatly reduced, but the our eye's grain looks more organic. hehe. The medium and light parts have little to no grain / noise. I like to see BI develop such a shader, making an truely organic looking grain, that is applied according to the games light levels.

With this organic grain, players will be on a level playing field AND have a more realistic night-time / cave experience! - Win win!

[tommekk 26]

1 hour ago, Troll_Hunter said:

However I don't like the rounded blurry blue and red blob like edges.

I assume you are referring to Fig. 5b (the  image on the right). These red and blue blobs wouldn't be visible with normal gamma settings because these areas are too dark. (At least it is like that with my gamma settings)

Lets take Fig 14 as an example: Fig 14a is what a legitimate player sees now and Fig 14b is what the legitimate player would see if the shader is activaded. With my gamma settings (game, system and display) there is no difference visible. Fig 14c and Fig 14d are the same images with "not so legitimate" gamma settings. (I think I should add a description...)

1 hour ago, Troll_Hunter said:

Look at what happens to camera's that are underexposed, with applied high gain (ISO). With the increasing grain in the dark parts of the image (not in the light parts!) the details in the dark are obscured by the grain / noise.

With eyes, in low light, the detail is also greatly reduced, but the our eye's grain looks more organic. hehe. The medium and light parts have little to no grain / noise. I like to see BI develop such a shader, making an truely organic looking grain, that is applied according to the games light levels.

 

I have been experimenting with adding noise to the result to mitigate the use of advanced countermeasures like an adaptive deconvolution. This could be adapted to achieve a more organic grain too. I think I could readd and configure it accordingly in the next iteration of the shader. :-)

 

[Troll_Hunter 54]

Hehe Tommekk,

Thanks for the clarification.

You need to charge BI when you have made the smooth organic darkness shader :D - Good job man!

Take note there is also another 'Gamma' topic, perhaps offer you next (smooth fading organic noise) shader design there too.

Maybe you can record an animated gif of a screenshot + your organic smooth noise shader at work? - From film I know it's rather challenging to make grain / noise look smooth and organic, like we experience in a very dark situation. DSLR's noise usually shows the chip's read-out patters, and when seen at pixel level the contrast between pixels is not looking smooth and organic. My eyes see more like small smooth rounded blobs, nearly devoid of colour.

I'm looking forward to it.

[Asmondian 372]

What a great idea. Don´t know if "the solution" but is the path we need to follow to make the game a real survival.

Thx for the good work.

[shinjuru 114]

Tremendous effort!   It's too bad all of the video cards and displays used for playing DayZ Standalone aren't exactly the same.

[tommekk 26]

7 hours ago, Troll_Hunter said:

Maybe you can record an animated gif of a screenshot + your organic smooth noise shader at work?

That is not easy as ReShade isn't compatible with screen capturing software (at least not with Fraps AFAIR) and is one of the reasons why I made the sources public as the screenshots can show only a part of the results. A good example is low scrub: A screenshot only shows "boring" dark blobs but if you see the scene in motion it is a whole different story. Especially if you set the blur to a realistic level. (At the moment it is set quite low to reduce possible negative effects like gaming sickness.)

7 hours ago, Troll_Hunter said:

My eyes see more like small smooth rounded blobs, nearly devoid of colour.

Maybe the old screenhots made with ArmA 2 would fit your description better than the ones from ArmA 3:

In the end it is a matter of the balancing as the shader provides a wide range of possible settings. And if it would be integrated directly into the renderer, you would have even more control over the results.

 

7 hours ago, Asmondian said:

Don´t know if "the solution" but is the path we need to follow to make the game a real survival.

OK. At the moment it is more of a proof of concept than a finished solution but the integration into the renderer should be a step closer to that goal, especially because it would offer more options and control over the configuration.

[tommekk 26]

On 13.6.2016 at 2:16 AM, Troll_Hunter said:

Maybe you can record an animated gif of a screenshot + your organic smooth noise shader at work? - From film I know it's rather challenging to make grain / noise look smooth and organic, like we experience in a very dark situation. DSLR's noise usually shows the chip's read-out patters, and when seen at pixel level the contrast between pixels is not looking smooth and organic. My eyes see more like small smooth rounded blobs, nearly devoid of colour.

I'm looking forward to it.

Here we go:

I added some adaptive noise (additive and multiplicative) to the shader, took like two dozens screenshots and assembled them into a .gif. It's important to note that the conversion to .gif messed with the colors; everything became a bit reddish and some dark red blobs appeared. Because of that I'll add one original frame I used for the .gif as a comparison.

At first the original scene without any additional shader applied:

 

The second is the .gif to demonstrate the look of the noise. (The amount of it is a bit exaggerated to make it more visible and the colors are off due to the consersion to .gif.):

 

And here we have one frame of the .gif before its conversion to .gif to compare the colors with:

 

This version of the shader isn't optimised and not really ready but FEEDBACK is welcome!

Not only about this version but especially about the one in the OP. Has somebody already tested it? No? Do it! :D

I hope you like it and I'm looking forward to get some more feedback.

[Troll_Hunter 54]

I love it.

It looks smooth, but still a bit digital (maybe due to the short gif repeating ?), like a CMOS chip, but the noise has the right amount of variation and smoothness.

Suggestion: make a 3 layered shader, with the first shader blurring the dark area's according to their darkness: light level 0 (Black) = radius 15(?), 100% blurred, light level 5, 75% blurred, light level 10 blurred 50%, LL 15, 25% blurred, LL 20 (and up) 0% blurred. The second layer is the noise layer, applied in a similar way, with no noise from light levels 25 and up. Light levels are from 0-255 btw. The third layer is another gradual blur, making the dark noise like smooth fading round blobs, but not affect anything lighter then light level 25 and up.

The scene you choose is nice, maybe you can have a white flare lit up a wall to a light level of 125? To see if this has noise and blur or not?

Are the links to the latest version the same as the ones listed above?

Effectively Tommekk has done the experimental phase for Dayz, I really hope they pick this up, and build upon it.

ps. I just imported the screen shots, and raised the levels and noticed two players in the shadows. :) The shader does obscure the player a bit = good.

Edited June 20, 2016 by Troll_Hunter

[tommekk 26]

On 20.6.2016 at 7:47 PM, Troll_Hunter said:

It looks smooth, but still a bit digital (maybe due to the short gif repeating ?), like a CMOS chip, but the noise has the right amount of variation and smoothness.

Ok, i'll look into it.

On 20.6.2016 at 7:47 PM, Troll_Hunter said:

Suggestion: make a 3 layered shader, with the first shader blurring the dark area's according to their darkness: light level 0 (Black) = radius 15(?), 100% blurred, light level 5, 75% blurred, light level 10 blurred 50%, LL 15, 25% blurred, LL 20 (and up) 0% blurred. The second layer is the noise layer, applied in a similar way, with no noise from light levels 25 and up. Light levels are from 0-255 btw. The third layer is another gradual blur, making the dark noise like smooth fading round blobs, but not affect anything lighter then light level 25 and up.

:D

The (updated, noisy) shader already works almost like that. The "additive noise" has a lower frequency and is added to the original image right before the blurring. The blur smooths this noise too to make it more "organic". After the blurring the image is multiplied by the "multiplicative noise".

Here we have a plot of the "distributions" of the blur and noise. The x-axis is the light level (0-255), the y-axis the amount of blur/noise (0-1 -> 0%-100%).

The "additive noise" (blue) is only used in very dark areas and uses the same "distribution" like the blur. The "multiplicative noise" (orange) is used in a wider range of light levels to be at least slightly visible.

On 20.6.2016 at 7:47 PM, Troll_Hunter said:

The scene you choose is nice, maybe you can have a white flare lit up a wall to a light level of 125? To see if this has noise and blur or not?

Going to test that in the next tests, but at this light level there shouldn't be much visible noise/blur.

On 20.6.2016 at 7:47 PM, Troll_Hunter said:

Are the links to the latest version the same as the ones listed above?

You mean the source? No. I haven't uploaded it yet as the code needs a bit clean up and optimisation. But I'll post the link if I have it ready!

On 20.6.2016 at 7:47 PM, Troll_Hunter said:

ps. I just imported the screen shots, and raised the levels and noticed two players in the shadows. :) The shader does obscure the player a bit = good.

Yep. I put them there to test the effectiveness. Maybe I'll hide some more the next time...

 

[tommekk 26]

UPDATE (with source)

Since the previous version the noise has changed pretty much. The way the "multiplicative noise" is applied is different and now it has two different noise types:

Fig. 1: Noise
Left: The "digital" noise looking a bit like "static"
Right: The "smooth" interpolated noise looking a bit like slow moving ripples on water

The "additive noise" was replaced by the effects the saturation of the rod cells (not related to the color saturation) is responsible for:
E.g.: You are in a dark environment, your eyes have adapted to the darkness, then you look at some bright object like a flashlight, a fire or the full moon and then back into the darkness again. Then you'll see a dim, white-yellowish "copy" of that object.

Fig. 2: Saturation - moon
After moving the mouse around a bit you can see the very dim "trace" of the moon

And if you use a lightsource and turn it off in a dark environment, then you're blinded for some time until you eyes have adapted to the darkness.

Fig. 3: Saturation - headlights
After turning off the headlights it takes several seconds to adapt to the darkness again.
Left: Headlight on
Middle: Immediately after turning off the light
Right: After several seconds

The distribution of the noise has changed too. Now there is slightly more noise in the dark and less noise in brighter environments.

Fig. 4: Distributions
The effects in relation to the luminance

Fig. 5: Another example
Left: Shader off
Right: Shader on

 

Source:
http://pastebin.com/TV1Tmu95

Edited July 1, 2016 by tommekk
formatting

[Troll_Hunter 54]

DEAR BI, please take note of this excellent work!

I think the JPG screenshots don't do the effect justice.

The saturation of the rod cells effect looks great!

Edited July 8, 2016 by Troll_Hunter
spelling

[idie 57]

You deserve to be hired by BI ;)

Nice dude.

This gamma issue is really worrying.

A while ago, I thought about a system where players should buy "live sensor calibration device" for their monitors, the calibration device will check the gamma of the screen and report to the server like an anticheat system, good idea ha ? ^^ I know, this will maybe happens in one century. ^^

 

[Luftpancake 94]

On 6/12/2016 at 3:06 PM, tommekk said:

This is a suggestion for a post-processing shader to improve the gameplay at and the look of the night in DayZ. It makes the abuse of the gamma and brightness settings almost useless (especially for PvP) without restricting these settings.

---Update 2016-06-12---

• Updated description
• Created thread in suggestions section of the forum

---Update 2015-12-18---

• NEW: Uploaded source to pastebin.com for public testing
• NEW: Added controls to toggle the shader and the gamma correction independently at runtime
• Updated and tested the shader for ReShade with ReShade 1.1.0
• Performance optimization, code clean-up and updated documentation/comments
• Further optimization for low visual impact while playing with normal gamma settings
• New images

---Update 2015-02-23---

• NEW: Shader for ReShade (HLSL)
• NEW: Shader for SweetFX (HLSL)
• NEW: Shader & Sketch for Processing (GLSL)
• Optimized performance
• Optimized for low visual impact while playing with normal gamma settings
• New images

---Submitted 2013-12-18---

0) INDEX:

0) Index
1) Theory
2) Implementation
3) Results
3.1) Images
3.2) Performance
4) Summary
5) Conclusion
6) Source
6.1) Installation and use

1) THEORY:

If you aren't interested in the theory, skip to the "RESULTS" section to see the screenshots.

One important problem in DayZ on night time servers is the abuse of the gamma settings as it is easy to change the in-game gamma-/brightness settings, the systems graphic settings or even the display settings accordingly. And since it is that easy and literally undetectable many players seem to use it for their advantage today. This forces each player to decide to either play with normal gamma settings and having a disadvantage but a nice looking game or to increase gamma to have a greater chance to survive but everything looks awful. Aside from its aesthetic appeal the night doesn't make much sense if the main aspect, the restricted vision, can be circumvented so easily like at the moment.

 
Fig. 1: Gamma
Comparison of normal gamma settings (left) and abusive gamma settings (right)

To make things worse the nights in DayZ (and ArmA 2 & 3) seem to be either to dark or to bright:
On the one hand we have the very dark nights around new moon which are quite realistic in terms of lighting: After all it's the apocalypse; the power grid seems broken down, so large scale electric light sources shouldn't work. But then it is nearly impossible to even navigate, not to mention looting, fighting, etc. and this could lead even players not really interested in PvP to exploit the gamma settings. On the other hand we have the nights around full moon which are so bright that you can see and recognize players several hundred meters away. But this is neither realistic nor authentic.
Opposed to the big difference between full and new moon nights the difference between winter and summer nights seems quite small: While the winter nights are OK, the summer nights seem too dark and you can see much less than in the real world.

Fig. 2: Lighting
On the left an example of the relatively dark new moon, summer nights at 2035-06-12 20:20 in-game time. At least in Central Europe this darkness isn't reached until around 23:30 at this date.
On the right an example of the relatively bright full moon nights.
(The white object in the background is a radome)

But only changing the ambient light wouldn't be a solution as the problem is not just the lighting, but that the maximum "useful" view distance, up to which you can recognize important details, doesn't correlate with the actual brightness of the ambient light. If the brightness exceeds the level at which you are able to recognize your close surroundings, you can also recognize details hundreds meters away. But below that level you almost instantly can't see your hand in front of your face. What is missing is a continuous transition from details being recognizable to details being not recognizable for the transition from day to night and back.

 
Fig. 3: Details
While on the left there are some details visible both nearby and a few hundred meters away too, there are no significant details visible on the right, neither close nor at a distance.

In the real world at some point everything looks more and more blurred the darker it gets and even in a full moon night you are barely able to recognize an idle person >100m away in a field of tall grass or >30m in the woods. But you are able to see and recognize large objects like trees, large rocks or bushes at great distance but way less detailed compared to daytime.

To understand why this happens we have to look at the physiological characteristics of the human eye. The human eye contains mainly two different types of photoreceptor cells; the cones and the rods:

Fig. 4: Retina
Simplified representation of the retina showing the photo receptive rods (gray) and cones (colored) and some of the neurons (brown) responsible for the summation of the values of several rod cells and for connecting all photo receptors to the next layers of the retina.

The cone cells are responsible for the vision in daylight. They provide color vision and are connected to the brain more or less one by one to achieve a high resolution, but they need relatively bright light to work.
At night the rod cells take over. They are about a hundred times more sensitive to light compared to the cones but can not differentiate between colors. Additionally they form clusters of several rod cells to accumulate and amplify the light, leading to even more sensitivity. But because each of these clusters is connected to the brain as a whole, instead of every single cell on its own, they provide a lower resolution. That leads to a blurrier vision at night compared to the vision in daylight.

To simulate this behavior the shader needs to blur each pixel depending on the brightness of the area around it. The darker an area is, the more it gets blurred: An object below a bright street light isn't modified at all, while a dark tree line gets heavily blurred.

In addition to the more natural look of the night the shader significantly reduces the benefits of abusing the gamma settings: If you increase the gamma value the dark areas may be brighter but stay blurred and the removed details can not be recovered. While the shader is optimized for low visual impact when playing with normal gamma settings, you still can't reveal significant additional details by increasing gamma.

 
Fig. 5: Effectiveness
Comparison of abusive gamma settings without (left) and with the shader (right)

This leads to a much wider range of brightness of the ambient light usable for balancing:
You neither need pitch black nights to restrict vision nor you need to reveal details hundreds meters away if you just want to allow player to travel the map at night without additional light sources. Besides that you wouldn't have to restrict the gamma correction itself as simply restricting the gamma correction is unfair to those players who really need it.

While the blurriness is barely visible it could induce gaming sickness for some sensitive players. This is most likely to happen at some point during the in-game (nautical) twilight. At this time the ambient light reaches a level at which most of the surroundings are already a bit blurred but it is not yet dark enough to "cover it up". While this is the same in the real world, it is rather unusual in computer games. This discrepancy between expectation and reality could cause negative symptoms for some players.
But it is also possible that the shader reduces the symptoms for other players if these symptoms are caused by small, noisy, barely visible details in the dark surroundings. As these details are hard to see, the tracking and focusing of them could draw much concentration or be tiring for the eyes and lead to headache or dizziness. Since the shader removes exactly these details it could reduce or even prevent this sort of gaming sickness.
To evaluate these assumptions more players need to test the shader and report back. I know from my own experience that it takes a while to get used to the slightly changed look and feel of the game. When you test the shader for the very first time or the first time after a bigger change of some parameters it can take up to two hours until you have adapted to it. That means you shouldn't give up too early and keep on testing for a while even if you aren't pleased by the results right from the start.
As described above; the shader is configured to cause as little visual blur as possible. This leads to even less blurriness than in the real world and should be kept in mind when comparing the results to real world examples.

2) IMPLEMENTATION:

The following descriptions of two possible approaches are simplified as the detailed ones would go beyond the scope of this overview.

Fig. 6: First approach
rectangle - data
ellipse - processing step

The first approach uses adaptive blurring of the image:

1. The luminance of the neighborhood of each pixel is determined by calculating the luminance of each pixel, save the result in a separate buffer and then blur it with a fixed radius.
2. The result of step 1 is used to calculate the radii for the actual blurring of the frame. For this a function similar to 1.0/luminance is used. That means the brighter the neighborhood of a pixel is, the less blurred it gets.
3. The frame gets blurred according to the result of step 2.

 

Fig. 7: Second approach
rectangle - data
ellipse - processing step

The second approach uses the blending of the original and a blurred version of the image:

1. To get the luminance of the neighborhood of each pixel the image is blurred first and the result is saved in a separate buffer. Then the luminance is calculated and saved to another buffer.
2. Almost the same function like in the first approach is used to calculate the proportion of the original image and its blurred version for each pixel.
3. The original image and its blurred version are blended pixel-wise according to the result of step 2.

If we want to compare the two approaches we just need to examine the third step as the first two steps are almost the same for both approaches.
It is easy to see that the second approach is faster because a simple interpolation is faster than a convolution with a kernel of reasonable size.
When it comes to the efficiency of the gamma exploitation prevention the first approach is superior. This becomes clear when we look at the frequency spectra of the results of both approaches:

We assume that all frequencies of the input are equally distributed so its spectrum looks like this:

Fig. 8: Input spectrum
Homogeneous input spectrum

Then the output of a shader using the first approach would have the following spectra:

Fig. 9: First approach output
The output spectra of the first approach for different light intensities.
From left to right: Day, early twilight, late twilight, midnight.

As you can see the darker the input is the lower is the cut-off frequency. All frequencies significantly higher than the cut-off are suppressed enough to be technically unrecoverable as their amplitude reaches virtually zero. So more and more details are removed beginning with the smallest(highest frequency) ones.

For the second approach we assume the same input and the following spectrum of the blurred version of the input calculated by the first step:

Fig. 10: Fixed radius blur spectrum
The low-pass filtered intermediate result calculated during the first step.

Then a shader using the second approach would have the following output spectra:

Fig. 11: Second approach output
The output spectra of the second approach for different light intensities.
From left to right: Day, early twilight, late twilight, midnight.

In contrast to the first, the second approach lowers the amplitudes of all higher frequencies altogether. That means that unless in complete darkness all higher frequency details have significant non-zero amplitudes and could be recovered by unsharp masking. This renders the second approach ineffective against gamma exploitation as even a simple sharpen effect of the graphics card or the display could recover significant details.
As the effectiveness against gamma exploitation is one important requirement the shader has to  meet, it has to use the first approach.

In addition to that you need to consider the following:

• The shader should be applied prior to the tone mapping to guarantee that it only takes effect in low light environments like at night and not on dark surfaces at day
• If you aim for physiological correctness you would need to configure the 2. step that the effect starts to fade in at the same luminance at which the colors start to fade out as at this time the transition between day- and night vision starts
• To reduce the abuse of the gamma settings it has to be applied prior to the gamma correction.
• If you plan to allow post-processing injectors like ReShade and shader suits like SweetFX (I hope you will!) then you need to decide whether you want to make the depth map accessible or not. Because if you do, you need to blur it too; otherwise it could be exploited as well to get something like "bat-/sonar-vision".

If you follow these recommendations then you should have a fair solution, because everything has exactly the same level of detail for everybody regardless of their gamma- and brightness settings.

The shader could be controlled server side, as it might lower performance on very old systems or could induce gaming sickness, but would really improve the fairness. Then it would be up to the player to choose between servers with and servers without the shader, but if they choose one with it enabled, they can be certain everybody uses it.

3) RESULTS:

3.1) Images:
The screenshots were taken with ArmA 2 & 3, because it is easier to get a greater variety of scenes while ArmA and DayZ are comparable in terms of brightness/gamma. The previous screenshots (2015-02-23) were taken with ArmA 2 CO in Chernarus, the new ones (2015-12-18) with ArmA 3 in Chernarus and Altis.

Fig. 12: Example 1
NWAF

 

  
Fig. 13: Example 2
Hills north of Chernogorsk

 

  
Fig 14: Example 3
Country house on Altis

More examples in the same order like above for every row:

New screenshots (2015-12-18):
http://imgur.com/a/7G0b7/all (Chernarus)
http://imgur.com/a/kuhsl/all (Altis)

Previous screenshots (2015-02-23):
(These are outdated, but I left them for comparison)
https://imgur.com/a/1wWjW/ (Chernarus)
https://imgur.com/a/S7Db4/ (Chernarus)

3.2) Performance:

Tested on:

• AMD Phenom II X4 965 Black Edition @ 3,4GHz
• NVIDIA GeForce GTX 560 Ti
• ASUS M4A89GTD-PRO/USB3
• 12GB RAM @ 1333MHz

With ReShade 1.1.0 and the recent version of the shader.

Test of ArmA 2 CO:
The Shader takes 0.1-0.2ms to process one frame.
No reduction of the frame rate observed.

Test of ArmA 3:
The Shader takes 0.01-0.02ms to process one frame.
No reduction of the frame rate observed.

The difference of one magnitude in the processing time between ArmA 2 and ArmA 3 might be a result of the different DirectX versions used by both engines.
The shader should be at least as fast as in the test with ArmA 3 if it is included directly in the post processing of the new renderer (DirectX 11).

4) SUMMARY:

Pro:

• Combats the abuse of the gamma correction without restricting the gamma correction itself
• Improves realism and authenticity
• Gives the night a more natural look
• Good performance

Con:

• Might reduce the performance on very old or minimalist systems without dedicated graphics
• Could cause some problems for players who are prone to gaming sickness

5) CONCLUSION:

The shader improves the authenticity and gameplay without changing the rest of the game or forcing the players to certain gamma and brightness settings.

6) SOURCE:

The source can be found at:
http://pastebin.com/Cm4y12Qq

The attached source files are outdated and for the record only.

6.1) Installation and use:

IMPORTANT NOTE: The shader hooks into the renderer and thus it might modify the memory of the running game. This could be classified as a sign of cheating/hacking and because of this it is recommended to test the shader in singleplayer/editor with disconnected networking. After testing and before reconnecting you should uninstall it by deleting the two files you copied during the installation. That might sound a bit paranoid, but better safe than sorry... :)

During the installation you need to rename two files and change their file extensions too. For this you need to activate/show the file extensions in the windows explorer, if you haven't done it already:
http://windows.microsoft.com/en-us/windows/show-hide-file-name-extensions

1. Download ReShade
   During the testing Reshade is needed for hooking into the renderer and loading/starting/toggeling the shader. It is available at http://reshade.me/
2. Copy ReShade32.dll from the downloaded archive into ArmAs main directory (same directory like the arma2.exe/ArmA2OA.exe/arma3.exe)
3. For ArmA 2 (OA/CO) rename the copied ReShade32.dll to d3d9.dll, for ArmA 3 rename it to dxgi.dll
4. Download the shader file
   http://pastebin.com/Cm4y12Qq
5. Rename it to ReShade.fx
6. Copy this ReShade.fx into ArmAs main directory too.
7. That's it. If you now start ArmA ReShade should load the shader.

Press [Insert] to toggle the shader on/off
Press [Home] to toggle gamma 1.0/2.0

ReShade is not compatible with software like screen capturing or overlays and you need to deactivate them in order to test the shader. To take screenshots ReShade has its own screenshot functionality (but sometimes it seems to miss some keystrokes):
Press [Print] to take a screenshot (they are saved to the same directory where you put the shader files)

I think I love you.

[exwoll 255]

I think Rust stole your idea XD

However, I still see that it gives some extra detail. Enough for a silhouette to be spotted from distance :(

Edited September 7, 2016 by exwoll

[MaxTheSurvivor 152]

My rig is quite outdated with 2x GeForce GTX 460 SLI and in worst conditions I'm getting 25 -26 fps. I have no budget to upgrade my rig so if this tweak has the slightest impact on performance and eats 2-3 fps then I'm against this. We don't all have the money to upgrade our hardware to the latest GPU and with the new engine I think many people with low quality rig like me are happy the way the game is right now. Good ideas might not always be good for everyone and implementing a tweak that has a performance impact could reduce the player base as these people would not be able to play in good conditions anymore.

Devs could give it a try on low-end/outdated configurations to measure the performance impact.

[Troll_Hunter 54]

Currently Dayz isn't optimised at all. So I'm sure they can find 2-3 fps, and trade those for this simple shader that levels the playing field in the night.

On the other hand I hope they do a lot of optimising and improve the looks and sounds of the game to feel current. Levels of detail as shown in the newest flashy games like Kingdome Come Deliverence. I'm saving up for an upgrade for when the game goes into beta. Basically I need a new video card to go along with the 4k screen I want. I hope to buy the stuff early 2017, when the first discounts come along :) With the transition to DX12 I think it's wise to save for a compatible card.

[exwoll 255]

12 hours ago, MaxTheSurvivor said:

My rig is quite outdated with 2x GeForce GTX 460 SLI and in worst conditions I'm getting 25 -26 fps. I have no budget to upgrade my rig so if this tweak has the slightest impact on performance and eats 2-3 fps then I'm against this. We don't all have the money to upgrade our hardware to the latest GPU and with the new engine I think many people with low quality rig like me are happy the way the game is right now. Good ideas might not always be good for everyone and implementing a tweak that has a performance impact could reduce the player base as these people would not be able to play in good conditions anymore.

Devs could give it a try on low-end/outdated configurations to measure the performance impact.

Oh common, you can find decent gpu's on ebay for nothing.