HELP: PfSense, ESX, NAT, Virtual IP's - if this sounds familiar i need advice!

[gerryn 2]

Hi.

Copy/Paste from a thread that got buried, I'm hoping this will gain some attention :)

These are the EpicNL1-6 servers, so 6 in total, running on Windows 2008 R2.

I am running the servers on ESX, they are all individual machines with a local (to the ESX) network connection.

I am using PfSense firewall on the DMZ with two interfaces, one for WAN and one for internal, where all the DayZ servers sit.

On the firewall I have configured Virtual IPs for all the public IP addresses that the servers use and have NATed range 2302-2310 (too many I know) UDP to their respective internal IP's.

This is a problem immediately, with the above setup no servers show up in the server list. Also, all my outbound traffic will seem to come from a single IP (the main WAN IP), and that IP is probably what GameSpy uses since I CANNOT TELL GAMESPY WHAT IP THE SERVER HAS (STUPID!).

There is a switch to the arma2oa dedicated server that lets me set the IP it will -listen to-, now - all the game servers have internal IP's and nothing else - if I set an IP that doesn't exist on the machine the server won't start, naturally - it can't bind.

My workaround which is borderline retarded is to create a virtual IP on each of the internal NICs for the DayZ servers that is identical to the public IP NATed into it from the firewall, i don't set -ip= in the config file, just go with it as it is because I read on a forum that arma will send all IP's on the machine to gamespy (some guy had ran a sniffer to check) - this seems to work to tell gamespy what IP the server has anyways, but its really flaky. Some people can connect because the servers are half-full all the time, sometimes they are full, I can also connect sometimes, and sometimes not.

I am hoping that someone with a similar setup can help me out with some advice as to how to set it up - please don't tell me to put the machines in the DMZ because that's not happening :)

Cheers,

Gerry

[milancz 10]

EDIT: deleted. didnt read your post properly :)

[gerryn 2]

Yes for sure, you are able to connect if you do REMOTE, type in epicnl<1-6>.gerryn.eu AND THE SERVER COMES UP IN THE LIST ... ah its weird...

/G

[demine0 43]

Can you describe your Setup a little more?

Is pfsense running on a separate box, or are you running pfsense as an esxi guest and creating another private network inside esxi itself, and using pfsense to do the NATing? It sounds like your not setting up your VIPS correctly. What type of VIP did you make(CARP, other?). And I assume you have a dedicated public IP for each server? Did you setup your 1:1 NATing correctly? Read these two articles:

http://doc.pfsense.org/index.php/What_are_Virtual_IP_Addresses%3F

http://doc.pfsense.org/index.php/1:1_NAT

That should point you in the right direction. If your also virtualizing pfsense, than you need to create another virtual network inside ESXi as well, to act as your internal LAN for all of the servers.

[gerryn 2]

Everything is running on one ESXi 5, it has two physical interfaces, one is connected to the network and one is disconnected.

I have setup a DMZ network on the physical NIC in ESX and assigned that to a guest running PfSense, configured WAN as the first ip in my range, and the rest of my public IP's are configured as Virtual IP's in PfSense.

The disconnected physical adapter on ESX is configured for the internal network, I have all DayZ guests (2008R2) connected here with static IP's on the subnet of the PfSense LAN GW.

Rules are setup as described earlier, NATing each Virtual IP to its respective internal guest IP, and allowing 2302-2310 as a UDP range through that.

And then the hack - add virtual IP's on the DayZ guests so that they are being fed to gamespy.

All outbound traffic is most likely coming from the main WAN IP - which doesn't seem to be causing a problem in-game - but with the server browser and such...

I think it all stems to the fact that you can't tell gamespy what IP your server will have, I'm hoping for a solution to that - or a workaround :)

Thanks for showing interest!

/G

[Mojo (DayZ) 66]

As Syk pointed out, I'm wondering mostly about 1:1 NAT. Setups like this tend to require 1:1 a lot of things (ie: 1:1 virt/phys NICs), and a lot of things that tend to go wrong if you forget one of those things.

Basically, check all your pairs first and foremost. Double check, triple check.

[demine0 43]

Your running pfsense 2.0 right?

Check your 1:1 nat settings. It sounds like an issue with those settings.

http://www.hpchost.com/knowledgebase.php?action=displayarticle&id=180#

Can you screen shot your 1:1 nat, and firewall pfsense config pages,(Any any other page thats in that url i linked above) and just remove any private info? I'm at work ATM, but I could provide more help on teamspeak in about 4 hours when im home from work.

Essentially, he has a Dedicated server with a bunch of public IP's. He installed ESXi on that box, and made a bunch of guests. He wants to use pfsense as an actual firewall on one of those guests for his virtual machine network. This way he binds all of his public IP's to the pfsense guest, and that controls where all the traffic goes on a more granular level. Essentially a small server farm in a single esxi box. :P

[gerryn 2]

Actually this is already in production running :)

[Mojo (DayZ) 66]

Essentially' date=' he has a Dedicated server with a bunch of public IP's. He installed ESXi on that box, and made a bunch of guests. He wants to use pfsense as an actual firewall on one of those guests for his virtual machine network. This way he binds all of his public IP's to the pfsense guest, and that controls where all the traffic goes on a more granular level. Essentially a small server farm in a single esxi box. :P

[/quote']

Oh yeah I know, I had nearly the same problem with another hypervisor 5 years ago. It was the 1:1 pairs that were cooked. Bloody nightmare.

[gerryn 2]

Yeah its most likely an issue with 1:1 NAT which I don't have :) I'm going to look over the links you posted Sykotix, but I work with this kind of stuff as well and this week has been so incredibly exhausting, I'll get to it tomorrow though.

Thanks!

[tansien 15]

Just move them to WAN and firewall everything but the ArmA2 ports. In fact, why are you even trying to run this shit on VMware? It's bound to create performance issues, and it's not like the ArmA2 dedicated server does not have enough of those...

[gerryn 2]

No, it doesn't create performance issues. Who still thinks that?!

/G

[knightcat 0]

On a machine that is capable, VMWare will run like a well oiled machine. Now, you throw VMWare on a second rate system and you'll get a virtual machine of whatever allocation of the second rate system.

VMWare doesn't create instability, stupid system administrators do.

[tansien 15]

We've done extensive testing at work for a very long time regarding high-performance applications in VMware, and while it's gotten alot better over the years there's still quite a bit of performance loss.

I'm not saying VMware will create instability, i'm saying you're trowing away anything from 10-30% in performance...

[gerryn 2]

I have 6 servers (+ PfSense and a domain controller) running with 1vCPU 2vCore each with 1.5GB of memory on a Hexacore Xeon, this runs without issue, are you saying that I'm better off running 1 server instead of 8? Doesn't make any sense, Arma 2 OA dedicated server is not a "high-performance application", all of the DayZ VMs run 50% cpu, no more, the total CPU usage on the ESX host is around 98%, no complaints from users about performance - the opposite.

I also work with this stuff, but we're not hosting Oracle here and I'll tell you this, We've done extensive testing with low-performance crappy applications, just like the Arma 2 OA dedicated server (its crashing all the time - no critical apps crash like this, i.e. crappy application, not bashing the devs or anything - love the game so much I'm investing hundreds of euros of my own money just to host servers for other people) - and we've found that crappy applications run as good on VMWare as they do on physical hosts - obviously with the reduced management overhead which made it possible for me to build all this in a few days while working my day job and playing the game - imagine with physical machines even if you had iLO.

(EDIT)

Here is a screenshot of the current CPU usage on the host, but I think it's low-time now: http://dl.dropbox.com/u/815682/dayz/performance_statistics.png

Anyways, the issue is for SURE related to 1:1 NAT which I don't have, I have configured the IP's are pure aliases (I'm a server guy, not a networking guy - there's another team for that) and just NATed them into the private IP's, I am positive that the outgoing traffic only comes from the WAN interface - a single IP, and I think that gamespy picks up on this IP together with all the other IP's like the private, the virtual i set in Windows on the guests, and the WAN IP - and this is what is causing it to work sometimes, and sometimes not.

Had such a fucking awesome time playing today, didn't feel like changing it - servers are full anyways even when the intermittent connection issues persist.

Will change it tomorrow (today) when I wake up.

Cheers,

G

[demine0 43]

Yup, well there's your problem :P Once you have 1:1 let me know how it all works out :)

[Mojo (DayZ) 66]

We've done extensive testing at work for a very long time regarding high-performance applications in VMware' date=' and while it's gotten alot better over the years there's still quite a bit of performance loss.

[/quote']

Yeah though, years, there's your problem. VMware missed their bare metal release in what, just as recent as 2010? Type 1's been the norm with ESX and ESXi though, and in terms of virtualization, yeah, it's an overhead thing, but 10-30%? No way. I would have to ask to see the actual benchmarks on that, if only out of morbid curiosity.