Temporary Change to the Hosting Rules regarding Persistence Backups

[ImpulZ 2491]

Hello Survivors,

as we are aware that the issue with server crashes causing persistence wipes is still occurring, we will temporarily lift the Server Hosting Rule regarding restore/reset of persistence files. You can read the complete rules HERE.

OLD RULE:

Restore/Reset Persistence files

• Once a day
• Request has to be processed manually by game server provider (edited)

NEW RULE (effect immediately):

Restore/Reset Persistence files

• Unlimited
• Can be done manually by customer via GSP (game server provider) control panel

This means that if you are working with a server provider, you can ask them now to provide options to regularly save and restore your persistence files to prevent major persistence losses during crashes.

The team is working on a permanent solution for the issue, but as this will take some time, we will temporarily enable this option, to hopefully ensure a better persistent gameplay experience.

See you in Chernarus!

[PsyStormZ 20]

Fantastic!

[spaggie 39]

Hi

Our GSP helped us start doing more than one backup a day a few days ago on our main servers

But this seems to have caused some issues, we couldn't schedule the backup to be done aligned to a restart (backup while server is down) due to the way the scheduler works so we scripted 4 backups a day into their new format dated backup directory

We think a manifestation of doing it in this way is that players'db gets locked out while performing file operations (copies and FTPs) so we have players who lose their character or roll back and this sql lite disconnect stays manifest until the server is restarted. We have not experienced a players.db disconnect since disabling backups (we are currently running no backups).

The issue isn't just with making backups, that's a general triviality to create a backup (above issues we experienced aside) the issue is with being 'brave' enough to restore them

As a side note I was once woken by someone shouting that 'ALL THE BASES ON THE SERVER HAVE BEEN WIPED', I asked him how he could possible know and it quickly transpired that HIS base had been wiped. This made me think very carefully and very differently about backups

If Team A has a base and it gets wiped on a restart

But between the restart and Team A noticing their base is wiped Team B builds a base

Then there's two options when the alarm is raised

1). Restore the backup and mitigate Team A at the expense of Team B (generally inconveniencing 2 teams) and taking A HUMAN ACTION TO REMOVE SOMEONES BASE

2). Do nothing, and generate a fairness in the game through inaction 'you were just unlucky' (which also means Team B innocently continue and perceive no issue until THEY have some bad luck at a later time)

We are generally choosing the second option, bases are not going missing due to admin actions (let alone the risks of fail rates on restores)

At this time we do not recommend people build bases excepting to see how it works, and that they have no long term plan for any base built. This is especially painful for the good folk of our The Village server and I know the Admin team and BarelyInfected feel they carry personal responsibilities for the sense of loss players feel.

Thank you for the suggestion of this mitigation, but I do not think it ultimately viable. 

We look forward to a more robust fix (timed mem flushes to disk etc) and we look forward to the gameplay that not only persistent bases, but also balanced bases (more than 2 minutes with a hammer to destroy) will bring to the game

Love You

Spag

 

 

 

[THEGordonFreeman 664]

50 minutes ago, spaggie said:

Hi

Our GSP helped us start doing more than one backup a day a few days ago on our main servers

.........

We look forward to a more robust fix (timed mem flushes to disk etc) and we look forward to the gameplay that not only persistent bases, but also balanced bases (more than 2 minutes with a hammer to destroy) will bring to the game

Love You

Spag

 

 

 

1000% agreement with this.  I am tired of fighting this battle.  I tried and tried to explain that not having persistence fixed prior to release was a major, MAJOR mistake.  And I am no longer interested in band-aids that will help one person or a group while others get reset back.  That is not fair, nor a viable "solution".  When the player base dumps again because the end game is broken, will BI continue to insist that server owners and game hosting providers do something else to "patch" what is obviously a very broken system of saving the state of the game that they are ultimately responsible for?  I am a little salty because of the constant comments I have seen come from Eugen and others from BI that have laid the problem at other people's feet instead of really, REALLY trying to resolve this with the community.  I'm sure fixing server crashes is not easy, but to let this simmer over the holidays when people don't have school and time off from work only to be disappointed because any progress they make is SUNK because the persistence system is broken is unacceptable.  Yes, the devs have made great strides in many areas, but they are leaving the end game in a perpetual broken state because of this release.  If I had done this when I worked for a software company, I would have been fired.  Leaving a broken mess for users because of an "arbitrary" deadline and then go on vacation, is inexcusable and quite frankly a huge middle finger to the community that has supported you through the best and mostly worst of times.

You guys deserve a holiday, but you have neglected your community that has supported you through it all, the disappointment and resentment is thick.  I have seen long time supporters become adversarial over this release, and I understand why.  What will BI do to fix this problem ASAP?  Talking about it is great, but now action is needed more than words.  Telling people to send in crash data AFTER you made the decision to arbitrarily release this game in a KNOWN broken condition doesn't work anymore.  "It's Alpha" is no longer an excuse.  What will BI bring to the table immediately to address this game breaking issue?

Edited December 21, 2018 by THEGordonFreeman

[Rob MacDonald 0]

12 minutes ago, THEGordonFreeman said:

  Telling people to send in crash data AFTER you made the decision to arbitrarily release this game in a KNOWN broken condition doesn't work anymore.  "It's Alpha" is no longer an excuse.  What will BI bring to the table immediately to address this game breaking issue?

I don't even host a server and I'm 100+10000000% in agreement here.  I would like to see some actual reasonable responses and actions on this major issue.

If I stood up at system for work that was as unreliable in core features as this build I would expect to be fired and blacklisted.  This is not the server providers fault, it's not the server owner and it's not the players.  This is a core system, if not THE core system, of the game and it doesn't work.  The entire loop is now broken.  That influx of 30,000 unique players during the free-weekend are likely operating under false pretenses.  The game runs great on some servers, and like pants on others.  That's fine.  Persistence being broken would take time for new players to actually notice, and in fact they might never notice depending on their style of play.  But the big things right now are cars (still kinda iffy) and base-building (in no way worth the effort EVEN IF persistence worked.  3 hours to build, 15 minutes to dismantle).

This is supposed to be a feature complete and stable build.  We've mostly gotten over the feature complete part.  But this is not stable.  It's fun in spurts, but that's not acceptable for a full price retail released game.

 

[Guest]

24 minutes ago, Rob MacDonald said:

3 hours to build, 15 minutes to dismantle).

 

5 Hours to build, 1 minute to dismantle. 

Edited December 21, 2018 by Guest

[MaxTheSurvivor 152]

23 hours ago, WhiteWalker1 said:

 

5 Hours to build, 1 minute to dismantle. 

Dude I built a huge base, I mean massive, about 100 hundred hours of work, half-way through (10 watchtowers, 15 tents, fences between the towers) and the persistance crash wiped everything. 1.0 was supposed to be a stable release with the core mechanics being reliable, persistance is not, cars are not, what remains of that 1.0? Not much. Well, I supported the game until the 1.0 release, now I'll start to be a little less kind with the devs. They wanted a rushed 1.0? They can't hide anymore behind the "it's alpha", there's no more "I understand" buton.

[THEGordonFreeman 664]

On 12/22/2018 at 11:46 AM, MaxTheSurvivor said:

Dude I built a huge base, I mean massive, about 100 hundred hours of work, half-way through (10 watchtowers, 15 tents, fences between the towers) and the persistance crash wiped everything. 1.0 was supposed to be a stable release with the core mechanics being reliable, persistence is not, cars are not, what remains of that 1.0? Not much. Well, I supported the game until the 1.0 release, now I'll start to be a little less kind with the devs. They wanted a rushed 1.0? They can't hide anymore behind the "it's alpha", there's no more "I understand" buton.

^^^^THIS^^^^ is the net result of what has been a debacle from the start.  This is why I warned that the release should have been postponed.  How much money would have it cost to postpone the release vs the negative blowback from the free weekend and beyond.  How much negative PR and the continuous damage being done to DayZ's core loop and good will from the core DayZ community will be lost over the next two to three weeks because of this arbitrary release date? 

I am now including a message on my server that building a base is at your own peril.  My message every hour... "NOTE: This game is not in early access, it has been released.  If you build a base and it disappears it is BI's fault, let them know, they are responsible for your bad experience.  I accept no responsibility for their bad decisions.  #persistenceisBAF "

And on that note, we are trying to recreate another issue with tents and dead players near them and the CLE cleanup of bodies that may be causing tents to get deleted simultaneously.  If we manage to replicate, we will send to Feedback tracker, but this will likely be our last time spending anymore time on bugs in this game when it is now released to the public.  These things should have been found prior to release.  I am convinced their testing methodologies are not working and they need to be rectified pronto.

[eno 1049]

On 12/21/2018 at 8:23 AM, ImpulZ said:

NEW RULE (effect immediately):

Restore/Reset Persistence files

• Unlimited
• Can be done manually by customer via GSP (game server provider) control panel

This means that if you are working with a server provider, you can ask them now to provide options to regularly save and restore your persistence files to prevent major persistence losses during crashes.

Okay. So what I am reading here- and what I've read from another survivor is that the GSP can be asked to increase the rate at which backups are created. ie: by the original rule it was once per 24 hour period but it could now be, as mentioned by spag, 4 times per day. 

So I reached out to our GSP (Gamerservers.com) and asked them about setting up something new. Pretty much something more frequent... to ask if we could even do it. They said they didn't know what I was talking about so I pushed a little further with a link to this thread. This was the eventual reply: 

Hello,

After looking at the new rules, we have implemented the ability to restore and wipe the persistence from the client area, however they do not state anything about the ability to create backups. Was there anything else that I could assist with today? 

Thank you,

 

Am I missing something? 

And if I'm not missing something and it seems like Gameservers just isn't interested... then for those of you hovering on renting a server to explore base building with some extra security for awhile I recommend you look for another. 

Edited December 27, 2018 by eno

[drgullen 596]

12 minutes ago, eno said:

After looking at the new rules, we have implemented the ability to restore and wipe the persistence from the client area, however they do not state anything about the ability to create backups.

How the fuck are you supposed to do a restore without a backup? LOL

[eno 1049]

Just now, drgullen said:

How the fuck are you supposed to do a restore without a backup? LOL

From what I gather they're saying "we can restore from our 1 backup a day whenever you'd like but you can't back up your files with more frequency." 

What I'm confused about is that it seems obvious to me that what the new rule indicates is up to "unlimited" backups with "unlimited" restores... I doubt unlimited is practical- but they didn't want to set a specific number so I get it. Let's say 4 times per day... 

But in reviewing the rule they have interpreted it differently... so now I'm left wondering if it's me who misinterpreted. 

[drgullen 596]

Well, the new rule says "you can ask them now to provide options to regularly save and restore your persistence files".

I personally wouldn't call "regularly" unlimited, but I would think that's more than 4 a day -- it seems the devs are leaving it up to the individual GSP to decide how many you get per day.  The rule does say "saves" though, which I would interpret to mean backups.

Backups need to occur hourly I would say and a restore needs to happen immediately after a server crash, otherwise you risk losing many hours of persistence.  For example, let's say you have a backup scheduled for 9:00am and the GSP schedules the next one for 5:00pm -- the 9:00am backup occurs, the server is relatively stable all day, but then crashes at 4:20pm (LOL) -- since no backup has happened since 9 that means everything built/placed/buried between 9:00am and 4:20pm is now gone.  The backup at 5:00pm would only preserve the world for the past 40 minutes, since the restart following the 4:20pm crash would have reloaded the world.

[eno 1049]

19 minutes ago, drgullen said:

Well, the new rule says "you can ask them now to provide options to regularly save and restore your persistence files".

I personally wouldn't call "regularly" unlimited, but I would think that's more than 4 a day

Yeah I agree with you- ultimately I plucked my definition from the original post and one from Spag. 4 times a day... hourly... whatever it looks like I suppose is whatever a person works out with their GSP. 

Suffice to say it seems like gameservers just isn't interested in being very helpful.

 

Anyone else have experience with other GSPs that have been more cooperative? @spaggie who are you guys dealing with? 

 

Edited December 27, 2018 by eno

[ut marsian 2]

After 5 years of development, there is still a broken persistance system? Is this April 01? Oh my god ... hobby devs....

[mrwolv 46]

comon guys get this peristance fixed im having contant persistance wipes every day this is getting to much for a "full game" this is more like pre alpha!!!

[eno 1049]

So our nightmare with gameservers continues- we had a wipe a couple days ago and wanted to restore. Had to file a ticket with gameservers to ask for them to restore a point for us. 10 hours later we get a reply that they could only do the backup from the last period otherwise the ticket needed to be elevated. 

Of course the 10 hour delay made it exceed the last backup so we needed to elevate the ticket. Fuck that nonsense. 

So we figured we’d dump gameservers early and try out host havoc. Got all set up with them for half the price and things seemed to be going well... except it turns out they only deal with private servers and from their own tech support doesn’t sound like they knew community public and official publics were different hives... and they only have private... so we had to dump that for an instant refund. 

All we want to do is run a community public- that will give us enough control to be able to restore the game on account of this ridiculous persistence issue. What a fucking nightmare. 

 

Anyway- gameservers and host havoc are out... can anyone make a solid recommendation for another one of the server providers that can provide reasonable support and some versatility?thanks. 

[mrwolv 46]

3 hours ago, eno said:

So our nightmare with gameservers continues- we had a wipe a couple days ago and wanted to restore. Had to file a ticket with gameservers to ask for them to restore a point for us. 10 hours later we get a reply that they could only do the backup from the last period otherwise the ticket needed to be elevated. 

Of course the 10 hour delay made it exceed the last backup so we needed to elevate the ticket. Fuck that nonsense. 

So we figured we’d dump gameservers early and try out host havoc. Got all set up with them for half the price and things seemed to be going well... except it turns out they only deal with private servers and from their own tech support doesn’t sound like they knew community public and official publics were different hives... and they only have private... so we had to dump that for an instant refund. 

All we want to do is run a community public- that will give us enough control to be able to restore the game on account of this ridiculous persistence issue. What a fucking nightmare. 

 

Anyway- gameservers and host havoc are out... can anyone make a solid recommendation for another one of the server providers that can provide reasonable support and some versatility?thanks. 

well game servers are the worst going used to use them back in 2004 and they were laggy as hell and so unreliable i ditched them i went to gaming deluxe and never had a problem since