Minecraft server purge and archival

Posted by aeron on Sept. 22, 2013, 4:43 p.m. 🔒

We recently had an incident with the 64digits server running completely out of hard drive space. While the majority of our space is used to host our site data (games, user uploaded files), a decent chunk was being taken up by minecraft and steam, but mostly minecraft due to world backups.

Over the years of maintaining the minecraft server, we never had a consistent backup plan and most of our worlds were just copied to a folder and forgotten. For a time we were cautious in deleting anything from that folder in the event that one of the worlds would be needed or wanted again someday, but when lack of disk space took the main site down, we had to compromise and get rid of some of them, as well as the steam (TF2) server.

Now, several weeks later, we are trying to free up more space to bring back the TF2 server (and also have some breathing room for the main site), and to help accomplish this Jeremy has moved all the old unused backups offsite and created archives of the important worlds. These will be available for download for a short time, but will eventually be removed from the server entirely. In addition, we are trying to come up with a plan to retire the creative server, which is huge but inactive. If there is no objection, it will be archived and made available for download, and a fresh map will be put in it's place (with boundaries enforced to keep the size down).

So for now, if you want to download any of our old maps, you may do so here:

http://www.64digits.com/~minecraft/

These will be available for at least a month from today. If there are no objections to retiring the creative server, it will be done a week from today and also be available for a month after retirement. After that time there will be no guarantee that the maps will remain available, but it's possible that our users could set up mirrors of their own to keep the archives alive offsite. We hope these terms our acceptable to our minecraft users, as we simply can't maintain all these worlds on the site forever without upgrading our disk space.

Comments

Quietus 11 years, 3 months ago

<_<

>_>

*absorbs*

JuurianChi 11 years, 3 months ago

Quote: How I read the blog

Something.

Something.

Something.

Something.

Something.

bring back the TF2 server

Something.

Something.

Something.

KaBob799 11 years, 3 months ago

Actually you should wait until 1.7 to start a new creative world or we're going to have a ton of chunk borders. But there might be some ways to reduce its size, it might still have the old map format files mixed in with the new format which would really increase its size.

twisterghost 11 years, 3 months ago

Have you considered offloading the minecraft server to another host altogether? Perhaps community fund it or something?

Linodes are cheap enough with good specs, or some other cloud host or something.

aeron 11 years, 3 months ago

Quote:
Uncompressed, how much space do the worlds take up?
A single copy of the creative world takes up around 3 gigs. This is probably our biggest world though, most average < 1gb. The problem only really starts to show when there are dozens upon dozens of backups carelessly scattered about.

Quote:
Actually you should wait until 1.7 to start a new creative world or we're going to have a ton of chunk borders. But there might be some ways to reduce its size, it might still have the old map format files mixed in with the new format which would really increase its size.
Hmm, this may or may not be an issue depending on how we go about it (Though it's ultimately up to Jeremy when we make the switch). One thing we discussed previously though was to add constraints to the creative world so that we don't get too sprawled out like the current map. We could also pre-generate the chunks in the beginning to ensure there aren't any borders (but if we expanded the map later there could yet be issues).

Quote:
Have you considered offloading the minecraft server to another host altogether? Perhaps community fund it or something?

Linodes are cheap enough with good specs, or some other cloud host or something.
This is an option, but really our current setup works fine as long as it is maintained properly. We have the space to run a few maps on the server for each game type (one for survival, one for creative, and one for tekkit/FTB marathons), and also for archives of retired maps. Really old maps can eventually be taken down or offloaded to a different host. The root of our problem was backups that everyone was afraid to sort through, but it's under control now. We went from 50gb of minecraft files to 13gb (with the possibility to cut more if needed).

Josea 11 years, 3 months ago

I like the idea of retiring the creative map and making it available for download, and I agree with Kabob, wait until 1.7, the changes to the map generator are huge.

Grand-High Gamer 11 years, 3 months ago

If there no simple way to trim the creative server down? I mean the place has some really nice stuff, but you guys spend so long building railways that are 70,000 blocks long and take 45 minutes to cross for no reason that of course it is bloated up.

Castypher 11 years, 3 months ago

Thank you, GHG, the railways were the things I found most pointless in a creative world where everyone can teleport, and when only 20% of what the railway passes is actually worth seeing.

Anyway, I hope we'll still be able to play a bit of Minecraft, even if we have to start new maps every time. Thanks for putting up these maps, Aeron. I'll be sure to shed a few tears of nostalgia when I realize nobody will ever play these maps with me again.

aeron 11 years, 3 months ago

@Kilin, thank Jeremy for putting them up, he did all real the work cleaning up the ~minecraft account (And also making the archives). I'm just making sure everyone knows about the downloads.

And Grand-High, a fresh start is really more feasible, because of the sheer size of the sprawl. The dynmap was a nightmare, it looked like someone took a pen tool in paint and scribbled chunks till it looked like a plate of spaghetti that got flung all over the table, and carpet, and walls…

Maybe we'll work out a permanent download (a torrent might make sense :o) to keep it available, but there's a good chance we'll need those 11gb for something else in the near future (keep in mind the server only has 450gb space). That is, unless we work out a plan to prune our mass of user and game files (which take up 300+gb) on the site and give us the breathing room we need (for more user and game files, of course :P).

Alternatively, we can start the "Buy a TB for 64Digits fundraiser", and we'll never worry about deleting anything again ;)

KaBob799 11 years, 3 months ago

The only unnecessary railway I know of was the one connecting old spawn with beakton or whatever it was called that was like 30k blocks away, but even then the chunks were already loaded so it didn't make the map larger. All the rest are in chunks that would have been loaded regardless because people have built nearby. Really we could probably just toss all the far west chunks (well separate them as a backup, then delete later) cause I don't think anybody cares about that place anyway.