What happened (and what has been fixed)

For technical issues, problems, bugs, suggestions on improving these forums, discussion of the rules, etc.
Post Reply
Jedi Master Spock
Site Admin
Posts: 2161
Joined: Mon Aug 14, 2006 8:26 pm
Contact:

What happened (and what has been fixed)

Post by Jedi Master Spock » Mon Jan 15, 2018 10:10 pm

As some of you may have noticed, the forums were down for about a week and a half. What happened?

My host decided to migrate the website to a new server, suddenly and without warning me. When they did so, they trashed the old server before making sure everything migrated correctly. I was on the phone with them an hour after they finished the migration, and still couldn't get them to fix the situation.

Their "emergency backup" contained a truncated version of the forum database. Fortunately, it included the posts table. Unfortunately, while I had a fairly recent backup of the posts table, I did not have very recent backups of the user and topics tables.

I have reconstructed the topics table (I believe), updated to a newer version of phpBB (this would have been much less painful if I had gotten around to my planned updates earlier), and restored what I could of the user tables. Some user account information has been lost permanently.

Some of you may remember similar issues arising in the summer of 2015, the last time my host decided to migrate my site to a new server (also without telling me, also trashing the old server before they made sure everything migrated correctly. Once is unfortunate, twice in a row smacks of gross incompetence, and it's probably time to change hosts.

The wiki seems to be trashed as well. I haven't even started trying to fix that.

In the long run, I have clearly lost the passion for the subject that kept me invested in spending time keeping this forum running smoothly; it may be time to pass the baton on to someone else with regard to running things around here.

Jedi Master Spock
Site Admin
Posts: 2161
Joined: Mon Aug 14, 2006 8:26 pm
Contact:

Re: What happened (and what will happen)

Post by Jedi Master Spock » Mon Jan 15, 2018 10:38 pm

Okay, to update this... I know that a lot of us spent a lot of time working on wiki articles, even if it hasn't seen much activity in the last couple of years.

The good news: The wiki database seems to have escaped being similarly truncated (being smaller), not that much has changed since the last backup. I now have a good idea of how to fix it, as well, although it will likely be a 5-10 hour project. Content has not been destroyed.

Iscander
Padawan
Posts: 45
Joined: Sun Nov 06, 2016 2:45 am

Re: What happened (and what will happen)

Post by Iscander » Tue Jan 16, 2018 12:08 am

Glad this place is back up JMS. Thank you for the alert and the info dump.

Anything e cam do to help moving forward?

Iscander
Padawan
Posts: 45
Joined: Sun Nov 06, 2016 2:45 am

Re: What happened (and what will happen)

Post by Iscander » Tue Jan 16, 2018 3:27 am

Looks like reviews and the general SciFi discussion are still missing.

Iscander
Padawan
Posts: 45
Joined: Sun Nov 06, 2016 2:45 am

Re: What happened (and what will happen)

Post by Iscander » Tue Jan 16, 2018 11:56 am

Iscander wrote:
Tue Jan 16, 2018 3:27 am
Looks like reviews and the general SciFi discussion are still missing.
My mistake. These are under "Other" right now

359
Jedi Knight
Posts: 450
Joined: Mon Aug 31, 2015 8:28 pm

Re: What happened (and what will happen)

Post by 359 » Wed Jan 17, 2018 8:55 am

Amazing job restoring the board... again.

And same as Iscander, I offer my help where I can.

Jedi Master Spock
Site Admin
Posts: 2161
Joined: Mon Aug 14, 2006 8:26 pm
Contact:

Re: What happened (and what will happen)

Post by Jedi Master Spock » Thu Jan 18, 2018 2:12 am

359 wrote:
Wed Jan 17, 2018 8:55 am
Amazing job restoring the board... again.

And same as Iscander, I offer my help where I can.
Not sure I can call it an amazing job when the whole board was invisible to non-logged-in visitors without my realizing it. Just found that out and fixed it just now. That's a weird default setting.

There are two ways in which I could use help. One is finding a different hosting solution for the forum's needs, since my current host is presumably going to keep screwing up; the other is passing off the baton for someone else to run it. In the latter case, the lot of you should probably be talking it over (whoever is still around).

I don't have the combination of enthusiasm and spare time for this area that I used to - you've seen it in my absences - and the forum and community has suffered because I was the only admin around. I gave Mike admin permissions recently, but I don't think he had much time to get used to it (and I didn't give him much guidance on how to use the admin tools), and I think he's gotten a little burned out as well, since coming here was more like doing work than having fun (when you have mod permissions, you see all the hidden spambot posts waiting in the moderation queue).

One of the big problems is that without someone actively working to upgrade the software regularly - necessary to protect against spambots - the moderation queue was flooded, which meant that new users could not effectively join the board (I think some were accidentally deleted in the last couple of years, while others lost patience and left after their post didn't show up after a couple of days). That's the real death knell for a community - you need new members if you're going to continue a community.

Mike put a lot of hours into clearing out spambots the hard way, both out of the wiki and the forum - I think more than I have put into cleaning up the occasional catastrophe - and he deserves a lot of appreciation for that. Looking at the inactive user list, the updated software is really effective at blocking spambot registrations compared to the old software, but someone needs to be doing regular maintenance (and regular manual backups).

User avatar
Trinoya
Security Officer
Posts: 651
Joined: Sat Aug 19, 2006 6:35 am

Re: What happened (and what will happen)

Post by Trinoya » Thu Jan 18, 2018 3:07 am

Glad to see that there was enough to be salvaged, I saw the email and was extremely concerned.

Mike DiCenso
Security Officer
Posts: 5767
Joined: Fri Aug 18, 2006 8:49 pm

Re: What happened (and what will happen)

Post by Mike DiCenso » Thu Jan 18, 2018 11:21 pm

Speaking of being an admin, I've noticed that post-restoration of the forum, I don't have any admin privileges anymore, so you may want to restore that to myself and to Trinoya as well.
-Mike

Jedi Master Spock
Site Admin
Posts: 2161
Joined: Mon Aug 14, 2006 8:26 pm
Contact:

Re: What happened (and what will happen)

Post by Jedi Master Spock » Fri Jan 19, 2018 12:42 am

Mike DiCenso wrote:
Thu Jan 18, 2018 11:21 pm
Speaking of being an admin, I've noticed that post-restoration of the forum, I don't have any admin privileges anymore, so you may want to restore that to myself and to Trinoya as well.
-Mike
Noted. Should be fixed, details in PM.

Jedi Master Spock
Site Admin
Posts: 2161
Joined: Mon Aug 14, 2006 8:26 pm
Contact:

Re: What happened (and what will happen)

Post by Jedi Master Spock » Sat Jan 20, 2018 4:54 pm

Fixed the "Topic Started By Guest" issue.

Would have been nice to have known how to do that back in 2015.

User avatar
2046
Starship Captain
Posts: 1891
Joined: Sat Sep 02, 2006 9:14 pm
Contact:

Re: What happened (and what will happen)

Post by 2046 » Sat Jan 20, 2018 7:20 pm

To summarize and enhance some points I made elsewhere…

1. There's too much valuable info here to lose. The Vs. Debate community cannot afford another STrek-v-SWars scenario of lost analyses. Also, foes would claim victory, and retroactively, to boot. "What, no, no one ever seriously questioned us about such-and-such."

2. Maintaining a forum, especially a high-maintenance one, is too great an investment of time and drains the joy out of the thing. To use a racing analogy, most of the folks here are fact-finding wheelmen, and their time is best utilized behind the wheel. Wheelmen may not be auto mechanics and even if they are, spending all the time in the engine bay detracts from the time behind the wheel where the fun is.

(In researching here I see that JMS has been over to ASVS to have a little time behind the wheel, and I imagine it was nice to log in and toss out an argument without having to worry about administration at the same time.)

3. This topic requires long-form interaction that is organized (threaded, certainly, topic-based preferably), which a forum like this provides nicely. It also helps if anonymity is available, what with the crazies out there (plus the fact that many would prefer to geek out anonymously).

Even at 280 characters, Twitter cannot sustain an argument in any sufficiently readable way, and even Reddit really isn't designed for this without some user/mod input (tagging, enforcement of quoting, et cetera). Facebook makes anonymity difficult.

4. Databases are too finicky and too immobile to serve as a proper backup. Hauling around a database that requires phpBB and related things to meaningfully access is messy at best.

The ideal scenario would be to pull off something like I did for the saved STrek-v-SWars pages, converting them to basic portable HTML, but that's still messy and labor-intensive. On the other hand, there are supposedly scripts and scrapers to do the job in a semi-automated way per my searches. One looks especially hopeful.

https://ibboard.co.uk/Programming/how-t ... phpbb.html

4A. I don't even know of it is possible to truly close a phpBB in such a way that no hacking or breakage is possible in an archiving scenario.

Putting all this together, obviously the best scenario would be to have a paid admin with a vigorous automated anti-spambot system and a way to archive to HTML fairly regularly. Moderation duties would still be handled locally, of course, but would be much less trouble in that scenario.

Realistically, though, the best thing to do is get the forum administration off of JMS or anyone else completely, archiving what's here to HTML and moving everyone en masse to some hosted forum or Reddit or something that can still be backed up or is otherwise sufficiently reliable to be trusted not to disappear without warning.

Naturally, I'd be willing to host such a backup. Alternately, I could possibly consider taking on the forum/site as-is but I can't guarantee much in the way of upkeep time, either.

I've been considering opening up a Reddit forum just to experiment, and if there's any interest I can look at that more closely.

Thoughts?

Post Reply