News Server problem

[AgentPete]

Following software upgrades we had a cascading server problem earlier this morning leading to site unavailability for about 25 minutes. During this time it became clear that the issue was not going to be tracked back easily and downtime would probably extend for several hours or more.

I therefore took the decision to get the Colony up and running again as quickly as possible by restoring from last night’s automatic backup. Unfortunately, this means that any posts or personal messages you’ve made overnight (UK time 10:20pm – 9:00am) will have disappeared.

I’m really sorry about this: it’s not an ideal solution but tracing back all the consequential issues arising looked like it was going to be a lengthy piece of work.

Hope this doesn’t too greatly inconvenience those members who may now have to re-create overnight posts.

 

[Bloo]

Well...that explains what happened to my post

 

[izi 出久]

Well...that explains what happened to my post

You’ve been censored

 

[AgentPete]

Well...that explains what happened to my post

I'm so sorry, MD
(the upside is that the backup/restore function works well).

 

[Bloo]

You’ve been censored

I posted in the Spring Reading thread. Irony is that I was kvetching about the crappy doc in the motor control code I'm working on. Maybe some A.I. routine in the server took offence

Anyhoo...I just love it when someone makes changes and doesn't update the comments. Or sprinkles the code with magic numbers.

And I know SPaG is atrocious in source. But would it hurt to use proper spelling for doc, function names, and #defines? It would make them a lot easier to find.

 

[Bloo]

Thanks, AP. I'm sure mine wasn't the only post to get zapped. I just complain more

 

[Katie-Ellen]

@M. Dupré It will only be even better for being re-posted

This last week has been one vexatious IT thing after another. So much so, it has been peculiar, and curiously, all happening the same week that Pluto has ingressed into Aquarius (symbolic-speak for death, rebirth and spending in the domain of IT, also space tech, medicine,mining and crypto - and politics) Yes, we can have glitches any time. Of course. And they tend to come all at once. Murphy's Law. Even so. This ingress will last until 2044. Historical correlations, the last time Pluto was in Aquarius was 1778 to 1798. An age of revolution. American War of Independence from 1775 to 1783, French revolution 1789, but the movement started 1787 and ran till 1799.
Such larks.

 

[Jonny]

And I know SPaG is atrocious in source. But would it hurt to use proper spelling for doc, function names, and #defines? It would make them a lot easier to find.

Err... not for me it wouldn't.

 

[izi 出久]

@Bloo✒️ and I responded to you… that message is also dead.

 

[Bloo]

@Bloo✒️ and I responded to you… that message is also dead.

Bummer...I should've known you'd respond to that one

 

[Peyton Stafford]

Following software upgrades we had a cascading server problem earlier this morning leading to site unavailability for about 25 minutes. During this time it became clear that the issue was not going to be tracked back easily and downtime would probably extend for several hours or more.

I therefore took the decision to get the Colony up and running again as quickly as possible by restoring from last night’s automatic backup. Unfortunately, this means that any posts or personal messages you’ve made overnight (UK time 10:20pm – 9:00am) will have disappeared.

I’m really sorry about this: it’s not an ideal solution but tracing back all the consequential issues arising looked like it was going to be a lengthy piece of work.

Hope this doesn’t too greatly inconvenience those members who may now have to re-create overnight posts.

Ah, technology... Hang in there. I am sure you will sort it out.

 

[Peyton Stafford]

Lost some of mine, too. Until this is resolved, I plan to make copies outside Litopia so I do not have to rewrite posts.

 

[AgentPete]

Lost some of mine, too. Until this is resolved, I plan to make copies outside Litopia so I do not have to rewrite posts.

It is resolved, Peyton. Also, the new "save draft" function will auto save your posts as you're writing them, every 60 seconds.

Just to explain a bit more. I wanted to create a test site (exact copy of the production site) to experiment with some new functionality. I’ve done this before many times without problem, it’s a bit tedious but nothing special.

The server control software we use now provides a “clone site” function that I’ve never used before, seemed pretty safe, and offered to set up a test site with just one click. The test site itself was fine but – in the process, it took the production site (this one) offline. It didn’t just disable the site to copy it, it appeared to disconnect the database from the software that runs the site. After 25 mins looking at logs, credentials etc I was no nearer working out what had gone wrong. It also became clear that the cloned site was in fact still using the main site’s database, not the test one.

At that point, I had to decide whether to involve tech support and risk downtime of many more hours, or simply to get things up and running again while losing the overnight posts.

Again, I’m sorry about the hassle for you.