What happened to GFF?

This site disappeared and didn't work for almost three weeks in August and September 2023.

To understand what happened, you may need a little understanding of domains, web servers and name servers.

DNS-errors

Martin Joergensen

DNS for fly fishers

This is the simple explanation (even though it’s complex enough):

1. A website runs on a web server. It holds all the content and the programs that serves it.
2. The server is attached to the internet, and has a so called IP-address where it can be found.
3. But in order for people not to have to remember four or six basically random three digit numbers, web sites have a domain name.
4. A domain is a human readable name, in our case globalflyfisher.com
5. When you want to see the Global FlyFisher, that is what your browser looks for
6. It asks the so called DNS (Domain Name System) where to find the domain
7. The DNS directs it to a name server, which holds information about the domain name
8. And the name server tells the browser which IP-number to go to to get to the web site
9. So finally your browser connects to the web server, and the web server sends the requested content

This is s swift, smooth and painless operation, which lasts a few milliseconds – under normal circumstances.

Russian ransomware

In our case, the company that runs the name server – previously SN-Hosting, then Azero, then AzeroCloud, now CloudNordic – was hit by ransomware and had to shut down all service on all their systems and allegedly lost all data on said systems.
Like all data!
Production data, primary backups and secondary backups.

Great backup strategy by the way: having all backup systems connected to each other. Ever heard about off site backup, guys?
Well, enough irony.

The result was that in the above list, item 7 was a dead end. Whoever asked for globalflyfisher.com was sent to a non-existing or non-responding name server and met with an ugly error in their browser.
And even though the name servers ns.domainteam.dk and ns2.domainteam.dk were up and running after some days, they were devoid of content, and inaccessible to me as an administrator. So I had no way of getting the DNS-records back in place, which would have been a fairly easy job.

GFF was running all the time

GFF runs on a web server maintained by me. It has been ticking along steadily for years – and did so through this whole ordeal – and still does. But it just wasn’t reachable during the outage, so no matter what I did on this server, it was to no avail. People couldn’t find it.

Soon after the breakdown I registered globalflyfisher.dk, and set up that domain to point to the server, and in spite of some initial problems with security certificates, that worked.
But no one looks for globalflyfisher.dk.
Everybody knows the site as globalflyfisher.com, and that includes users, other sites, search engines, social media, newsletters and whatnot.
So all these incoming links had no benefit from the .dk domain, but just bumped into that DNS wall, and came no further. Same thing with my mail, which was also just lost.
Many will have given up, and concluded that the site was dead an gone.
But it wasn’t!

Server load

Martin Joergensen

Trying to regain control

Usually the information on name servers is controlled by the registrar, which is the company that originally handled the registration of the domain. The thing is that globalflyfisher.com was registered in 1998 by Steve Schweitzer, and we haven’t had to deal with administration of that domain at all for all these years. I think Steve handed it over to me years ago, but I haven’t had the need to interact with the registrar, and if he did, he probably handed it to martin@globalflyfisher.com, and guess what … that mail was also dead due to the whole DNS-problem.

So imagine me on the mail, phone, water pipes, jungle drums and whatnot to try to resurrect the domain.
Absolutely fruitless.
None of the operations answered their phones or replied to any mails, even though they urged you to contact them. I guess they were pretty busy trying to salvage the remains from the fire.

The registrar of the domains, Ascio, denied me access to their system because I wasn’t a customer with them. The domain was registered through a reseller they said.
But contacting a reseller … from 1998 … duh?
Ascio probably didn’t even exist in 1998! And it was up to me to find the information on this reseller and find “the original receipt” or the “data mailed to me in connection with the registration”.
Sure! Everybody has their mails from 1998 stored and readily available.
Well, I actually do have my mail back from 1996 and on, but since I didn’t register the domain, it wasn’t in my archive.

So it was really, really, really (like REALLY!) difficult to prove my ownership of this domain, and none of the companies, I asked to help, were really willing or able to do so.

Nasif to the rescue

At long last I got what I needed. It took a lot of moaning and bitching to all the supporters I could reach, complaining about their incompetence and repetitive knee jerk replies.
They sent out the same cookie-cutter messages again and again, directed me in circles between Ascio, AzeroCloud, RockyNordic, CloudNordic and WhichEverFrigginNordicCloud they could think of, never coming up with an actual solution.

But a supporter named Nasif from whichever of the companies it was, obviously felt a little sting of conscience after I had mailed their ticket system basically all my personal information including pictures of ID-cards, shoe size, preferred ice cream flavor, all current and previous addresses and all else that they requested.
He had the key to a solution: the authentication codes needed to re-delegate the domains to a new host and name server. So finally I could order the domains again with another hosting company, enter the codes and get the moving started.

This of course required a confirmation via mail, which I got for some of the domains, but not for GFF, which is registered to martin@globalflyfisher.com – who can’t check his mails!
Luckily a five day wait was all it took, and they would then start the process without me directly confirming it.

Meanwhile I set up all the DNS records with the new host – one that I’ve been using professionally for more than a decade and trust – and as soon as the transfers went through, all was OK – sort of …

Changing DNS this way does give some glitches, and I had also set up the site to respond to globalflyfisher.dk, and had to reset that. But after a couple of days it almost seems that I have found all the nook and crannies where little errors hide, and I hope I have the site back in shape.