everydns.net go BOOM!

[giza]

So, I run a few hobbyist/non-profit domains, and I've been using the DNS services of EveryDns.net.

At 3:45 PM EST today, all 4 of everydns's nameservers stopped responding. All 4 servers are on seperate networks and in seperate physical locations.

This has "DDoS attack" written all over it.

The following domains are currently having interruptions in service:

- anthrocon.org
- claws-and-paws.com (including pafurry.claws-and-paws.com and lists.claws-and-paws.com)
- jbadger.org
- saveardmorecoalition.org

Not much we can do, other than to wait things out. Unless this DDoS lasts for more than 48 hours (typical time to propagate new NS records for domains), switching DNS providers probably won't help.

"What is DNS?"

DNS is the service that maps machine names to network addresses. That way, if you move a website onto a new machine, you simple need to update the DNS entry with the new address. Think of it like a phonebook.

"So the websites themselves are not affected?"

Correct. The websites and the machines they are running on are just fine. Though they aren't processing much traffic right now. :-P

"So, who is responsible for this?"

I don't know.

"How long will this continue?"

I don't know.

"How is this even possible?"

The main tactic for DDoS attacks these days seems to involved 0wned Windows boxes that are used as zombies. Thousands of 0wned boxes will be "commanded" at the same time to flood a specific host (or hosts in this case) with traffic. Since the traffic is coming from computers all over the world, it's non trivial to filter out, especially if it is disguisedto look like normal HTTP or DNS traffic.

Boxes typically get owned through spyware, viruses, and/or worms introduced through trojan horses.

"Are there any preventative measures we can take?"

Yes, if you own a Windows box, make sure it is kept up to date. Also, make sure that you are running both anti-spyware and anti-virus software. Having Internet access is a wonderful thing, but certain responsibilities come with that. If you don't know how to keep your machine up to date, find someone to help you out.

[Edit: Things are back to normal now.]

Comments

[rebelsheart]

If we convince all the furs with Windows boxes to turn of their machines, would it have an affect?

Would it be more effective to talk to their parents and otehr family members about turning of theirs? ;)

Or do we need to start targeting random corporations?

[giza.livejournal.com]

Neither.

It's home users in general who tend to be the least computer saavy.

And turning off machines won't help. An infected machine can still participate in a DDoS while it is being used at the keyboard.

[shockwave77598.livejournal.com]

Wasn't there something in the paper midweek about AlQueda calling for attacks against the internet to cripple our financial well-being?

[giza.livejournal.com]

That was against Wall Street.

And everyDNS isn't part of the core infrastructure like say, the root servers. Chances are, someone specifically has a grudge against EveryDNS, or perhaps one of their customers.

[greenreaper.livejournal.com]

Probably the botnet organizers themselves, since EveryDNS is rather active in combatting them.

[giza.livejournal.com]

Holy crap, you're right! (http://64.233.161.104/search?q=cache:M8nznQgcSBwJ:blog.opendns.com/category/everydns/+everydns+botnet&hl=en&gl=us&ct=clnk&cd=1)

[sr-foxley.livejournal.com]

So... have you started looking into other options for hosting your DNS, assuming the attack on EveryDNS persists for a while? (On that note... I do run DNS on one of the servers I own for several domains...)

[giza.livejournal.com]

Yes. If this keeps up when I get home tonight, I'm going to throw the switch for a temporary solution.

[sr-foxley.livejournal.com]

Excellent! And do give a shout out if you want an extra DNS slave server in the registration list, eh.

[giza.livejournal.com]

Thanks for the offer. I might look at that as a potential longer term solution.

Are you running DJB's DNS server by any chance? :-)

[sr-foxley.livejournal.com]

Nope, just a chrooted ISC BIND 9. :)

I've not actually futzed with DJB's DNS... is it decent?

[giza.livejournal.com]

Oh yeah. It's fairly simple, and security is a big goal. Zone transfers are just done via ssh, for example.

I got away from BIND and Sendmail years ago because of all of the security issues that they were having. DBJ's qmail is pretty awsome, too. When I test a new build, I can just start the binary from the command line and initiate an SMTP conversation right there.

[kreggan.livejournal.com]

Bind 9 is a complete rewrite. Where bind 4 and 8 shared the same codebase written in the early days of the 'net when security wasn't a consideration, bind 9 was written just recently ('99) with security in mind from the beginning. It does all the right things, like trap overflows, check inputs and outputs, and if there's an assertion failure, it shuts down instead of giving the user any sort of access to the box it's running on. With the exception of a buffer overflow in a linked library (openssl), the only vulnerabilities in bind 9 have been DoS attacks - The server shut itself down rather than allowing itself to be compromised.

[giza.livejournal.com]

I stand corrected. Thank you for pointing that out.

Most of my experience with Bind was Bind 8 (8.3.3, I think), and I only used Bind 9 for a few months before I ended up switching to UltraDNS, and eventually to EveryDNS.

[bigtig.livejournal.com]

Perhaps the actual server could itself, be DNS backup.

Bind takes very little overhead for AC's traffic and could easily be slaved to EveryDNS's main server for the site.

Mussing about with the TTL or snapshotting an AXFR would give you the ability to live if this goes again.

Probably something handy to do for your own sites as well.

Oh. I'm more than willing to do redundant DNS for AC and your own sites to give it some geographical redundancy and health. My stuff is currently hosted in Toronto.

[giza.livejournal.com]

My only concern is the RAM on our box. It's already pretty scarce.

I did manage to find another DNS provider, though. I set up the records and I'm going to throw the switch when I get up in the morning.

[sr-foxley.livejournal.com]

Cool-- Glad to hear you were able to get things worked out there. Let me know if you ever want another DNS slave in the mix in any case, eh. (And yes-- with Bind 9, it'd probably be easiest just to set things up with AXFR (assuming, of course that that works between djbdns and bind-- there seems to be a lot of ranting about bind on the djbdns site...))

[kellic.livejournal.com]

You might want to post this on AC's Livejournal so people don't start freaking out and start jumping to the conclusion that AC has closed its doors. Hey whoever said that furs can be rational? :xP

[giza.livejournal.com]

Done

[zeeth_kyrah]

By the way, babsbunny has the undamaged version of that userpic.

[ferdiaferlin.livejournal.com]

Will the move just be temporary? I think it would be nice to support a group that is trying to make things better.

[giza.livejournal.com]

I don't know yet.

I think EveryDNS is awesome, but from a sysadmin perspective, I have a responsibility to keep those sites up and running.

[jbadger.livejournal.com]

I figured I would check here on LJ if you noticed anything. I noticed btw a bit earlier around 7:30am this morning something was going on.

Now it seems more machines are being attached. I can not seem to ping mail.claws-and-paws... at the moment via the last known IP address. 69.56......

5 packets transmitted, 0 received, 100% packet loss, time 3999ms
Ohh well time to sleep.

[giza.livejournal.com]

This appears to be something different.

It's actually a problem I've had before with where that machine is hosted. I opened up a trouble ticket with them.

[giza.livejournal.com]

On investigation, it appears that I borked a couple of DNS entires in my paniced moving over of them.

I'm fixing the problem now. With a TTL of 600, you should have access again soon.

[jbadger.livejournal.com]

Ohh very good, mail is started again!

Thanks Giza!

[jbadger.livejournal.com]

EEEP something happened yet again this morning (Tuesday)
but it just seemed to be a bump, back up after 2 hours.

[giza.livejournal.com]

I don't see any interruption in the machine's uptime. I'm guessing it was just a transient network issue.