HF and Pages Loading Slow

GasBandit

Staff member
Just got this from the host:

[#QOD-994-85308]: IMPORTANT: Migrating your OS following CentOS 7 sunset

Recently, we have noticed that you have a server with CentOS 7, which reached its end of life on June 30, 2024.
The IP address of the server in question is as follows: 192.64.115.225
Please be aware that continuing to use CentOS 7 carries significant security risks, since there is no support for this OS anymore (including fixes for any vulnerabilities). That's why we recommend that you update your OS.
Please make sure to take a fresh backup of your server before upgrading OS. Here is a video guide for your convenience: https://www.namecheap.com/support/k...255/video-how-to-create-a-full-cpanel-backup/ .
Recently, there's been an increase of reported failed cases of migrations via Elevate script used for CentOS 7 upgrade. That's why if you perform the upgrade on your end using it or other methods, make sure to have a backup.
Alternatively, we can assist with the OS upgrade by transferring all your cPanel accounts to another server with the desired setup. This ensures a rollback option, unlike the Elevate script.
We look forward to hearing from you soon.
I do recommend letting them migrate to an upgraded OS.

But I don't think this is related to the recent slowdowns, which seem to have abated.
 
I knew CentOS was being sunset (sorta -- Thanks a lot, Red Hat!) but of course had no idea it impacted HF. The only thing I ever personally used CentOS for was as a free, Java-capable OS for my first attempt at a standalone Minecraft server. So maybe all these slowdowns were due to other shared VMs doing their upgrades?

The "good" news is that it looks like there is a "most compatible" upgrade path directly to full-fat RHEL, though that means the headache of migrate and config and all that, plus of course having to subscribe to RHEL. The "good" part is that it looks like RHEL now allows you to register for a free production-capable license/subscription tier/option for people using 16 or fewer systems.

--Patrick
 

Dave

Staff member
I do recommend letting them migrate to an upgraded OS.

But I don't think this is related to the recent slowdowns, which seem to have abated.
If everyone else was migrating could that have not taxed the servers enough to slow us down?
 

GasBandit

Staff member
Says the man I'm pretty sure is still running Windows XP
Actually (begrudgingly) window 10 now, but there's also a difference between an outdated pc with one tech-savvy user behind a router and an outdated webhost fully exposed to the cloud, literally blocking new bot registrations constantly and already experiencing performance issues.

If everyone else was migrating could that have not taxed the servers enough to slow us down?
Ehhh, if they all did it simultaneously, maybe, but that'd have been an astonishing coincidence that everybody on our VM chose to migrate at the same time. My money's still on a DDOS attack.

Regardless, getting off CentOS 7 is a good idea. There's some other issues we've been ignoring or band-aiding too because fixing them would require updating something to a version that requires something else that in turn requires an OS update.
 
If everyone else was migrating could that have not taxed the servers enough to slow us down?
maybe all these slowdowns were due to other shared VMs doing their upgrades?
...I don't know which of you would have to register for the developer account. But if it's Gas, then that would mean Gas also gets to use that license to legally install/use RHEL on up to 15 of his own personal machines for ... free.

--Patrick
 

GasBandit

Staff member
It's not mysql doing it this time. I see it happening in real time - something is hammering the CPU via php.... not sure what I can do about this. Might be a DOS attack.
 

GasBandit

Staff member
Well, while I was hunting around for the stuff to fix, the CPU usage dropped from 20% to 1%... so... guess it passed again for now
 

GasBandit

Staff member
Had another instance going around 9:10 am Central again, but it went away before I could pin down anything useful. It really smells like either DDOS or attempting to do a buffer overrun or something.
 

GasBandit

Staff member
I’m getting it right now but I’m at work and can’t look at shit.
I'm seeing it and looking into it too. I'm actually thinking somebody is trying to brute force nihsen.com, and because both "sites" are really the same site, it's affecting halforums too.
 

Dave

Staff member
Strange. Wonder if it’s our political threads. I might need to lock things down for a bit if so.
 

GasBandit

Staff member
Strange. Wonder if it’s our political threads. I might need to lock things down for a bit if so.
I don't think it has anything to do with our actual content. The error messages I'm seeing looks like automated attacks against PHP and looking for a wordpress admin page (which this site doesn't run wordpress)
 

GasBandit

Staff member
Ok, so has anybody noticed a slowdown since yesterday? I haven't, and I'm wondering if the thing I did "worked."
 

GasBandit

Staff member
Just to note, what I did, is I explicitly blocked the IPs of the 4 most aggressive SEO bots I could find just by eyeballing apache logs.
 
I was noticing slowdown a few minutes ago but now I'm not. Fingers crossed, what you did seems to have helped.
 

GasBandit

Staff member
Ok, wow, that seems to have immediately helped. Down side is - the drastic action was I had to make the forum only allow any kind of access to people who are registered, verified, and logged in. We were getting hammered on tons of outdated threads by a vast array of disparate IP addresses.

I hate having to circle the wagons and pull the drapes like this.. .but it did immediately fix the problem.

So I guess... now is the FORTRESS HALFORUMS era.
 

GasBandit

Staff member
So it's helping some... normal CPU load is 0.5-2%. Shit starts to get shitty around 15-20%, and these apparent intermittent attacks have been pushing things above 20% for periods of time. Locking out unregistered users, CPU usage has fallen to ~6%, which tells me that the attempts are still happening, but now just rapidly serving 403-Forbidden pages take a lot less CPU than the full on mysql queries that were getting hammered out via PHP to deal with this nonsense, and apparently 6% isn't enough to notice a visible performance degradation.
 
Ok, wow, that seems to have immediately helped. Down side is - the drastic action was I had to make the forum only allow any kind of access to people who are registered, verified, and logged in. We were getting hammered on tons of outdated threads by a vast array of disparate IP addresses.

I hate having to circle the wagons and pull the drapes like this.. .but it did immediately fix the problem.

So I guess... now is the FORTRESS HALFORUMS era.
I already fear the outside. They are dangerous and we should attack any that approach.
 
Top