Mabinogi World Wiki:Wiki Server News/2011/Q2

April 5, 2011

April 15, 2011

Earlier this week, the system went into a high CPU usage mode for unknown reasons. One side effect was that the system went into swap and stayed there. Once Apache's configuration was tuned so that there were fewer PHP processes running and not going into swap, it was then it was discovered that PHP was at the root of the problem. I have since updated PHP to what's currently in Natty Narwhal. At the rate I am going, I may end up going fully Natty even before its final release, even if in piecemeal fashion.

April 26, 2011

I will be putting a hold on any system upgrades for the next few days owing to G14's forthcoming release as well as the ~~forthcoming~~ joint Nexon-Mabinogi World event that ~~will take~~ took place on IRC. I will only install upgrade packages on an as-needed basis, e.g. to fix a stability or security problem.

April 27, 2011

May 3, 2011

Upgraded the operating system to 11.04, codenamed "Natty Narwhal". Will continue to monitor.

May 20, 2011

Both drives in the server's storage array received a firmware update from Seagate. Total downtime: None. The system is running a RAID 1 array which allowed the server to remain completely operational while each drive was removed from the server, its firmware updated and reinserted.

May 21, 2011

May 22, 2011

Overnight, I received an alarm from the server. A user was using a "web crawling" program to "spider" the site, causing the load to jump considerably. As a result, I blocked the software's user agent so any attempts from that user's software would return errors rather than go through the interpreter processes, increasing the system load.

June 21, 2011

Due to a security issue in PHP 5.3.6 and prior releases, uploads have been disabled until further notice until new packages have been released by Ubuntu or I have built a PHP interpreter with the fix in place.

June 23, 2011

Implemented a fix for CVE-2011-2202. The process has taken place at Debian, but not Ubuntu that I can tell. Reporting the issue to Ubuntu.

June 27, 2011

Took the server down to replace four cooling fans. The issue was only that the original fans were quite loud as this server is located within living space. As such, I replaced the fans with something quieter.

Migrated the database filesystem from XFS to ext4fs. This is a move to ensure reliability since some of the recent events of extreme resource usage have had XFS-related messages accompanying them. This will be the first of several such moves. The other moves will be planned and executed soon with announcements to the appropriate areas for notification.

June 28, 2011

Migrating the server log partition to ext4fs. The operation is presently ongoing.

UPDATE: The move is complete. Required a total downtime of a minute and a half to two minutes split between both before and after the operation.

Migration is underway on the partition which contains this and other websites to ext4fs. It is presently being served from a different partition in the meantime while the rest of the partition gets populated. The site needed to be taken down briefly to destroy the underlying filesystem in favor of a new one. Total downtime was approximately ten minutes. I also migrated another filesystem during this time.

UPDATE: The move is now complete. Required a downtime of less than two minutes to sync up the on-new-filesystem copy of the site from the on-temporary-filesystem copy.

June 29, 2011

Had an outage for approximately an hour and a half out of a two-hour scheduled window starting at 2300 hrs on Jun 28, 2011. Procedure involved removal of one of the new cooling fans which had failed, along with migrating the remaining filesystems to ext4fs. The server is now running entirely on ext4fs. Will schedule another brief outage to install the replacement fan.

At 1158 hrs PDT, I took the server down for ten minutes to install the replacement fan. The server now has four working, quiet cooling fans again.

Navigation menu