giza: Giza White Mage (Default)
At 9 AM this morning, we opened up the hotel registrations for Anthrocon 2011. Here are the performance graphs, with annotations:

Network traffic graph (Annotated) Memory usage graph (Annotated)

Netstat (Annotated) CPU usage graph (Annotated)

The arrows point to the spikes at 9 AM when I made the changes to the site live. Here's a breakdown of those stats in more detail:
  • Bandwidth went from a weekly average of 326 kilobits/sec to just over 5 megabits/sec.

  • # of concurrent network connections went from ~41 to 350

  • CPU usage went from 45% of 1 core to roughly 200% of one core (we have 4 cores)

  • Memory usage was largely unaffected. Yay for an async I/O webserver!
So the site held up just fine to the onslaught of users.

One thing I could have done better was the deployment. At about 8:30 AM, I replaced with a static HTML page with the status, and pointed another DNS name at our Drupal installation. Unfortunately, Drupal is a bit finicky with its $base_url variable, and I had to tweak that by hand, as well as tweak the webserver config by hand. I then went to our temporary URL, published the hotel pages, made sure everything was okay, and undid all of those changes to re-deploy the website. Unfortunately, I forgot to clear out the cache in Drupal, which meant some people did not see the new hotel page until as late as 9:02 AM. Okay, so it's only 2 minutes, but I felt that it was a silly mistake that shouldn't have happened.

Next year, I think I'll look into an "auto-publishing" system of some sort to automagically make the pages in question live at 9 AM. That should simplify things for me quite a bit, and maybe not even require me to actively do anything.

Other than these minor issues, the whole hotel thing went over pretty well this morning. I monitored Twitter for the next hour or so, and was able to Tweet status updates to everyone as things happened.

As of this writing, the Courtyard is completely sold out, and Doubles in the Westin are sold out. (but Kings are available) Rooms at the Doubletree are still available (along with suites!), as well at The Omni William Penn.

(X-posted to [ profile] anthrocon)
giza: Giza White Mage (Default)
I've been kinda quiet here for the last month or so, and that's because, after my workplace got a visit from the FBI(!!) a few months back, I decided that was a sign it was time to move on. I had been there for 5 years (an eternity for a software engineer) and really felt like I was outgrowing the position. I wanted to move up in my career and into a position with more responsibility. So I spent a few weeks in November quietly doing job interviews, eventually found an employer that was a good fit, received an offer, and started work for them 2 weeks ago.

The new position has me working as the senior most engineer with a startup in Center City Philadelphia. In addition to actual programming, I also play the part of the sysadmin and get to be a one-man NOC. It's a big step up in responsibility, and with the corresponding pay raise I got at the new job, it's like a promotion of sorts. The company is still small, and only a month old, so I also have the chance to build something really cool as the company grows. It's the kind of challenge that attracted me to computer programming in the first place.

Yes, they're a Drupal shop, why do you ask? :-) Seriously, Drupal is a big piece of software, and I've learned even more about it in the last 2 weeks. This in turn means I do more neat things with the Drupal sites that I run. Everybody wins!

In health-related news, I finally got over to the Rothman Institute (which is like the Anthrocon of sports medicine) and got a diagnosis on my knee issues. And the diagnosis is: patellar tendinopathy, also known as chronic tendinitis. Here's my MRI:

It turns out that this wasn't brought on by exercise, but rather because the length of my patellar tendons are a few millimeters shorter than is normal. This predisposes me to issues with that tendon, and normally affects people when they get to be around my age. The rest of the knees are "perfectly healthy" according to my knee doctor, and this particular injury is treated with physical therapy. This would be different from the physical therapy I had in the spring, in terms of what muscles are exercised.

I asked the doctor if chondromalacia had anything to do with this. He explained to me that my problems were never chondromalacia. The previous issues I had were just warning signs of this. So I see more physical therapy in my future, and hopefully I can get the knees back to normal.
giza: Giza White Mage (Default)
My mental state have improved a bit since last weekend's "Diet Emo Post". One thing that still bothers me are the dreams I am having that involve a specific individual. I didn't ask for these dreams, and I don't want them. I'd really like to stop having them. Anyway...

Onto some other issues I talked about, one was my concern that part of me feels like an asshole. It was pointed out to me that what it really comes down to is impulse control. And if I had a real impulse control problem in my life, I'd have become intimately familiar with the legal system by now. Obviously that has not happened.

Social contact is something else I need to work on. I guess I need to get out more and interact with other people, be they furry or non-furry. That's easier said than done, though, because of transportation to the suburbs, and some interpersonal issues (see the part about me being a bit of an asshole). Not having a car also makes seeing people difficult, which leads me to...

Should I buy a car? Part of the reason why I sold my car is that it's just not practical to own one since I work in the city and take the train to get to work. The rest of my town is fairly walkable, and I have AVIS and ZipCar in case I need wheels for awhile. But being carless makes it difficult to just get up leave on any given weekend. That said, I'm having difficulty convincing myself to buy a car that I'm not going to use 90% of the time. While it'd cost less to operate the car (gas, maintenance), insurance costs would still be the same.

Earlier in the week, I did a health assessment at work. Here are my stats:

Blood pressure: 114/74
Resting heart rate: 69 BPM
Weight: 174 pounds

That's not too bad, considering my injuries have kept me out of the gym for 4 months and I started the year weighing 197. Right now, I wear XL t-shirts, and they're feeling a bit baggy on me. I'm debating going down a size to buying t-shirts that are size Large, but it would mean replacing quite a bit of my wardrobe. If only I knew some gay men who had fashion sense...

On the technical side of things, I played around with a Drupal module called RPX, which allows you to integrate Facebook, Twitter, Google, Yahoo, and just about any other OpenID-based login with a Drupal site. I went ahead and deployed the module on the Save Ardmore Coalition site, as well as the Pennsylvania Furries site.

I also went and set up OpenID delegation using my own website: But then I found, so I think I'm going to look into implementing that at some point.

And now, Sarah Palin and The Rape Kits (no, it's not a band):

giza: Giza White Mage (Default)
There has been some increased activity on Pennsylvania Furries website recently, so I decided it justified finally biting the bullet and upgrading the site from Drupal 4.7 to Drupal 6.16. That upgrade was fun and... interesting. But now that I have a much more recent version of Drupal installed, I've been able to add some features such as these:

- Event Calendar. Want to know when upcoming furmeets, gathers, etc. are? We now have a calendar that anyone may post events to.

- Buddy lists. Yes, they are what you think they are. Users can now have an actual "friends" list on this site. Go to any user's page and click the link towards the bottom to become their buddy. (Here's an example)

- Private messages. Just what the name implies. Users on the site can now send each other private messages. When you're logged into the site, just click the "Messages" link to get started.

- Searching. I reindexed the entire site (and set up crontabs to keep that happening) so now the search engine works again, and posts (and users) on this site are fully searchable.

- Tagging posts. All posts can now be tagged. Even posts made by others. Go ahead. Give it a try. There is also a tag cloud that lists the most popular tags on the site.

- User profiles that don't suck. All user profiles now have links to a number of social networks, along with support for forum signatures, private messaging, and buddy lists.

And the URL for the Pennsylvania Furries website?

Check it out, and let me know what you all think!
giza: Giza White Mage (Default)
One of the neat things about the Drupal CMS is that it has a facility to schedule certain things to be run at regular intervals. For example: sending out subscription notifications or indexing posts for the search function. Those operations can take tens of seconds to complete, so you obviously don't want them executing in the middle of a page load.

The normal way to execute Drupal's crontabs is to load the script cron.php via the web interface. This is a terrible idea, since anyone with an Internet connection could cause your machine to run CPU intensive tasks at well, and bring the webserver to its knees.

An alternative way to run crontabs is with a third-party module called Poormanscron. It's a nice module, but the locking facility it has isn't so hot, and on more than one occasion I've had two cron runs execute in parallel, causing extra load on the machine. In especially bad (read: I/O bound) situations, multiple crontabs get "backed up" onto each other, and that creates huge load spikes and 60-second+ page loads until things calm down again.

There is yet a third way to run Drupal crontabs, and that's via the command line. It gives you maximum control over when and where crontabs run, and this is particularly important when you have multiple Drupal sites running. And I'm going to show you how to do it.

Step 1) Download Drush, the Drupal shell. It's a great little app for accessing your Drupal installation and performing common tasks from the command line. Install Drush (I usually put it in /usr/local/drush/, and make a symlink from /usr/local/bin/drush)

Step 2) Create a script in your home directory, and call it It should look something like this:


# Our main directory for holding websites

export DRUSH_OPTIONS="-q"
#export DRUSH_OPTIONS="-v" # Debugging

# Errors are fatal
set -e

cd $ROOT

# All of our drupal installations under $ROOT

# Loop through our sites to run crontabs on
for SITE in $SITES
   cd $SITE
   # Disable errors, since sometimes crontabs have issues.
   set +e
   drush $DRUSH_OPTIONS cron
   set -e
   cd ..

Be sure to season the $ROOT and $SITES variables to taste as per your specific installation.

If you run a chmod 755 on this script, you can execute it from the command line. However, you may run into file permission issues if the webserver is running as a different user than you (usually the case).

Step 3) Now you need to set up a crontab to run this script. My preferred way to do it is to create a file called /etc/cron.d/drupal-crontabs and place the following in it:

# Run all of our crontabs for Drupal once an hour

15 * * * * www-data /path/to/

Make sure you are running these crontabs under the same user as your webserver.

That's it! If you want to see emailed notifications once an hour, just uncomment the line with the "-v" option for Drush. Otherwise, crontabs on all your sites will be run once an hour. Provided they do not take longer than hour to run, all of your search indexes and subscriptions will be kept up to date, and you will no longer have to worry about crontabs stomping over top of each other with Poormanscron. Enjoy!
giza: Giza White Mage (Default)
Earlier today, I performed a major Drupal upgrade on another site that I run. Part of the upgrade involved me installing the Advanced Forum module to bring the forums a little more up to date with other sites that are out there.

Along the way, I learned something interesting: the Author Pane module does NOT display on blog posts.

It looked rather odd when comments on the blog posts had detailed user info, but the post itself did not. So I set out to fix that. I ended up commenting out the line print $picture; in node.tpl.php and instead adding in these lines:

$account = user_load($node->uid);
$template = "advf-author-pane";
$author_pane = theme('author_pane', $account,
        advanced_forum_path_to_images(), $template);
print $author_pane;

The code is fairly straightforward. It loads user info on the author of the post, and the theme() function loads the author_pane template, passing in the user data.
giza: Giza White Mage (Default)
Wow, it's been awhile since I've written here. Real life has had me very busy lately. I've done some neat things though, and I hope to post more about them soon.

The first neat thing I did recently was to roll out some badly needed updates for the user pages on

Before, I merely used the default pages that Drupal provided. The problem was that the pages looked a little... bland. Among other things, there were no icons for the various social networking services, and that just wouldn't do. So I read up on how to customize the user profile layout in Drupal and spent a couple of evenings writing some PHP code and making use of Drupal's theming functions.

Here are the old and the new pages side by side. Click on either to get a full page in a separate window.

Old and busted:New hotness:

Old default Anthrocon user page

New Anthrocon user page, with theming

The upside of this effort is that when I'm ready to upgrade the Save Ardmore Coalition site to Drupal 6, I can pretty much just copy over my user templates on a wholesale basis, and save myself from having to redo all that work. :-)

How has everyone else been here in LJ-land?
giza: Giza White Mage (Default)
It's been a busy week or so since I posted here last.

Over the weekend, I deployed some Drupal modules on a few sites that I manage. So now the Save Ardmore Coalition site has a tag cloud, and registered users may tag existing content with custom tags. I did the same thing on the Anthrocon website, their tag cloud is over here. I also played around with the "views" module and created some pages for all-time popular posts, highest rated posts, posts with the most comments, and posts with the most votes.

I heard from an old client of mine, who asked if I could debug some ancient PHP code. Something about PHP 3 code not working right under PHP 5. :-)

I got involved with another random programming project. This one is a volunteer gig, and more of a long-term project. I have some exciting ideas for things I'd like to do, and hope they'll save us all some time and effort, as well as increase code quality.

I played around with the Simpletest module for Drupal. The version for Drupal 5 seems to be a bit round around the edges, but useful for what I have in mind, since it will be a start.

I took a look at the docs for the Yii PHP Framework. It doesn't look so bad, and the documentation is fairly detailed. I have concerns about some of the naming conventions on their classes, and the fact that a lot of the classes are invoked via static calls. (Makes unit testing and dependency injection a little difficult...), but at least there was some thought put into the way things were done. (Are you listening, PHP team?)

I am getting ready for Furry Connection North next month. Last year's convention went really well for being a first year con, and I've been impressed at how well the preparations are proceeding for this year's con. It should be fun, or at least interesting. :-)
giza: Giza White Mage (Default)
The folks running thought they would upgrade the site from Drupal 5 to Drupal 6 tonight and proceeded to take the entire site down, not even leaving a copy in any sort of "read only" mode. It was supposed to be back up by 5 PM EST. Clearly that has not happened yet. Not only has the Drupal team been not very forthcoming on what the holdup is, they have the nerve to ask for donations while the site is down.

I'm sure there's more to the story of the botched site upgrade than we know so far, and I'd love to hear their side of things when the upgrade is all said and done.

And in more positive Drupal news, the Obama team has embraced Drupal, by using it to power Good for them.

Full article: Why the White House's Embrace of Drupal Matters

Drupal is the same software that powers the Anthrocon website, my website, and many other websites out there. Naturally, I am pleased by this. :-)
giza: Giza White Mage (Default) forum activity

And for the curious, here's the SQL to make it happen:

SELECT DATE_FORMAT(FROM_UNIXTIME(timestamp), "%m-%Y") AS date, count(*) AS num FROM comments GROUP BY date ORDER BY timestamp;
giza: Giza White Mage (Default)
While using the nginx webserver with a Drupal installation I noticed something odd. Whenever I submitted a form (with the GET method) or clicked on a link that had a space in it, it would be converted to a plus sign in the URL. Actually, that's not the odd path--the web browser is doing proper URL encoding since spaces are not allowed in URLs.

The weird part is that after the request got to the webserver, and the server processed it and passed it to PHP, and PHP loaded Drupal and executed the PHP code, the plus sign failed to be turned back into a space when using the nginx webserver. Works fine in Apache, but not in nginx.

The symptom is that if I tried to perform a search or edit my user profile, or anything else that involved a plus sign in the GET data, when that data was processed by PHP, the plus sign was still there. i.e., I would see "term1+term2" in the search box, instead of "term1 term2".

So, how to fix this? I could have spent some time reading RFCs, and looking through the source from PHP and nginx, or I could have my site working again. I opted for the latter, and found that this line of code at the very beginning of Drupal's index.php seems to do the trick:

$_GET["q"] = strtr($_GET["q"], "+", " ");

It's silly, and may even be a bit stupid, but it sure does the trick.
giza: Giza White Mage (Default)
Since I decided to stay at my place tonight to work on (yet another) website migration to Drupal, I figured I would invite over a few friends:

Talisker Gift Set
giza: Giza White Mage (Default)
The problem

After I brought Anthrocon's room share and ride share forums online, I noticed that last year's posts were still present. This was a problem because people needing rides or rooms for this year's conventions did not notice the date and were replying to those posts, thus wasting everyone's time.

Now, I dislike removing content from any website I manage, since that can potentially hurt Google's PageRank on the site. If only there were some way of removing the old posts from those forums without actually deleting the posts...

Then I remembered that Drupal's database is in third-normal form and came up with this query after about 15 minutes of fiddling:

CREATE TABLE temp_nids (nid INT(10) UNSIGNED);

INSERT INTO temp_nids (
   SELECT nid FROM term_node WHERE nid IN (
      SELECT n.nid FROM node AS n
         LEFT JOIN term_node AS tn ON n.nid=tn.nid
         LEFT JOIN term_data AS td ON tn.tid=td.tid
         WHERE IN ('Room Share', 'Ride Share')
         AND FROM_UNIXTIME(n.created) < '2007-08%'

DELETE FROM term_node WHERE nid IN (SELECT nid FROM temp_nids);

The innermost query (SELECT n.nid FROM node...) selects node IDs that belong to the Room Share or Ride Share forums with a creation date before August, 2007. The query around that (SELECT nid FROM term_node...) I originally had in to make sure that we got valid node IDs from term_node. Given how the query evolved, that's probably no longer necessary. The outermost query (INSERT INTO temp_nids) stored the matching node IDs in our temporary table for later use.

The final query (DELETE FROM term_node...) deletes the offending node IDs from the term_node table, which is responsible for linking nodes to taxonomy terms.

In other Drupal news, I stumbled across a nice little article the other day called 10 Reasons to Use Drupal CMS. While I knew some of the things mentioned in that article, I had no idea that entities such as The United Nations, Forbes, The Discovery Channel, AOL, and most surprisingly of all--The Grateful Dead.. all use Drupal. Fascinating stuff.

Also, I found the website Drupal Dojo which contains lots of tutorials on how to perform different tasks in Drupal. It looked just like another how-to type site (not that there's anything wrong with that!) until I came to the article on patch rolling and saw this:

Um, yeah... I sure wasn't expecting to see that particular graphic. :-P
giza: Giza White Mage (Default)
At 8:20 AM this morning, know where I was?

In a bar.

No, it's not what you think. I was just meeting folks there for breakfast. :-P

In other news, I made more progress on my script to convert a Mir CMS database to a Drupal database. I can't wait to finish it!

I think it's time to take a nap now.
giza: Giza White Mage (Default)
Earlier tonight, I was trying to figure out why a Drupal installation of mine was running kinda slow. So, I installed the awesome devel module and had it print out the list of database queries. What I found when going through the forums was on each post there were dozens and dozens of calls to drupal_lookup_path() for URLs which I never had (or will) make aliases of. Namely URLs that start with "comment/" or "user/".

So, I wrote a little patch for that function which will skip querying the database for URLs like that:

--- includes/   2007/09/14 00:06:12     1.1
+++ includes/   2007/09/14 00:15:21
@@ -44,6 +44,21 @@
   static $map = array(), $no_src = array();
   static $count;
+       //
+       // Drupal does an INSANE number of these queries for comment lnks
+       // and such on long threads, and that's just unacceptable.  Since
+       // I plan on like, never setting up a URL alias on a comment or 
+       // a user, I'm just going to stop here...
+       //
+       if (
+               stristr($path, "comment/")
+               || stristr($path, "user/")
+               ) {
+               return(false);
+       }
   // Use $count to avoid looking up paths in subsequent calls if there simply are no aliases
   if (!isset($count)) {
     $count = db_result(db_query('SELECT COUNT(pid) FROM {url_alias}'));

You will see the most gain with forum posts that have many comments, but even on regular pages you should see some benefit. Enjoy!
giza: Giza White Mage (Default)
Last night I dined at Morimoto's in Philly. The food was amazing, and the sake wasn't bad either! It was very much worth it.

This morning, while in a company meeting, we heard the following sounds from the nearby kitchenette in rapid sequence:

- "Oh shit!"
- The sound of something dropping and breaking
- The sound of someone running away from the kitchenette
- The crashbar on the outside door being pushed and the alarm going off.

We're still not sure what happened, and the whole thing was pretty surreal. When we finished up the meeting, nothing appeared to be broken.

And finally, I stumped across this Drupal success story that I enjoyed reading about. It talked about what makes Drupal so attractive to deploy with -- the fact that it is a fairly robust web publishing system that does not require lots of coding. It and its modules work right out of the box and you can instead concentrate on publishing.
giza: Giza White Mage (Default)
So, I still need to release my backup module for Drupal 5. But I haven't had a chance to sit down and actually read the documentation.

But, awhile back, I developed my own command-line utility for backing up my own Drupal websites. Since it was quicker to clean that up and release it instead of waiting for me to Finish the Drupal 5 version of the module, I figured I would do that.

The program can be downloaded from here.

It needs to be run from the root directory of DOCUMENT_ROOT from the command line. It will automatically read in your default settings.php file to get the database credentials and create its own backup of the database. Then it will backup the webserver's filesystem and store both database backup and filesystem into a nice little tarball, ready for download via your web browser.

For those of you who are worried because the file is essentially world-readable, there is a random number appended to the database dump and the final backup to prevent an attacker from guessing the name of the file. (your webserver would fall over from the 404s before someone successfully guessed a filename)

giza: Giza White Mage (Default)
Awhile back, I was working on a private Drupal website for members only. Now, Drupal has a module called Secure Site which can be used for this. However, Secure Site throws up an HTTP 401 error to request authentication from the user. If you've ever seen a "enter username and password" box pop up in your browser, that's an HTTP 401 error. That's okay, but I don't think it's the best UI out there. I wanted something a little user-friendlier.

So, I came up with the code below. Put it in your index.php file (or better yet, in a file included by index.php) right after the call to drupal_bootstrap(). It will redirect all but a certain subset of pages to the login page, and prompt users to log in.

$path = getenv("SCRIPT_URL");

// If a user is not logged in, they can only access certain unrestricted pages.
if ($user->uid == 0) {
   if (
      // strstr() is called for efficiency.  Keep in mind that ANY path that matches
      // these strings will be allowed to anonymous users.  So if you have something
      // like "/userlist", an anonymous user can view that.  I warned ya!
      !strstr($path, "user")
      && !strstr($path, "how-to-join")
      && !strstr($path, "contact")
      ) {
         form_set_error("", "You must be logged in first.");



[Edit: Added blurb about security. Thanks for pointing that out [ profile] taral.]
giza: Giza White Mage (Default)
It looks like Computer World now uses Drupal for their blogs.

Also, another Drupal-powered site that some of you may have heard of: The Onion.
giza: Giza White Mage (Default)
Yes, I know about The Drupal Association. 3 different people have told me in the last 24 hours. Y'all can stop telling me now. [Edit: But I appreciate folks thinking of me, though!]

Anyway, while I'm rambling, I wanted to briefly mention IM. No, I'm not in IM much anymore. I found it to be too distracting during the evenings while I'm trying to work on a website or just want to chill in front of the TV for a bit.

And what is it with people who I do not know, that keep trying to get my attention via IM, and then flip out when I don't give them 100% of my attention and reply instantly? I have no idea who you are -- which means I'm not likely to start up a conversation with you, okay? Is there any particular reason why you are unable to use email to communicate with me?

I'll shut up now.


giza: Giza White Mage (Default)
Douglas Muth

April 2012



RSS Atom

Most Popular Tags

Style Credit

Expand Cut Tags

No cut tags