After a little over a year running on Tech Blog 2.0, you are now viewing version 3.0. For this version, we've returned to WordPress from BlogEngine. There are several issues that colluded to drive this change, most of which surrounded PHP and its crazy behavior. (Geeky details follow - skip to the paragraph starting with “Bottom line:” if you don't want the geek stuff. I bolded it so it would be easy to spot.)

PHP's recommended configuration is to run under Apache using the pre-fork multi-processing module (MPM). The advantage to this is that Apache does not have to spin off another process to handle each request; it handles it in the same thread. However, this means that each instance of the server must have all enabled modules loaded. This means that each instance of the server (AKA “thread”) is very large, so the number of threads run is lower (typically 5-15 in a server the size we're on). Also, this means that each thread can only handle one request at a time; if you have 7 threads configured, each serving one of 7 requests, and an 8th request comes it, it has to wait for one to finish. If the requests are served quickly, this may not be a problem; however, the avalanche of request that follow the typical front-page mention on mega-blogs can easily overwhelm it.

To fix this problem, there is another MPM, this one called worker. In this scenario, there are spare thread waiting to fill requests, and these can spawn other threads to do further work if required. So, the Apache threads would realize that a request needs to be handled by PHP, and pass it off to that process to be completed. The Apache memory footprint is much smaller; it serves the images, scripts, and other static files, and passes off the requests that require heavy lifting. PHP, then, has a (FastCGI) process where it receives these requests, processes them, and returns the response to the caller. Because each of these threads only has to load the PHP requirements, they are smaller too, so you can have more threads processing at the same time; you just might survive that front-page mention! (This is the same technique applied by LightTPD and Nginx, two other servers I tried at various times.)

It is in this scenario where PHP fails to live up to its expectations. These PHP processes would simply stop responding, but the controller thinks they're still there. The end result to the user is a site that just sits and waits for output that will never come. Eventually, they may receive a Gateway Timeout or Bad Gateway error. The problem is worse on slower sites, but even popular sites seemed to fall victim to this from time to time. This was also a problem whether PHP controlled its threads, or Apache controlled them.

The one thing that really perturbs me is instability. If something is broken, I can fix it; if it works, I can fix it 'til it's broke. :) But something that works sometimes, and other times doesn't, simply won't fly. I was able to introduce some stability by restarting the server 4 times a day, but that's a band-aid, not a long term solution. I was tired of fighting.

Bottom line: the configuration required for a stable server is in opposition to a lean-and-mean configuration. So, I installed the required Apache modules, and will continue to run my PHP-serving server at a configuration twice as large as it needs to be. I'll eventually move the Mono (.NET) processes to another machine, where the fast configuration won't cause stability problems.

But, PHP isn't all bad. While I would still heartily recommend BlogEngine.NET to someone who was going to serve the blog from a Windows machine, but I had some issues getting upgrades to go smoothly under Mono. It also is optimized for fast serving, at the expense of RAM. At this point, that's not the tradeoff we need.

Finally, with this update, the blog has received its first new theme. It's a clean, clear theme that should serve the content well. Plus, the social media icons up in the corner are just too cool, IMO. I've also applied tags to all posts except the “My Linux Adventure” series, and this theme displays them. (Comments are not here now, but will be migrated shortly.)

So, there you have it. Enjoy!

After three years on WordPress, The Bit Badger Blog has moved to BlogEngine.NET. There are several reasons for this change, some technical and some not.

• PHP's Fast CGI processor has a problem where, if all of the processes are busy, the server will simply time out. While this hasn't afflicted my server as much as others, it has caused problems; when this problem occurred, none of the PHP sites were accessible.
• Through experience with a very heavily-used site, I became less enamored of WordPress's “read from the database every time” way of doing business. I also found that various caching plug-ins for WordPress, on this particular site, did very little to ease the load.
• Since I first looked at Mono (Linux's implementation of the .NET framework), it has matured significantly. It supports most of C# 4.0 already, which was released earlier this year.
• BlogEngine.NET is a rapidly-maturing blog platform, and the project has a stated goal of 100% compatibility with Mono. This is good, because you can mention Mono problems to the team, and you're not dismissed because you're running Linux.

As part of the move, the URL has changed; the new link is https://blog.bitbadger.solutions. I have implemented redirection for each post, the category and category feed links, and the main blog feed and home page from the old URL, so you may not have even realized that you're looking at the new site. The Bit Badger Solutions Software Repository remains at https://hosted.djs-consulting.com/software/.

I'm looking forward to this new setup!

(NOTE: The next-to-last paragraph was updated with correct links as of February 2017.)

I found a script over on the Lunarpages Forums about using PHP to back up your site. I have taken it, modified it a little, beefed up the documentation a lot, and am now posting it here. You can copy and paste it from below to customize it for your own use.

<?php
/**
 * Generic Backup Script.
 *
 * To configure this script for your purposes, just edit the parameters below.
 * Once you have the parameters set properly, when the script executes, it will
 * create an archive file, gzip it, and e-mail it to the address specified.  It
 * can be executed through cron with the command
 *
 * php -q [name of script]
 *
 * You are free to use this, modify it, copy it, etc.  However, neither DJS
 * Consulting nor Daniel J. Summers assume any responsibility for good or bad
 * things that happen when modifications of this script are run.
 *
 * @author Daniel J. Summers <daniel@djs-consulting.com>
 */

// --- SCRIPT PARAMETERS ---

/*  -- File Name --
	This is the name of the file that you're backing up, and should contain no
	slashes.  For example, if you're backing up a database, this might look
	something like...
$sFilename = "backup-my_database_name-" . date("Y-m-d") . ".sql"; */
$sFilename = "backup-[whatever-it-is]-" . date("Y-m-d") . ".[extension]";

/*  -- E-mail Address --
	This is the e-mail address to which the message will be sent. */
$sEmailAddress = "[your e-mail address]";

/*  -- E-mail Subject --
	This is the subject that will be on the e-mail you receive. */
$sEmailSubject = "[something meaningful]";

/*  -- E-mail Message --
	This is the text of the message that will be sent. */
$sMessage = "Compressed database backup file $sFilename.gz attached.";

/*  -- Backup Command --
	This is the command that does the work.

  A note on the database commands - your setup likely requires a password
	for these commands, and they each allow you to pass a password on the
	command line.  However, this is very insecure, as anyone who runs "ps" can
	see your password!  For MySQL, you can create a ~/.my.cnf file - it is
	detailed at //dev.mysql.com/doc/refman/4.1/en/password-security.html .
	For PostgreSQL, the file is ~/.pgpass, and it is detailed at
	//www.postgresql.org/docs/8.0/interactive/libpq-pgpass.html .  Both of
	these files should be chmod-ded to 600, so that they can only be viewed by
	you, the creator.

  That being said, some common commands are...

  - Backing Up a MySQL Database
$sBackupCommand = "mysqldump -u [user_name] [db_name] > $sFilename";

  - Backing Up a PostgreSQL Database
$sBackupCommand = "pg_dump [db_name] -h localhost -U [user_name] -d -O > $sFilename";

  - Backing Up a set of files (tar and gzip)
$sBackupCommand = "tar cvf $sFilename [directory]

  Whatever command you use, this script appends .gz to the filename after the command is executed.  */
$sBackupCommand = "[a backup command]";

// --- END OF SCRIPT PARAMETERS ---
//
// Edit below at your own risk.  :)

// Do the backup.
$sResult = passthru($sBackupCommand . "; gzip $sFilename");
$sFilename .= ".gz";

// Create the message.
$sMessage = "Compressed database backup file $sFilename attached.";
$sMimeBoundary = "<<<:" . md5(time());
$sData = chunk_split(base64_encode(implode("", file($sFilename))));

$sHeaders = "From: $sEmailAddress\r\n"
		. "MIME-Version: 1.0\r\n"
		. "Content-type: multipart/mixed;\r\n"
		. " boundary=\"$sMimeBoundary\"\r\n";

$sContent = "This is a multi-part message in MIME format.\r\n\r\n"
		. "--$sMimeBoundary\r\n"
		. "Content-Type: text/plain; charset=\"iso-8859-1\"\r\n"
		. "Content-Transfer-Encoding: 7bit\r\n\r\n"
		. $sMessage."\r\n"
		. "--$sMimeBoundary\r\n"
		. "Content-Disposition: attachment;\r\n"
		. "Content-Type: Application/Octet-Stream; name=\"$sFilename\"\r\n"
		. "Content-Transfer-Encoding: base64\r\n\r\n"
		. $sData."\r\n"
		. "--$sMimeBoundary\r\n";

// Send the message.
mail($sEmailAddress, $sEmailSubject, $sContent, $sHeaders);

// Delete the file - we don't need it any more.
unlink($sFilename);

While working on the Not So Extreme Makeover: Community Edition site, I came up with an algorithm that simplifies anything else I've ever written to deal with this condition. I'll set the scenario, explain the algorithm, share how I implemented it in PHP, and provide a modification if the scenario is a bit more complicated.

Scenario - You have two parent tables, and a child table with a many-to-one relationship with both parent tables, used to map entries in the two parent tables to each other. For this example, we'll use these three tables…

create table volunteer (
    vol_id  integer  not null,
    vol_last_name  varchar(50)  not null,
    ...etc...
  primary key (vol_id)
);

create table r_volunteer_area (
    rva_id  integer  not null,
    rva_description  varchar(255)  not null,
  primary key (rva_id)
);

create table volunteer_area (
    va_volunteer_id  integer  not null,
    va_area_id  integer  not null,
  primary key (va_volunteer_id, va_area_id),
  foreign key (va_volunteer_id) references volunteer (vol_id),
  foreign key (va_area_id) references r_volunteer_area (rva_id)
);

Algorithm - The three-step algorithm is as follows…

1. Create a comma-delimited string of IDs for the child table.
2. Delete the IDs from the child table that are not in the list.
3. Insert the IDs into the child table that are not there already.

Implementation - In PHP, if you have an array, it's easy to come up with comma-delimited list. To get an array of values back in a post, define your fields with “[]” after the name…

<input type="checkbox" name="area[]" id="chkArea1" value="1" />
<label for="chkArea1">Do Something</label><br />
<input type="checkbox" name="area[]" id="chkArea7" value="7" />
<label for="chkArea7">Do Something Else</label>

Here's the PHP code, using PHP Data Objects (PDO) as the database interface, behind a helper class that creates the statement, appends the parameters, and executes it. (The “quoting” escapes the statement to avoid potential SQL injection attacks - putting it in its own class would make the implementation here much cleaner.)

/**
 * STEP 1
 *    Create a comma-delimited list of IDs.
 */

// Quote will return the string as '2,3,4' - since we're using this
// as an IN clause of integers, we'll strip the quotes off.
$sAreas = $pdo->quote(join(",", $_POST["area"]));
$sAreas = substr($sAreas, 1, strlen($sAreas) - 1);

// Quote the volunteer ID.
$iVol = $pdo->quote($_POST["vol"], PDO::PARAM_INT);

/**
 * STEP 2
 *    Delete the IDs that are no longer in the list.
 */
$dbService->executeCommand(
    "DELETE FROM volunteer_area
    WHERE   va_volunteer_id = ?
        AND va_area_id NOT IN ($sAreas)",
    array($iVol);

/**
 * STEP 3
 *    Insert the IDs that are not yet in the list.
 */
$dbService->executeCommand(
    "INSERT INTO volunteer_area
        SELECT $iVol, rva_id
        FROM r_volunteer_area
        WHERE   rva_id IN ($sAreas)
            AND rva_id NOT IN
            (SELECT va_area_id
            FROM volunteer_area
            WHERE va_volunteer_id = ?)",
    array($iVol));

Modification - Suppose that now you accepted comments along with each of the checkboxes, so a simple two-integer insert/delete is no longer sufficient. You would still only need to break step 3 into two steps.

1. Get a list of IDs to update.
2. For each ID in the posted list
   1. If the ID exists in the update list, update it.
   2. Otherwise, insert it.

The implementation would then be able to use this list to make the decision without hitting the database every time.

// Assume this returns an associative array of IDs.
$aUpdates = $dbService->performSelect(
    "SELECT va_area_id
    FROM volunteer_area
    WHERE   va_volunteer_id = ?
        AND va_area_id IN ($sAreas)",
    array($iVol));

foreach($_POST["area"] as $iArea) {
    if (in_array($iArea, $aUpdates)) {
        // Update the table
        ...etc...
    }
    else {
        // Insert into the table
        ...etc...
    }
}

I think you'll agree that this is much better than spinning through a loop, doing a count on each ID to see if it exists, then either doing an update or an insert based on the count. And, while the implementation here is PHP, it could easily be implemented in any language that supports arrays and database access.