Mythos Image

I have done a lot of work on the jqOS project and the demo is mostly done and working. See it here. It now is integrated somewhat with Bootstrap and I converted the SCSS to LESS since it seems like that is what the majority of the industry is going with. It no longer uses Superfish for the menus, but instead the Bootstrap navigation controls. There are still plenty of bugs and plenty of unimplemented features but it has come a long way and I hope to soon be able to show you a release candidate and then get rolling on the Herp Hobbyists site.

This is one of the largest projects that I have ever developed and to get it up here to share is taking quite some time. See it as I go here. More documentation is coming soon here on this post. Consider the project as of now under GPL license, use it and contribute back to it. I am creating a Wiki on the project as well to aid in the communal development. It requires jQuery, jQuery UI, and SuperFish scripts to work properly.

Check back soon for more information

This article continues off of the last one for setting up a single WordPress install for multiple sites. Last time I went through the wp-config.php file. The next step is my newsite.php shell script. Just like last time, I am going to break up the shell script into pieces.

#!/usr/bin/php
<?php
$dev = "example.com"; # Development Domain
$maxlength = 6; # Maximum Client/Site ID Length
$wwwroot = "/path/to/client/sites"; # Base root for all client sites
$wpuser = "wordpress_user"; # MySQL user that WordPress uses
$dbauthdomain = "localhost"; # The authorized domain for WordPress MySQL account
$corepath = "/path/to/wordpress/"; # Path to WordPress core
$links = array(
"index.php",
"wp-blog-header.php",
"wp-cron.php",
"wp-mail.php",
"license.txt",
"wp-comments-post.php",
"wp-includes",
"wp-settings.php",
"readme.html",
"wp-config.php",
"wp-links-opml.php",
"wp-signup.php",
"wp-activate.php",
"wp-load.php",
"wp-trackback.php",
"wp-admin",
"wp-login.php",
"xmlrpc.php",
"wp-content/index.php",
"wp-content/plugins",
"wp-content/themes/twentytwelve"); # Array of files to create symbolic links for

This is all the variables for the script and are subjected to customization depending on the particular server environment. I think most of the variables are self-explanatory, but if I am wrong, feel free to put questions in the comments section below. There is one that I would like to highlight, the $dev variable. This is to be set on the domain name that you use for your development environment. Some people like to just have it as a subdomain of the main site (e.g. dev.livesite.com), but this was developed for a company that previously had their development environment off of a disorganized directory structure off of a single domain (e.g. previewsite.com/long/confusing/path/to/site) and I was making it to where each customer’s site is a subdomain of the development domain (e.g. client-site.previewsite.com). After setting it up, I liked this structure so much that I started using it for my personal projects. It allows me to start working before domain propagation, DNS propagation, and before the client gives me live access.

# -- Get input until unique ones are entered -->
do {
# -- Get client ID until a vaild one is entered -->
do {
fwrite(STDOUT, "Enter the client id: ");
$c = trim(fgets (STDIN));
if (strlen($c) >= $maxlength || preg_replace("/[a-z0-9]/", "", $c))
fwrite(STDOUT, "Error: Must be ".$maxlength." characters or less, only lowercase and numeric.\n");
} while (strlen($c) >= $maxlength || preg_replace("/[a-z0-9]/", "", $c));
# -- Get site ID until a valid one is entered -->
do {
fwrite (STDOUT, "Enter the site id: ");
$s = trim (fgets(STDIN));
if (strlen($s) >= $maxlength || preg_replace("/[a-z0-9]/", "", $s))
fwrite(STDOUT, "Error: Must be ".$maxlength." characters or less, only lowercase and numeric.\n");
} while (strlen($s) >= $maxlength || preg_replace("/[a-z0-9]/", "", $s));
if (file_exists("/var/www/clients/".substr($c, 0, 1)."/".$c."/".$s."/dev"))
fwrite(STDOUT, "Error: Site ".$c."-".$s.".".$dev." already exists.\n");
} while (file_exists("/var/www/clients/".substr($c, 0, 1)."/".$c."/".$s."/dev"));

This is getting the input from the user of the application to identify the new site. It works off of a client ID and a site ID and makes sure that they are both valid and not currently existing. I added a six character limit after getting ID’s from the Vice President that were close to 20 characters in length. No one wants to try to direct the customer over the phone to the development site at verylongclientid-evenlongersiteidblahblahblah.example.com as opposed to jsmith-store.example.com. So I told the VP that it would not allow more than six characters and refused to admit that it was a self-imposed limitation.

# -- Connect to registry and add new entry in site registry -->
require "regconnect.php";
mysql_query("
INSERT INTO
apache_vhosts (
host,
path,
virt,
spiders,
dbschema)
VALUES (
'".$c."-".$s.".".$dev."',
'clients/".substr($c, 0, 1)."/".$c."/".$s."/dev',
1,
0,
'cl_".$c."_".$s."_dev');") or die("Failure in INSERT query.\n");
# -- Create site's schema and give WordPress access -->
mysql_query("
CREATE DATABASE
cl_".$c."_".$s."_dev;") or die("Failure in CREATE query.\n");
mysql_query("
GRANT
SELECT,
INSERT,
UPDATE,
DELETE,
CREATE,
DROP,
INDEX,
ALTER
ON
cl_".$c."_".$s."_dev.*
TO
'".$wpuser."'@'".$dbauthdomain."';") or die("Failure in GRANT query.\n");

This connects into the database, creates the development database, and grant the proper permissions to the WordPress user account on this new database.

# -- Create directories and symbolic links for new WordPress site -->
$hostpath = "/var/www/clients/".substr($c, 0, 1)."/".$c."/".$s."/dev/";
$hostpathx = explode("/", $hostpath);
for ($i = 0; $i < count($hostpathx); $i++) { $curdir = ""; for ($j = $i; $j > -1; $j--) $curdir = $hostpathx[$j]."/".$curdir;
if (!file_exists($curdir)) shell_exec("mkdir ".$curdir);
}
shell_exec("mkdir ".$hostpath."/wp-content");
shell_exec("mkdir ".$hostpath."/wp-content/themes");
foreach ($links as $link) shell_exec("ln -s ".$corepath.$link." ".$hostpath.$link);
mysql_close();
fwrite(STDOUT, "WordPress install complete at ".$hostpath."\n");
?>

To end, I cycle through creating directories and symbolic links to the WordPress core. I end it all with the message telling the user it was successful. After running this script, you can then go to the development site address and see the typical initial WordPress page. Note: the initial user information entered into this page doesn’t matter because with the wp-config.php file, it will get deleted and replaced with the global users since it is in the reserved block for the global users.

The next file is launchsite.php. This is the shell script for launching the site from development to live.

#!/usr/bin/php
<?php
$dev = "example.com"; # Development Domain
$maxlength = 6; # Maximum Client/Site ID Length
$wwwroot = "/path/to/client/sites/"; # Base root for all client sites
$dblocation = "localhost"; # IP or domain name for the MySQL server
$wpuser = "wordpress_user"; # MySQL user that WordPress uses
$dbauthdomain = "localhost"; # The authorized domain for WordPress MySQL account
$wppass = "password"; # MySQL password that WordPress uses

Just like the last script, I start with the variables to customize it for the particular server environment that it is in.

# -- Get input until existing ones are entered -->
do {
# -- Get client ID until a valid one is entered -->
do {
fwrite(STDOUT, "Enter the client id: ");
$c = trim(fgets (STDIN));
if (strlen($c) > $maxlength || preg_replace("/[a-z0-9]/", "", $c))
fwrite(STDOUT, "Error: Must be ".$maxlength." characters or less, only lowercase and numeric.\n");
} while (strlen ($c) > $maxlength || preg_replace("/[a-z0-9]/", "", $c));
# -- Get site ID until a valid one is entered -->
do {
fwrite(STDOUT, "Enter the site id: ");
$s = trim(fgets(STDIN));
if (strlen($s) > $maxlength || preg_replace("/[a-z0-9]/", "", $s))
fwrite(STDOUT, "Error: Must be ".$maxlength." characters or less, only lowercase and numeric.\n");
} while (strlen($s) > $maxlength || preg_replace("/[a-z0-9]/", "", $s));
if (!file_exists($wwwroot.substr($c, 0, 1)."/".$c."/".$s."/dev/"))
fwrite(STDOUT, "Error: Cannot Locate Site for ".$c."-".$s.".".$dev."\n");
} while (!file_exists($wwwroot.substr($c, 0, 1)."/".$c."/".$s."/dev/"));
fwrite(STDOUT, "Enter the live domain name: http://");
$livehost = trim(fgets(STDIN));
$devdb = "cl_".$c."_".$s."_dev";
$livedb = "cl_".$c."_".$s."_live";
$devpath = $wwwroot.substr($c,0,1)."/".$c."/".$s."/dev";
$livepath = $wwwroot.substr($c,0,1)."/".$c."/".$s."/live";
$devhost = $c."-".$s.".".$dev;

Just like with the last script, I use do {} while (); loops to get proper input from the user to figure out which site it is. I didn’t put any validation on the live domain name since about the only thing I could catch was invalid characters and I figured most mistakes would be typos that would have to come to me for manual fixing in the Site Registry. After all the input was collected, I created variables for the environment.

require "regconnect.php";
mysql_query("
DELETE FROM
apache_vhosts
WHERE
host='".$livehost."';");
mysql_query("
INSERT INTO
apache_vhosts (
host,
path,
virt,
dbschema)
VALUES (
'".$livehost."',
'".$livepath."',
1,
'".$livedb."');") or die("Failure in INSERT (registry) query.\n");
mysql_query("
CREATE DATABASE IF NOT EXISTS
".$livedb.";") or die("Failure in CREATE DATABASE query.\n");
mysql_query("
GRANT
SELECT,
INSERT,
UPDATE,
DELETE,
CREATE,
DROP,
INDEX,
ALTER
ON
".$livedb.".*
TO
'".$wpuser."'@'".$dbauthdomain."';") or die("Failure in GRANT query.\n");
mysql_close();

This is the database part that works off of the site registry MySQL account. It inserts the live domain information into the site registry, creates the live site database, and grants access to the new database to the WordPress database account.

mysql_connect($dblocation, $wpuser, $wppass);
mysql_select_db($devdb);
$result = mysql_query("SHOW TABLES;");
$tables = array();
while ($row = mysql_fetch_array($result)) $tables[] = $row[0];
mysql_select_db($livedb);
for ($i = 0; $i < count($tables); $i++) { mysql_query(" DROP TABLE IF EXISTS ".$livedb.".".$tables[$i].";"); mysql_query(" CREATE TABLE ".$livedb.".".$tables[$i]." LIKE ".$devdb.".".$tables[$i].";") or die("Failure in CREATE TABLE query.\n"); mysql_query(" INSERT INTO ".$livedb.".".$tables[$i]." SELECT * FROM ".$devdb.".".$tables[$i].";") or die("Failure in INSERT (site) query.\n"); }

Now we connect with the WordPress user account and duplicate the database from development. It creates an array of all tables with the SHOW TABLES query. Then it cycles through the array creating the identical tables and inserts the identical data.

mysql_query("
UPDATE
".$c."_options
SET
option_value='http://".$livehost."'
WHERE
option_value='http://".$devhost."';") or
fwrite(STDOUT, "Error in updating domain name in live database.\n");
mysql_query("
UPDATE
".$c."_options
SET
option_value='1'
WHERE
option_name='blog_public';") or
fwrite(STDOUT, "Error in updating to public site in live database.\n");
if (!file_exists($livepath)) shell_exec("mkdir ".$livepath);
shell_exec("rsync -av ".$devpath."/ ".$livepath);
shell_exec("chgrp -R users ".$livepath);
shell_exec("chmod -R 775 ".$livepath);
fwrite(STDOUT, "WordPress Site has been launched for http://".$livehost.".\n");
?>

To finish this off, we fix the few WordPress options that need to change like the domain name and whether or not it is a public site. After that, we do an rsync of the file system and make sure the permissions are set properly.

The last script is updatedevsite.php and it is very similar to the last one. It updates the development site from the live data. This way all the customer changes to the content would not be lost when we ended up doing further development or designing on their site after launch. I am not going to go step by step since it really is the same as the last one, but in reverse direction copying live to dev instead of dev to live.

#!/usr/bin/php
<?php
$dev = "example.com"; # Development Domain
$maxlength = 6; # Maximum Client/Site ID Length
$wwwroot = "/path/to/client/sites/"; # Base root for all client sites
$dblocation = "localhost"; # IP or domain name for the MySQL server
$wpuser = "wordpress_user"; # MySQL user that WordPress uses
$dbauthdomain = "localhost"; # The authorized domain for WordPress MySQL account
$wppass = "password"; # MySQL password that WordPress uses
# -- Get input until existing ones are entered -->
do {
# -- Get client ID until a valid one is entered -->
do {
fwrite(STDOUT, "Enter the client id: ");
$c = trim(fgets (STDIN));
if (strlen($c) > $maxlength || preg_replace("/[a-z0-9]/", "", $c))
fwrite(STDOUT, "Error: Must be ".$maxlength." characters or less, only lowercase and numeric.\n");
} while (strlen ($c) > $maxlength || preg_replace("/[a-z0-9]/", "", $c));
# -- Get site ID until a valid one is entered -->
do {
fwrite(STDOUT, "Enter the site id: ");
$s = trim(fgets(STDIN));
if (strlen($s) > $maxlength || preg_replace("/[a-z0-9]/", "", $s))
fwrite(STDOUT, "Error: Must be ".$maxlength." characters or less, only lowercase and numeric.\n");
} while (strlen($s) > $maxlength || preg_replace("/[a-z0-9]/", "", $s));
if (!file_exists($wwwroot.substr($c, 0, 1)."/".$c."/".$s."/live/"))
fwrite(STDOUT, "Error: Cannot Locate Site for ".$c."-".$s."."\n");
} while (!file_exists($wwwroot.substr($c, 0, 1)."/".$c."/".$s."/live/"));
$devdb = "cl_".$c."_".$s."_dev";
$livedb = "cl_".$c."_".$s."_live";
$devpath = $wwwroot.substr($c,0,1)."/".$c."/".$s."/dev";
$livepath = $wwwroot.substr($c,0,1)."/".$c."/".$s."/live";
$devhost = $c."-".$s.".".$dev;
require "regconnect.php";
if ($r = mysql_query("
SELECT
host
FROM
apache_vhosts
WHERE
path='".$livepath."';")) $livehost = mysql_fetch_array($r)[0];
mysql_close();
mysql_connect("localhost", $wpuser, $wppass);
mysql_select_db($livedb);
$result = mysql_query("SHOW TABLES;");
$tables = array();
while ($row = mysql_fetch_array($result)) $tables[] = $row[0];
mysql_select_db($devdb);
for ($i = 0; $i < count($tables); $i++) { mysql_query(" DROP TABLE IF EXISTS ".$devdb.".".$tables[$i].";"); mysql_query(" CREATE TABLE ".$devdb.".".$tables[$i]." LIKE ".$livedb.".".$tables[$i].";") or die("Failure in CREATE TABLE query.\n"); mysql_query(" INSERT INTO ".$devdb.".".$tables[$i]." SELECT * FROM ".$livedb.".".$tables[$i].";") or die("Failure in INSERT query.\n"); } mysql_query(" UPDATE ".$c."_options SET option_value='http://".$devhost."' WHERE option_value='http://".$livehost."';") or fwrite(STDOUT, "Error in updating domain name in dev database.\n"); mysql_query(" UPDATE ".$c."_options SET option_value='0' WHERE option_name='blog_public';") or fwrite(STDOUT, "Error in updating to private site in dev database.\n"); shell_exec("rsync -av ".$livepath."/ ".$devpath); shell_exec("chgrp -R users ".$devpath); shell_exec("chmod -R 775 ".$devpath); fwrite(STDOUT, "WordPress Site has been launched for http://".$devhost.".\n"); ?>

That’s it. Scripts for creating new development site, launching to live, and updating from live. I ended up teaching my designers to use PuTTY and run these themselves. Ultimately though, I was wanting to integrate into the system where on the development site, there was buttons to launch and update from live that asked for username and password and did it for them right then and there. I also had an extra call in these script that I left out here, and that was where it logged the launches and updates in our database for tracking work. In our weekly meetings, these logs were used to track the progress on projects and save us time in discussing it.

I was the IT Director for an online marketing company and the CEO one day told me that we were going to now start using WordPress as the CMS for all our sites. I don’t mind WordPress, in fact this and many of the other sites that I have made are in WordPress. This particularly annoyed me because I had spent the past month or so building a custom CMS and engineering the server around this CMS which was now essentially going to be wadded up and thrown in the trash. One of the biggest reasons I did not want to got the WordPress route is because I was building the server around a centralized code base as opposed to a distributed one like in WordPress. I figured that this was something I could get around and make a single WordPress install stretch to be used for all of our sites. This strategy pays for itself when it comes time to upgrade the WordPress core or any well used plugin and also for managing users when an employee with access to nearly all the customer’s sites leaves the company.

At the core, I installed one WordPress, created a versatile wp-config.php file and some shell scripts to create new sites, launch from development to production, and update development from live (needed for when the customers made their changes). This was a long process to setup and get the bugs worked out, but once it was setup, I liked it enough to do it on my own personal server. First, I am going to start with the wp-config.php file, but I am going to break it up into sections to explain each part

<?php
require_once "prepend.php";

First thing here is I included a prepend.php, which is an optional line of code for the grand scheme of things, but it can be very helpful. I used it to create all kinds of custom variables, objects, and methods that made my life as a developer so much easier because I knew that in all the sites on this server had them included because of that first line of code.

require_once "regconnect.php";
$result = mysql_query("
SELECT
dbschema
FROM
apache_vhosts
WHERE
host='".DOMAIN."'
LIMIT 1;");
if ($schema = mysql_fetch_array($result)) $schema = $schema[0];
else {
include "error.php";
exit();
}
mysql_close();

If you read my previous articles about dynamic virtual hosting, this part will make a lot more sense. In this server environment, I had a MySQL database table which acted essentially as the site registry. I, in fact, called it the Site Registry which is why I had a file regconnect.php that I included whenever I wanted to connect to the registry. So in this portion of the code, I connect to the site registry and figure out the database for this site based on the domain name used. The DOMAIN constant is actually defined in my prepend.php, but it is essentially interchangeable with $_SERVER[‘HTTP_HOST’]. I just used it to force all lower case and strip “www.” off the front of the host name. Once it finds the correct database, it sets it to the $schema variable, or if it fails to find the right database, it pulls in the error page and terminates the application. Side Note: all these includes and requires leveraged on me customizing the PHP path configuration so it knew where to find my global scripts

define('DB_NAME', $schema);
define('DB_USER', 'wordpress_user');
define('DB_PASSWORD', 'password');
define('DB_HOST', 'localhost');
$table_prefix = explode("_", $schema);
$table_prefix = $table_prefix[1]."_";
define('DB_CHARSET', 'utf8');
define('DB_COLLATE', '');
define('WPLANG', '');
define('ABSPATH', dirname(__FILE__));
define('WP_CONTENT_DIR', DOCROOT.'wp-content');
require_once ABSPATH.'wp-settings.php';

This part is the familiar as it is the part that you already have to do for any wp-config.php file sans all their comments. The biggest notation to make here was that I decided to go with a universal wordpress DB user and this was something that a came about out of protest. Originally, I stored a username and password for each site in the site registry, but when it came to writing the shell scripts for updating between development and production, I ended up going this route for simplicity sake. I know that there were numerous ways to get it working with each site having its own DB user, but with GRANT queries, it became a very convoluted process when trying to manage separate users and ultimately if a user wanted to pull their site from our hosting and host it elsewhere, I already had to write a different wp-config.php, so the ultimately the arguments for individual users started dwindling in lieu of simplicity.

if (PAGE == "wp-login.php") {
require "regconnect.php";
$result = mysql_query("
SELECT
*
FROM
wp_global_users;");
$insertusers = "
INSERT INTO
".$table_prefix."users VALUES (";
while ($user = mysql_fetch_assoc($result)) {
foreach ($user as $value) $insertusers .= "
'".$value."', ";
$insertusers = substr($insertusers, 0, -2)."), (";
}
$insertusers = substr($insertusers, 0, -3).";";
$result = mysql_query("
SELECT
*
FROM
wp_global_usermeta;");
$insertusermeta = "
INSERT INTO
".$table_prefix."usermeta
VALUES (";
while ($usermeta = mysql_fetch_assoc($result)) {
foreach ($usermeta as $value) $insertusermeta .= "
'".preg_replace("/^wp_/", $table_prefix, $value)."', ";
$insertusermeta = substr($insertusermeta, 0, -2)."), (";
}
$insertusermeta = substr($insertusermeta, 0, -3).";";
mysql_close();
mysql_connect(DB_HOST, DB_USER, DB_PASSWORD);
mysql_select_db(DB_NAME);
mysql_query("
DELETE FROM
".$table_prefix."users
WHERE
ID < 1001;"); mysql_query($insertusers); mysql_query(" DELETE FROM ".$table_prefix."usermeta WHERE user_id < 1001;"); mysql_query($insertusermeta); mysql_close(); } ?>

This was probably the part that I liked the most about this strategy of using a single WordPress install. I managed central user tables in the same schema as my site registry for all global WordPress users. This way I could give my writers and my designers the right permissions and manage them centrally so that I could easily remove a user if they left the company (which that company was notorious about rapid turnovers mostly due to belligerent management). Now, the only problem was that the WordPress users work off of an auto-incrementing ID so the best way that I could do this was to create a reserve block at the lower numbers for the global users demarcated by a dummy account that I simply named "reserved" and gave it no real permissions (Subscriber). I assigned it an ID of 1000. Once this dummy account gets duplicated to the individual sites’s WordPress user table, then all site local accounts will automatically start from ID 1001.

The CEO of the company didn’t want the customer to see our accounts, so I also edited a file in wp-admin to only display the accounts which ID’s where greater than 1000. I didn’t like that solution as it was a change that I would have to redo with every update to the WordPress core, but as it was a one line change, it was not worth it to me to build a more automated way of doing it. Ultimately, I have no problems showing all the accounts in there and just explaining to the customers that those are our accounts. After all, even if they try to delete or edit the accounts, they just go back to the global settings every time anyone logs in.

Well, now that we have the wp-config.php file down, the next step would be my script for creating a new site which I will save for another post.

Have you or someone on your team ever accidentally launched the “don’t spider me” robots.txt file from the development environment into production and didn’t notice it until the search engine optimization nose dived? The first has happened to me, but luckily not the latter. When the file was launched live, it was noticed rather quickly by our SEO guru whose background programs stopped working on that site because of the accidental change. After a scare like that, I decided on the new server that I was setting up, I’ll just set it up to where I have two master robots.txt files, one for production and one for development, and just have Apache figure out which one to serve up based upon the request. Just like in my dynamic virtual hosting, I will accomplish this through Apache’s Rewrite Module piped through a PHP Script.

First, the PHP Script, which I named robots.php:

#!/usr/bin/php
<?php
$devDomain = "example"; // This is the domain name for the development environment
set_time_limit (0);
$input = fopen('php://stdin', 'r');
$output = fopen('php://stdout', 'w');
while (1) {
$original = strtolower(trim(fgets($input)));
$request = array_reverse(explode('.', preg_replace("/^www\./", "", $original)));
if ($request[1] == $devDomain) fwrite($output, "/path/to/robots.hidden.txt\n");
else {
require "regconnect.php";
$r = mysql_query("
SELECT
`spiders`
FROM
`apache_vhosts`
WHERE
`host`='".implode (".", array_reverse($request))."'
LIMIT 1;");
if ($robot = mysql_fetch_array($r)) {
if (!$robot[0]) fwrite($output, "/path/to/robots.hidden.txt\n");
else fwrite($output, "/path/to/robots.visible.txt\n");
} else fwrite($output, "/path/to/robots.visible.txt\n");
mysql_close();
}
}
?>

So with this script, it first checks the domain name closest to the TLD (e.g. for subdomain.example.com, it extract example). It checks to see if this is the domain name used in the development environment and if so, it serves up the “don’t spider me” file, robots.hidden.txt. If it is not in the development environment, then it queries the same vhosts table that I used in the dynamic virtual hosting, but this time leverages from a new column, “spiders.” This is a simple bit/bool to tell whether or not the site is set to be spidered. The default on this column is 1/true and since nearly all of the live sites had this set to 1/true so in any case where there may be a problem querying the database, it would serve up robots.visible.txt by default.

Now comes the part that’s always harder to read because of the regular expression involved, the Apache Rewrite. This one is a lot easier than the dynamic virtual hosts and you can just add it in to the same <IfModule/> and leave out the duplicate lines if you are doing them both at the same time (which I was).

<IfModule mod_rewrite.c>
RewriteEngine On
RewriteLock /tmp/httpd.lock
RewriteMap robots prg:/path/to/robots.php
RewriteCond %{REQUEST_URI} ^/robots\.txt$
RewriteRule ^/(.*)$ ${robots:%{HTTP_HOST}} [L]
</IfModule>

Real simple one here, if the request is for robots.txt pipe the HTTP_HOST to the PHP Script and it will output back to Apache the path to the proper robots.txt file for the site. Then it was as simple as having two robots.txt files that could be used for everything and no need to worry about the designers accidentally launching the development robots.txt to production.

Years ago I was responsible for setting up a Linux server for a company that did web design and marketing. I was the only person in this company that knew how to add additional virtual hosts into the Apache configurations. While I could have taught the employees how to do it, I really did not want to allow the Apache configuration files to be editable by people that really didn’t know what they were doing. I decided instead to pipe the configuration out to some shell scripts written in PHP that would query the local MySQL database to do virtual host look up.

I found very little documentation and example coding to help me do this, and so a lot of this came from trial and error. I’ll start with the the PHP-based shell script which I named docroots.php:

#!/usr/bin/php
<?php
$dev = "example"; # Development Domain, sans the TLD
$wwwroot = "/path/to/client/sites/"; # Base root for all client sites
$error = "/path/to/error/site"; # Base root for error page
set_time_limit(0);
$input = fopen('php://stdin', 'r');
$output = fopen('php://stdout', 'w');
while (1) {
$original = fgets($input);
$clean = strtolower(trim(preg_replace("/^www\./", "", $original)));
/* Create an array on the DNS request, e.g.
[0] = "com",
[1] = "example",
[2] = "subdomain"
for subdomain.example.com */
$request = array_reverse(explode('.', $clean));
if ($request[1] == $dev) { // If the request is in dev, use the subdomain info to deduce document root
/* Create an array on the subdomain requested, e.g.
[0] = "jsmith", (Client Name)
[1] = "store", (Site Name)
[2] = "3" (Design Choice)
for jsmith-store-3.example.com */
$client = explode ("-", $request[2]);
/* Server file organization is
/path/to/client/sites/j/jsmith/store/d3/
for the third design choice for John Smith's Online Store */
if (isset ($client[2]))
$docroot = $wwwroot.substr($client[0], 0, 1)."/".$client[0]."/".$client[1]."/d".$client[2]."/";
/* Once a design choice has been made, the design number drops off
jsmith-store.example.com
will direct Apache to a document root of
/path/to/client/sites/j/jsmith/store/dev/ */
else $docroot = $wwwroot.substr($client[0], 0, 1)."/".$client[0]."/".$client[1]."/dev/";
} else { // If the request is for the live environment, use MySQL to get the document root
require "regconnect.php"; // External file for database connection
$r = mysql_query("
SELECT
path
FROM
apache_vhosts
WHERE
host='".$clean."'
LIMIT 1;");
if ($data = mysql_fetch_array($r))
$docroot = $data[0];
else $docroot = $error; // If site not found, send to error site
mysql_close();
}
if (file_exists($docroot)) fwrite($output, $docroot . "\n");
else fwrite($output, $error . "\n");
}
?>

So to summarize this file, it predicted document roots for development sites and looked up document roots on live sites. I typically stored development environments in this manner:

/path/to/client/sites/j/jsmith/store/dev/

Then I would typically make an entry in MySQL for the live sites like this:

host	path
smithexamplestore.com	/path/to/client/sites/j/jsmith/store/live/

I had other columns in this table, but those are the only ones that matter for this experiment. This registered the live domain name to the proper client and site identifiers on the server. The script would then know to look for the live site’s document root at:

/path/to/client/sites/j/jsmith/store/live/

I also logged the development sites in the registry, but there was no need to look them up because it was well organized and could be deduced by the request. The required file was a very basic connect and select database script. I decided upon the procedural MySQL usage instead of the MySQLi object since I was just doing a single query then disconnecting. The regconnect.php file was simply:

<?php
mysql_connect("localhost", "registry", "password");
mysql_select_db("registry");
?>

Obviously, my the specifics were not as simplistic as these. Like all accounts that are only used by programs and services, I generated a long and complex user names and passwords that I would never remember, but was stored here in this file that was only accessible by root and the proper services, hidden away, but whose path was in the PHP configurations so it could be called easily without the path defined. Yes, a smart employee could figure it out, but they typically had access with their database accounts and the people I had on staff knew so little about *NIX that I doubt any would be able to find it.

The next thing to do is change the Apache configuration to use this script for server requests. I leveraged Apache’s Rewrite Module to accomplish this. I first create a rewrite map that is piped to my PHP script, then as long as it is not some of my global directories (for error pages or for site statistics) or robots.txt (I’ll explain how I handled this globally in another post), I piped the HTTP_HOST to the PHP Script to which its output would be the document root.

<IfModule mod_rewrite.c>
RewriteEngine On
RewriteLock /tmp/httpd.lock
RewriteMap docroots prg:/path/to/docroots.php
RewriteCond %{REQUEST_URI} !^/error
RewriteCond %{REQUEST_URI} !^/stats
RewriteCond %{REQUEST_URI} !^/robots\.txt$
RewriteRule ^/(.*)$ ${docroots:%{HTTP_HOST}}$1
</IfModule>

This shell script worked great when I piped out the Apache configurations to it, but I did start noticing some glitches in high traffic times and was going to add some short life caching. Since every request for every resource was another database hit, but if I cached it for just a few seconds, then it would do the database hit once and all the subsequent resource requests would get satisfied by the cache. However, as I was always vary overwhelmed with more work than humanly possible to finish at that job, I never quite got around to adding it in and I never heard of any complaints from the clients.

I called this trick “dynamic virtual hosting” and it works great for referencing clients and sites with a clear organization pattern and allows for people not too good with Apache to manage virtual hosts with no need to even soft reboot the daemon.

Update

I have added caching to the docroots.php file. This is using the APC caching extension, which you must install with pecl and then you need to use the #!/usr/bin/php-cgi script parser, without the “-cgi” at the end and it will not work (I just spent the last two hours about to pull my hair out trying to figure out why the caching wasn’t working before trying this and everything started working). I also changed a couple other minor details that do not really matter to the execution of the code, just thought it read better. See updated code below:

#!/usr/bin/php-cgi
<?php
$dev = "example"; # Development Domain, sans the TLD
$wwwroot = "/path/to/client/sites/"; # Base root for all client sites
$error = "/path/to/error/site"; # Base root for error page
$cachettl = 300; # Cache time in seconds
set_time_limit(0);
$input = fopen('php://stdin', 'r');
$output = fopen('php://stdout', 'w');
while (1) {
$original = fgets($input);
$clean = strtolower(trim(preg_replace("/^www\./", "", $original)));
/* Create an array on the DNS request, e.g.
[0] = "com",
[1] = "example",
[2] = "subdomain"
for subdomain.example.com */
$request = array_reverse(explode('.', $clean));
if ($request[1] == $dev) { // If the request is in dev, use the subdomain info to deduce document root
/* Create an array on the subdomain requested, e.g.
[0] = "jsmith", (Client Name)
[1] = "store", (Site Name)
[2] = "3" (Design Choice)
for jsmith-store-3.example.com */
$client = explode ("-", $request[2]);
/* Server file organization is
/path/to/client/sites/j/jsmith/store/d3/
for the third design choice for John Smith's Online Store */
if (isset ($client[2]))
$docroot = $wwwroot.substr($client[0], 0, 1)."/".$client[0]."/".$client[1]."/d".$client[2]."/";
/* Once a design choice has been made, the design number drops off
jsmith-store.example.com
will direct Apache to a document root of
/path/to/client/sites/j/jsmith/store/dev/ */
else $docroot = $wwwroot.substr($client[0], 0, 1)."/".$client[0]."/".$client[1]."/dev/";
} else { // If the request is for the live environment, use Cache or MySQL to get the document root
if (apc_exists($clean)) $docroot = apc_fetch($clean);
else {
require "regconnect.php"; // External file for database connection
$r = mysql_query("
SELECT
path
FROM
apache_vhosts
WHERE
host='".$clean."'
LIMIT 1;");
if ($data = mysql_fetch_array($r)) {
$docroot = $data[0];
if (file_exists($docroot)) apc_add($clean, $docroot, $cachettl);
else $docroot = $error; // If site not found in file system, send to error site
} else $docroot = $error; // If site not found in database, send to error site
mysql_close();
}
}
fwrite($output, $docroot . "\n");
}
?>

If you do not use APC, you can also use an array since this is a looped script that Apache keeps running ad infinitum, but with this method, there is no time to live on the cached data and so any update you make to the registry would then have to be coupled with an Apache restart which sort of defeats one of the main reasons to go this route. However, it does still add the simplicity that virtual hosting can be managed by users that have no access to the Apache configurations and you can always setup soft reboots as a CRON job to establish a TTL. I would not advise going this route for efficiency sake, but it may still be better than repeating database hits for hosts already looked up fractions of a second earlier.

#!/usr/bin/php
<?php
$dev = "example"; # Development Domain, sans the TLD
$wwwroot = "/path/to/client/sites/"; # Base root for all client sites
$error = "/path/to/error/site"; # Base root for error page
$roots = array(); # Array to cache lookups
set_time_limit(0);
$input = fopen('php://stdin', 'r');
$output = fopen('php://stdout', 'w');
while (1) {
$original = fgets($input);
$clean = strtolower(trim(preg_replace("/^www\./", "", $original)));
/* Create an array on the DNS request, e.g.
[0] = "com",
[1] = "example",
[2] = "subdomain"
for subdomain.example.com */
$request = array_reverse(explode('.', $clean));
if ($request[1] == $dev) { // If the request is in dev, use the subdomain info to deduce document root
/* Create an array on the subdomain requested, e.g.
[0] = "jsmith", (Client Name)
[1] = "store", (Site Name)
[2] = "3" (Design Choice)
for jsmith-store-3.example.com */
$client = explode ("-", $request[2]);
/* Server file organization is
/path/to/client/sites/j/jsmith/store/d3/
for the third design choice for John Smith's Online Store */
if (isset ($client[2]))
$docroot = $wwwroot.substr($client[0], 0, 1)."/".$client[0]."/".$client[1]."/d".$client[2]."/";
/* Once a design choice has been made, the design number drops off
jsmith-store.example.com
will direct Apache to a document root of
/path/to/client/sites/j/jsmith/store/dev/ */
else $docroot = $wwwroot.substr($client[0], 0, 1)."/".$client[0]."/".$client[1]."/dev/";
} else { // If the request is for the live environment, use Cache or MySQL to get the document root
if (array_key_exists($clean, $roots)) $docroot = $roots[$clean];
else {
require "regconnect.php"; // External file for database connection
$r = mysql_query("
SELECT
path
FROM
apache_vhosts
WHERE
host='".$clean."'
LIMIT 1;");
if ($data = mysql_fetch_array($r)) {
$docroot = $data[0];
if (file_exists($docroot)) $roots[$clean] = $docroot;
else $docroot = $error; // If site not found in file system, send to error site
} else $docroot = $error; // If site not found in database, send to error site
mysql_close();
}
}
fwrite($output, $docroot . "\n");
}
?>

I was trying this method when I was frustrated over the fact that APC was not working, however for this method to really be successful, multi-threading was the best way. When the array value was made, open up a thread that sleeps for the TTL and then unsets the value from the array. Unfortunately, multi-threading in PHP also comes from external extensions so I would be faced with the same issue as APC, so ultimately figuring out the APC solution was the easier route. If extensions are really not an option, the array could be multi-dimensional where another value is set off an equation from the server’s clock to signify an expiration date on the value. Then when it finds the key in the array, if the expiration date has passed, it unsets it and does the MySQL look-up again. However, since I got the APC method to work, I didn’t bother with writing out the code for this solution, however if you need it, I have brought you most of the way there and given you the logic to take it over the line with just a slight bit more coding. If someone takes the time to code this solution, leave it in the comments below and help the next person along.