Showing posts with label apache. Show all posts
Showing posts with label apache. Show all posts

Wednesday, January 26, 2011

Simple URL Shortening Script

If you're running Apache and are able to use the mod_asis module, it's very easy to setup a simple and efficient URL shortener. This URL shortening script doesn't require any sort of database and it doesn't have the overhead of something like PHP added to every single redirection.

If you read the documentation for mod_asis, the fundamentals of how this all works may be immediately obvious. If not, basically mod_asis lets you have a sort of static response cache. A lot like having a static HTML cache, but with HTTP headers.

Assuming you have your URL shortening domain setup and you have a nice empty DocumentRoot for your new URL shortening service, the first thing you need to do is create a directory named "stubs". If you want to setup some sort of UI to create new shortened URLs with setup your permissions on stubs so that PHP can write to stubs, but nobody else can.

Within the stubs directory create an htaccess file (ideally you'd use Directory or Location containers in your VirtualHost, but for example sake I'm using htaccess).
Within that htaccess file, add the fillowing line.

SetHandler send-as-is


What that line does, is force every requested file to pass through mod_asis, which if you've read the documentation for mod_asis you know that other than adding a Date and Server header to the response, mod_asis just sends the file to the visitor as-is.

So, if I put a file in that directory named "abc123" with the following contents, I'll basically be given a 302 redirect from the server pointing me to google.com

NOTE: There are two newlines after the Location line to signal the end of HTTP headers. This is important

Status: 302
Location: http://www.google.com/



So basically, at this point I could distribule the shortened URL "http://domain/stubs/abc123" and it would redirect visitors to google.com; This isn't all that nice though, as I have "stubs" in the URL and that sort of defeats the purpose of a shortened URL.

That's why in the parent directory of stubs, AKA the DocumentRoot, I add the following to the htaccess file.

RewriteEngine on
RewriteBase /
RewriteRule ([a-f\d]{1,8})$ stubs/$1 [L]


Now I can distribute my shortened URL as "http://domain/abc123" and it will redirect to google.com; I could still distribute the URL with stubs in it if I wanted and it would still work for both shortened URLs.

At this point I have a small, simple, and efficient URL shortener. I have to manually go into my stubs directory and add a new file every time I want to shorten a URL though.

For that, I have the following simple PHP script with a bulk shortened URL capable UI.

<!doctype html>  
<html lang="en">
<head>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1">
<title>My URL Shortener</title>
</head>

<body>
<div id="container">
<header>
<h1>My URL Shortener</h1>
<p>Enter as many URLs as you want, one per line.</p>
</header>
<div id="main">
<?php
/*
Contains backwards compatibility code;
If you comment this include() out and don't get any errors, it's safe to leave it out.
*/
include('./lib.php');

if( ! empty($_POST['u']))
{
$url_list = '<ul>';
foreach(preg_split('#[\r\n\f]+#', $_POST['u'], -1, PREG_SPLIT_NO_EMPTY) as $_url)
{
$url = parse_url($_url);
if($url && http_build_url('', $url) == $_url)
{
$crc = sprintf('%x', crc32($_url));
if( ! file_exists("./stubs/{$crc}"))
{
file_put_contents("./stubs/{$crc}", "Status: 302\nLocation: {$_url}\n\n\n");
}
printf('<li><a href="http://%1$s%2$s%3$s">http://%1$s%2$s%3$s</a> » %4$s</li>',
$_SERVER['HTTP_HOST'],
str_replace('//', '/', dirname($_SERVER['REQUEST_URI']) . '/'),
$crc,
$_url
);
}
}
if(strlen($url_list) > 4)
{
echo $url_list, '</ul>';
}
}
?>
<form action="index.php" method="post">
<fieldset>
<legend>URL</legend>
<textarea name="u" id="u" rows="10" style="width:400px;"></textarea>
<p><input type="submit" name="s" id="s" value="Shorten!"/></p>
</fieldset>
</form>
</div>
<footer>© Me; 2011</footer>
</div>
</body>
</html>


Also, because the PHP function "http_build_url" is a PECL function, I have the following code I include in "lib.php".

<?php
if (!function_exists('http_build_url'))
{
define('HTTP_URL_REPLACE', 1); // Replace every part of the first URL when there's one of the second URL
define('HTTP_URL_JOIN_PATH', 2); // Join relative paths
define('HTTP_URL_JOIN_QUERY', 4); // Join query strings
define('HTTP_URL_STRIP_USER', 8); // Strip any user authentication information
define('HTTP_URL_STRIP_PASS', 16); // Strip any password authentication information
define('HTTP_URL_STRIP_AUTH', 32); // Strip any authentication information
define('HTTP_URL_STRIP_PORT', 64); // Strip explicit port numbers
define('HTTP_URL_STRIP_PATH', 128); // Strip complete path
define('HTTP_URL_STRIP_QUERY', 256); // Strip query string
define('HTTP_URL_STRIP_FRAGMENT', 512); // Strip any fragments (#identifier)
define('HTTP_URL_STRIP_ALL', 1024); // Strip anything but scheme and host

// Build an URL
// The parts of the second URL will be merged into the first according to the flags argument.
//
// @param mixed (Part(s) of) an URL in form of a string or associative array like parse_url() returns
// @param mixed Same as the first argument
// @param int A bitmask of binary or'ed HTTP_URL constants (Optional)HTTP_URL_REPLACE is the default
// @param array If set, it will be filled with the parts of the composed url like parse_url() would return
function http_build_url($url, $parts=array(), $flags=HTTP_URL_REPLACE, &$new_url=false)
{
$keys = array('user','pass','port','path','query','fragment');

// HTTP_URL_STRIP_ALL becomes all the HTTP_URL_STRIP_Xs
if ($flags & HTTP_URL_STRIP_ALL)
{
$flags |= HTTP_URL_STRIP_USER;
$flags |= HTTP_URL_STRIP_PASS;
$flags |= HTTP_URL_STRIP_PORT;
$flags |= HTTP_URL_STRIP_PATH;
$flags |= HTTP_URL_STRIP_QUERY;
$flags |= HTTP_URL_STRIP_FRAGMENT;
}
// HTTP_URL_STRIP_AUTH becomes HTTP_URL_STRIP_USER and HTTP_URL_STRIP_PASS
else if ($flags & HTTP_URL_STRIP_AUTH)
{
$flags |= HTTP_URL_STRIP_USER;
$flags |= HTTP_URL_STRIP_PASS;
}

// Parse the original URL
$parse_url = parse_url($url);

// Scheme and Host are always replaced
if (isset($parts['scheme']))
$parse_url['scheme'] = $parts['scheme'];
if (isset($parts['host']))
$parse_url['host'] = $parts['host'];

// (If applicable) Replace the original URL with it's new parts
if ($flags & HTTP_URL_REPLACE)
{
foreach ($keys as $key)
{
if (isset($parts[$key]))
$parse_url[$key] = $parts[$key];
}
}
else
{
// Join the original URL path with the new path
if (isset($parts['path']) && ($flags & HTTP_URL_JOIN_PATH))
{
if (isset($parse_url['path']))
$parse_url['path'] = rtrim(str_replace(basename($parse_url['path']), '', $parse_url['path']), '/') . '/' . ltrim($parts['path'], '/');
else
$parse_url['path'] = $parts['path'];
}

// Join the original query string with the new query string
if (isset($parts['query']) && ($flags & HTTP_URL_JOIN_QUERY))
{
if (isset($parse_url['query']))
$parse_url['query'] .= '&' . $parts['query'];
else
$parse_url['query'] = $parts['query'];
}
}

// Strips all the applicable sections of the URL
// Note: Scheme and Host are never stripped
foreach ($keys as $key)
{
if ($flags & (int)constant('HTTP_URL_STRIP_' . strtoupper($key)))
unset($parse_url[$key]);
}


$new_url = $parse_url;

return
((isset($parse_url['scheme'])) ? $parse_url['scheme'] . '://' : '')
.((isset($parse_url['user'])) ? $parse_url['user'] . ((isset($parse_url['pass'])) ? ':' . $parse_url['pass'] : '') .'@' : '')
.((isset($parse_url['host'])) ? $parse_url['host'] : '')
.((isset($parse_url['port'])) ? ':' . $parse_url['port'] : '')
.((isset($parse_url['path'])) ? $parse_url['path'] : '')
.((isset($parse_url['query'])) ? '?' . $parse_url['query'] : '')
.((isset($parse_url['fragment'])) ? '#' . $parse_url['fragment'] : '')
;
}
}
?>


That just gives me a simple textarea that I can enter a list of URLs into, and automatically have stubs for short URLs written to the stubs directory and get a list of shortened URLs back.

Saturday, November 13, 2010

client denied by server configuration: .htaccess

I was getting the error "client denied by server configuration: /.htaccess" in one specific location recently and I wasn't sure what was going on. Permissions allowed the .htaccess file to be read, because the directives set in that .htaccess were taking effect. There was also no special reason for anyone to try and navigate directly to this directories .htaccess file.

It turns out, that the problem had to do with mod_autoindex. The directory in question is setup to use mod_autoindex and list a bunch of changelogs in the directory. For whatever reason, mod_autoindex wants to access .htaccess while it's building a list of files and since .htaccess files are disallowed for all in the normal Apache configuration, it's getting that permission error.

I've found that adding .htaccess to the IndexIgnore directive will stop this error from happening. Since I also have mod_autoindex ignore the link to a parent directory, my IndexIgnore directive looks like this.

IndexIgnore .. .htaccess

Saturday, October 31, 2009

X-Pad: avoid browser bug

If you're looking for an explanation to this "X-Pad: avoid browser bug" header you should feel lucky. The X-Pad header is a work around Apache uses for a bug in really old versions of Netscape and it only shows up if there's a chance the 256th or 257th byte of a response is a newline.

It's a junk header, all it does is prevent the 256th and 257th byte of the response from being a newline. If Apache didn't do this, old versions of Netscape would hang.

It's a wonder why this is still part of Apache, we're talking really old versions of Netscape here.

Sunday, November 16, 2008

PHP Robot Check

I've seen quite a few methods for determining if a request was made by a robot or search engine spider using PHP over the years, and they always have something to do with checking the User Agent. That works, but you have to keep up with User Agent strings which can get to be annoying.

I had an idea how to check for robots using PHP and htaccess today that doesn't involve a User Agent string at all. Instead it uses sessions and exploits the fact that all well behaved robots are going to request robots.txt before they request anything else.

I start by making sure my usual robots.txt file is in place. Then I upload a robots.php file to the same location as robots.txt with a tiny bit of code in it.

<?php
session_start();
$_SESSION['robot'] = 1;
echo file_get_contents('robots.txt');
exit;
?>


All that does is start a session, set a robot $_SESSION variable that I can check in subsequent scripts, then return the contents of robots.txt.

In htaccess I have the following line which transparently redirects requests to robots.txt made by visitors/spiders to robots.php, which in turn returns the contents of robots.txt

RewriteEngine on
RewriteRule robots\.txt robots.php


Now in my applications I can easily check for robots and drop things like advertisement banners to speed up page loads since spiders don't look at advertisements anyway.

<?php
session_start();
echo isset($_SESSION['robot']) ? 'ROBOT !!' : 'Not a robot';
?>


Here are some of the benefits of doing it this way.

  1. I can continue to modify robots.txt as I normally would
  2. I don't need to keep up with changing User Agent strings
  3. Checking for the existance of a session variable is quicker than performing pattern matching on a variable

Friday, September 26, 2008

Broken Pipe: ef_unified_filter() failed

I was getting this error message in my Apache error logs while trying to setup an Apache based output filter using ExtFilterDefine and AddOutputFilter.

The error that accompanied it was
Broken pipe: apr_file_write(child input), len 0.

Not sure what either of those meant, but guessing the ef_unified_filter() failed was just a notice that the filter was broken, I tried a bunch of things and only found maybe 50 results while searching around, none of which helped.

It turns out that the problem was in the way I was defining the cmd="" argument for ExtFilterDefine. I, and the people who were having the same problem but with different commands, weren't using the full path to our command applications.

My original ExtFilterDefine directive looked like this.

ExtFilterDefine AutoWrap \
cmd="cat -"


However I needed to specify the full path to cat, which in my case looked like this.

ExtFilterDefine AutoWrap \
cmd="/bin/cat -"

Friday, September 5, 2008

Web File Formats 101

These are file extensions or formats you're likely to come across at one point or another if you're going to get into designing web sites. They're split into common groups, include a brief description of where you're likely to see or use them, and clicking the *.EXTENSION link in the titles will take you to a Wikipedia entry with a lot more details on that file format.

This list is a collection of about 45 file formats I've used enough times to remember when presented with a huge list of file formats after roughly five years of experience.




Browser File Formats

Browser file formats are files that instruct web browsers to do something directly. Many technologies may work with browser file formats, but browsers are generally the things that make use of them.

*.AS Adobe/Macromedia Action script file

AS files are generally used as library or optional include files in Flash or Flex applications.

*.CSS Cascading Style Sheet file

CSS files contain CSS rules which web browsers use to decide what web pages should look like.

*.HTM / *.HTML Hyper Text Markup Language

HTML files are web pages. HTML is what files such as PHP, ASP, CFML, and JSP produce when they're executed.

*.ICO Icon file

ICO files are commonly used as Favicons (that little icon in the address bar)

*.JS Javascript file

JS files are used to store Javascript. Javascript is a cornerstone in many of the applications on the web today. Javascript provides a way for web developers to utilize the browsers rendering engine and make changes to the content of a web page without needing to make a trip back to the web server to generate a new page with the updated content. Javascript is becoming more popular for producing animation in web pages since the jQuery, Prototype, and Moo-Tools Javascript libraries were developed.

*.SWF Adobe/Macromedia Shockwave Flash file

SWF files are generally used in Rich Internet Applications and require the Adobe/Macromedia Flash Player to view.

*.VBS Microsoft Visual Basic Script file

VBS files are similar in concept to JS files but they often have control over operation system functionality normally off limits to JS, however VBS files are only supported by Microsoft Internet Explorer versus JS which is available on almost every popular web browser. VBS is generally only found on private Microsoft Windows based Intranet sites, though even there it's being replaced by newer offerings from Microsoft.

Image File Formats

Image file formats are files used to store and transfer photographs, vector art, logos, favicons and things like that. They're also used to store information artists use to modify the images in image manipulation applications such as Photoshop and Fireworks.

*.BMP Bitmap Graphic file

BMP files are an early image format with little or no compression applied. BMP files are sometimes found on websites, but have generally been replaced by formats such as PNG, GIF, and JPG in recent years.

*.GIF Graphic Interchange Format file

GIF files are image files that sometimes include simple frame-based animation and alpha transparency. PNG has gained popularity over GIF in recent years do the the superior color quality of PNG, however transparency problems with Microsofts Internet Explorer 6 have slowed the transition from GIF to PNG.

*.JPG / *.JPEG Joint Photographic Group image file

JPG files are image files designed specifically for photos. The compression method used has been optimized for working with complicated images, making it especially well suited for transferring photos on the web.

*.PNG Portable Network Graphic file

PNG is an image file format that has gained popularity over GIF in recent years do the the superior color quality of PNG, however transparency problems with Microsofts Internet Explorer 6 have slowed the transition from GIF to PNG. PNG is also the native file format used by Adobe/Macromedia Fireworks, similar to how PSD is the native file format used by Adobe Photoshop. PNG has the ability to store information about layers when used in Fireworks in much the same way that PSD can store information about layers in Photoshop.

*.PSD Photoshop Document file

PSD is an image file format and is the native format used by Adobe Photoshop. PSD allows image authors to store information about layers in an image when edited in Photoshop in much the same way that PNG enables layer information when edited in Fireworks. Unlike PNG which generally has multiple export modes and can be exported for use on the web, PSD is primarily used for raw or drafts before exporting the finished pieces of the image to other image formats such as PNG, GIF, or JPG.

*.SVG Scalable Vector Graphics file

SVG files are vector image files. Unlike common raster image formats that store information about each pixel in the image, vector image formats only store information about how to draw the image, which generally consists of coordinates to place simple shapes and how to fill them with color.

Audio/Video File Formats

Audio/Video file formats are used to store and transfer audio and video.

*.AVI Audio Video Interleave file

AVI files are commonly used to distribute video.

*.FLV Flash Video file

FLV files are usually not observed directly, but rather played through an SWF file. FLV files are used for distributing streaming video.

*.MOV / *.QT Apple QuickTime Video Clip file

MOV or QT files are used to distribute video, similar to AVI files. Quicktime is generally known for a great level of quality in video.

*.MP3 MPEG Audio Stream, Layer III file

MP3 files are used to distribute audio. MP3 files use a compressed version of the audio information and are generally many times smaller in size compared to WAV files, despite including extra information about the audio such as the author, title, album name, and description.

*.MPEG / *.MPG MPEG 1 System Stream file

MPEG or MPG files are used to distribute video, similar to AVI, MOV, and QT files.

*.WAV Waveform Audio file

WAV files are an early audio format file which has all but been replaced by MP3 on the web due to superior compression and information embedding provided by the MP3 audio file format.

*.WMA Microsoft Windows Media Audio file

WMA is an audio file similar to MP3 and WAV developed by Microsoft. Like WAV, WMA has been replaced to a large extend on the web by MP3.

Compression File Formats

Compression file formats are used to apply compression to single, or groups of files in order to reduce the size needed to store and transfer the files, as well as making large groups of files easier to transfer at once.

*.BZ2 BZIP2 compressed archive file

BZ2 files are generally used by programmers to distribute compressed source code packages.

*.GZ GZIP compressed archive

GZ files are sometimes used in the same way BZ2 files are, for programmers to distribute source code packages. Some systems use GZIP compression on their old log files to save space which results in file names such as "date.log.gz". Some web servers use GZIP compression to compress data before sending it from a server to a browser.

*.RAR winRAR compressed archive

RAR files are often used by individuals to compress and transfer large files, or large groups of files. Similar to GZ, BZ2, and ZIP files.

*.ZIP Compressed archive file

ZIP files are similar to RAR, BZ2, and GZ files however ZIP files are generally a lot more common. ZIP files package and to a point compress other files to be transferred from system to system.

Misc File Formats

Misc file formats are generally not designed specifically for web sites, but have found numerous uses on them.

*.*.BAK Backup copy of file

Backup. The second asterisk is generally the original file extension.Rather than trying to open this file, you will usually rename it and remove the .BAK from the end, then open it with the appropriate application

*.DOC Microsoft Word document

DOC files are often used by companies to distribute bundled information suitable for printing. DOC files are a favorite of companies for sending contracts via e-mail. DOC files are similar in concept to PDF files.

*.INI Information/configuration file

INI files are commonly used to store persistent configuration data for web servers and web browsers. INI files are also used in the same way for applications that have nothing to do with web design. Using XML files to store configuration data instead of INI files has been gaining popularity in recent years.

*.KML Google Earth/Keyhole place mark file

KML files are used to share specific locations, or placeholders in the popular Google Earth application. KML files are becoming increasingly popular on business web sites who want to provide directions for customers.

*.PDF Acrobat Portable Document Format file

PDF files are often used by companies to distribute bundled information suitable for printing, similar to DOC files. Some print shops will ask for PDF files when you need to have fliers or brochures printed out. Some companies will make PDF copies of important information found on their web site available for people do download and read offline, or print out and read away from their desk.

*.RDF Resource Description Framework file

RDF files are often used to deliver RSS content. RDF RSS feeds are generally more specialized in comparison to the generic XML versions of RSS files.

*.TXT Plain text file

TXT files are plain text files. TXT files generally display the characters as-is with no consideration for special style codes.

*.XML Extensible Markup Language file

XML files are generally used to store structured information. Many web sites use AJAX to load XML in the background and update small portions of web pages without needing the server to generate the entire web page over again with slightly changed information. XML files are also gaining popularity as a settings or preference file format. The generic yet descriptive format used to section data off makes XML the perfect file format for storing raw information that will be displayed in many different ways.

Server File Formats

Server file formats are generally scripts that web servers execute in order to produce web pages dynamically. these types of files are also often used to perform maintenance on the server through automation.

*.ASP Microsoft Active Server Pages

ASP files are an earlier version of ASPX files. ASP files are similar in concept to CFML, PHP, ASPX, RB, and JSP files.

*.ASPX Microsoft ASP.NET source code file

ASPX files are a more recent version of ASP files. ASPX files are used on web servers to prepare HTML documents to send to a web browser. ASPX files are similar in concept to ASP, CFML, PHP, RB, and JSP files.

*.CFM / *.CFML Coldfusion Markup Language file

CFML files are used on web servers to prepare HTML documents to send to a web browser. CFML files are similar in concept to ASPX, PHP, ASP, RB, and JSP files.

*.CGI Common Gateway Interface file

CGI files are generally used to provide a bridge, or gateway, between a web server and applications installed on the server such as Python, Perl, sed, and many others.

*.JSP Java Server Pages

JSP files are used on web servers to prepare HTML documents to send to a web browser. JSP files are similar in concept to ASPX, PHP, ASP, RB, and CFML files.

*.LOG Log file

LOG files are generally exactly what the name implies, a log. Web servers use them to store details about each visit to a web site as well as to store information useful to administrators and developers in the event of a problem. Old, or archived LOG files are often compressed using GZIP compression, which results in "filename.log.gz" types of file names.

*.MDB Microsoft Access Database file

MDB files are a Microsoft database data storage format. The MySQL database engine has been gaining popularity over MDB in recent years. There are many MDB to MySQL converters and conversion methods available on-line.

I posted a how to for Importing MDB to MySQL on a Debian server not too long ago.

*.PHP Hypertext PreProcessor script file

PHP files are used on web servers to prepare HTML documents to send to a web browser. PHP files are similar in concept to ASPX, JSP, RB, ASP, and CFML files. PHP is one of the most popular pre-processors used on the web today.

*.PL Perl script file

PL files are Perl script files. Perl is one of the first scripting languages to be used on the web. Perl is often used in conjunction with CGI. In recent years PHP has stolen much of Perls thunder, leaving Perl used primarily by veterans.

*.RB Ruby script / Ruby on Rails class file

RB files are used on web servers to prepare HTML documents to send to a web browser. RB files are similar in concept to ASPX, JSP, PHP, and CFML files.

*.SHTML / *.SSI HTML including Server Side Includes

SHTML files are used on web servers to tell the web server that there's lines in the file that need to be replaced with the contents of other files before sending the contents of the file to the web browser. SHTML is generally a very simple version of what file types such as ASP, ASPX, CFML, JSP, PHP, and RB are designed for.

*.SQL Structured Query Language data file

SQL files generally contain the description of a database and the database table layouts. In some cases SQL files will contain information dumps from database tables, though this is generally reserved for small databases.

.htaccess Apache dynamic configuration file

.htaccess files are commonly used to make runtime adjustments to an Apache web server configuration without needing to restart the Apache server. Not all configuration options are able to be altered via .htaccess and depending on the servers configuration at start up there might not be any options that can be configured via .htaccess

Wednesday, December 26, 2007

Database Storage of Apache Virtual Hosts

I came up with a workaround for storing Apache Virtual Hosts in a database. I figure it could be adapted to work on Windows using batch files, but I'm only going to go over doing it on a UNIX-like system here. Specificly Ubuntu, or Debian.

At least basic knowledge of the following is assumed.


The first thing that needs to be done is creating a database schema for storing virtual host containers. I'll stick with a simple schema for now so focus stays on Apache to Database connection.

This schema is for MySQL.
CREATE TABLE `vhosts` (
`active` BOOL NOT NULL DEFAULT '0',
`id` INT UNSIGNED NOT NULL AUTO_INCREMENT PRIMARY KEY ,
`container` TEXT CHARACTER SET utf8 COLLATE utf8_general_ci NOT NULL ,
INDEX ( `active` )
);


`container` will store the actual vhost container. For instance somthing like this.
NameVirtualHost *
<VirtualHost *>
ServerAdmin webmaster@domain.com
DocumentRoot /home/me/public_html/
</VirtualHost>


`id` is for relation to other tables. Pretty standard stuff.

`active` is an on/off switch for each virtual host. Apache will only include a virtual host from the database if it is active.


With a database table ready to be read from we can move to the part where Apache uses the data from that table.
In httpd.conf, apache#.conf, or whatever your system uses for Apaches' main configuration file there's a directive available called Include. If you edit your configuration file and look at the end of the file you may see an example of Include being used already.
If you're on a Debian system you're likely going to find somthing similar to the following.
# Include the virtual host configurations:
Include /etc/apache2/sites-enabled/


If you're not on a Debian system, you're going to need to create a directory in your Apache directory similar to that. It should be owned by root. You will also need to add an Include directive like the one shown above to your main conf file pointing to the directory you create so Apache knows to load any configuration files found in that directory.

If you are on a Debian system you're eventually going to be emptying that directory so there's no conflicts with the database method. So now would probably be a good time to import the sites from sites-available into the database & mark the `active` field according to which sites are also in `sites-enabled`, as well as backing up that directory.


Now that we have a database to store and retrieve virtual host containers from, & a place where Apache can load configuration files from it's time to write a command to build a configuration file full of virtual host containers from the database for Apache to load.

Here's an example of such a command. Note that you'll have to sort out permissions on your own, the --user & --pass are used here for example sake, though MySQL documentation warns against doing it that way because it leaves the password visible in the pid file for a short time.

Some options are on a new line for ease of reading. The command ends with output redirection to a file in the sites-enabled directory for Apache to load.
I'm using a UNIX domain socket here, but you could use a TCP connection if need be. You might also want to alter the file output redirection is beind aimed at.
mysql --user=root --pass=pass --batch --raw --silent
--database=Websites
--execute="SELECT container FROM vhosts WHERE active = 1"
--skip-column-names
--protocol=socket
--socket=/var/run/mysqld/mysqld.sock
>/etc/apache2/sites-enabled/vhosts.conf


Of course instead of using that directory as an include directory in Apaches' configuration file, you could comment out that particular include & point an include directly at the file you have the above command outputting to.

Now we have a method to connect Apache with a Database for obtaining virtual hosts, but there's still manual labor involved in needing to run that mysql command when Apache is [re]started or reloaded.

To help with this we will have to edit the shell script used to stop/start/restart Apache so that every time Apache is [re]started, or reloaded, this file is refreshed.
On Debian this script is /etc/init.d/apache2, since I'm using Apache2.

Basicly anywhere there is a log_daemon_msg that [re]starts Apache in this script, we add our own logging & our database command right before it. So we end up with some sections looking somthing like this.
log_daemon_msg "Extracting vhosts from database" "apache2"
mysql --user=root --pass=pass --batch --raw --silent --database=Websites --execute="SELECT container FROM vhosts WHERE active = 1" --skip-column-names --protocol=socket --socket=/var/run/mysqld/mysqld.sock >/etc/apache2/vhosts.conf
log_daemon_msg "Starting web server" "apache2"


Now whenever Apache is [re]started, or reloaded, it will be using the vhost containers for active sites stored in the database.
sudo /etc/init.d/apache2 restart

Tuesday, December 11, 2007

Easier .htaccess On Windows

Now and then you'll see someone wondering how to create an .htaccess file for Apache on Windows. This usually appears to be people who have a private server setup on their local intranet for testing, then uploading to a remotely hosted server.
Windows apparently complains with an error message similar to the following when you attempt to create or rename a file to a name starting with a dot. Such as .htaccess.

You must type a filename !



What alot of people aren't aware of is that Apache actually has a setting you can alter to change the name of the file used for .htaccess.
The name of that setting is AccessFileName.

Now at first glance it would seem that altering the AccessFileName directive could cause major problems, and it can. You probably shouldn't alter this setting on anything other than a system in an intranet enviroment. You should also be prepared to rename existing .htaccess files when doing this.

That said, here's an example of what can be done with the AccessFileName directive. Note that this needs to be done in the Server Config or a Virtual Host container.
AccessFileName ht.htaccess

<Files ~ "^(ht)?\.ht">
Order allow,deny
Deny from all
</Files>


The line AccessFileName ht.htaccess should be pretty self explainitory. It makes the filename used ht.htaccess instead of just .htaccess
This way it's simple to create an ht.htaccess file on the Windows machine, then simply rename it to .htaccess once it gets uploaded to the server.

The second part of that example code isn't required, but it's done just in case you ever have other people with access to the machine. There's a similar section in your actual Apache configuration file already. That section doesn't include the (ht)? part that prevents the new ht.htaccess from being viewed as currently happens with the default .htaccess files.
It also will not conflict with anything if you later decide to revert to the default setting for AccessFileName & forget to remove it.

The Apache Manuals' htaccess page goes into much more detail about htaccess files.

Wednesday, September 26, 2007

Simple Gallery Using htaccess & Javascript

There's alot of great galleries out there for images, one of my old school favorites being a Flash-based one called Imagevuex. Recently I've been using a PHP and MySQL based wallpaper script I wrote myself.

But what if I just want to toss a bunch of images in a folder & have thumbnails for them without having to do a bunch when the contents of the folder changes ?, nothing fancy really.

Well, on a server with all the following, which are pretty standard issue anyway, I can.

  • Apache 1.3+

  • mod_autoindex

  • Ability to override "Indexes" in htaccess

  • A simple HTML file with some Javascript in it



First I create a directory to toss my images in, then I place a file named HEADER.html in that directory with the following contents in the file.
<script type="text/javascript">
var list;
var images;

window.onload = function()
{
list = document.getElementsByTagName('ul')[0];
images = document.getElementsByTagName('li');

for(var i=1; i<images.length; i++)
{
var thumb = images[i].firstChild.cloneNode(true);
thumb.innerHTML = '<img src="' + thumb.href.replace(/^about:/gi, '') + '" alt="' + thumb.href + '" style="max-width:100px; max-height:100px;"/>';
with(thumb.style)
{
width = '120px';
height = '120px';
display = 'block';
cssFloat = 'left';
border = '#ccc 1px solid';
margin = '2px';
padding = '2px';
textAlign = 'center';
}
thumb.onmouseover = function()
{
this.style.borderColor = '#999';
}
thumb.onmouseout = function()
{
this.style.borderColor = '#ccc';
}
list.parentNode.appendChild(thumb);
}
list.style.display = 'none';
}
</script>


What this Javascript does is read the usual directory index HTML that Apache generates & turns it into a bunch of clickable thumbnails once the page has been loaded in the browser.

The following htaccess addition will cause that files' contents to be appended right after the opening <body> element in the document, & tweek some of mod_autoindexes' options to get a simpler output from Apache to work with.
<IfModule mod_autoindex.c>
# Use simple XHTML output
# Ignore case in file sorting
IndexOptions XHTML IgnoreCase

# HeaderName is the name of a file which should be prepended to
# directory indexes.
HeaderName HEADER.html

# Ignore html files, mainly HEADER.html
IndexIgnore .??* *.html
</IfModule>


That last bit causes the directory indexing to skip HTML files, if you only put images in that directory you should be just fine.

So, with those two files & my new directory for photos which I'll call "photos", my structure looks somthing like this.
/photos/.htaccess
/photos/HEADER.html
/photos/image.png
/photos/image.jpg
/photos/image.gif


When I look at http://mysite.net/photos/, I get a screen full of thumbnails instead of a list of images once the page loads, if I toss more images in there I get an updated list & I don't have to update anything !

Now somthing to be well aware of is that each actual image is getting loaded here, not smaller thumbnails as it appears, the CSS used just makes it look like thumbnails are used.
Now, if you have maybe 50 photos that are less than 100KB each in the directory, it's only about the size of an MP3 to download in order see every images' thumbnail, but if you toss images straight from you camera where images tend to be 1MB+ each in that directory you're going to end up waiting about as long as it takes to download 25 MP3s just to see the thumbnails.

In either scenario, you aren't going to have to wait for the full images to load after the thumbnails show up, because the full images will already be loaded at that point.

Tuesday, September 25, 2007

Silky Directory Indexes

I really like the Silk Icons at famfamfam, alot more than the default FancyIndex icons used by Apache2 in fact.
So, now seems like a good time to patch those icons into my directory indexes for FancyIndexing.

First I get the zipfile fill of icons, which is actually a zipfile that has two readme files & a directory called icons in it.

Next I navigate to the directory Apache is already using for icons
Alias /icons/ "/usr/share/apache2/icons/"

In my case it's /usr/share/... look for "Alias /icons" in your *.conf file for Apache to find out where it is for you.

Now, I extract the icons folder from the zipfile into Apaches' icons directory & rename the extracted directory to "silk" so that I have the following.
/usr/share/apache2/icons/silk/


Now to change the icons in the Virtual Host container for the site in question, *.htaccess files could also be used to use different icons per-directory.
I'll start with the ones that have listings already, making the following changes to the AddIcon lines.

AddIconByEncoding (CMP,/icons/silk/page_white_zip.png) x-compress x-gzip

AddIconByType (TXT,/icons/silk/page_white_text.png) text/*
AddIconByType (IMG,/icons/silk/picture.png) image/*
AddIconByType (SND,/icons/silk/sound.png) audio/*
AddIconByType (VID,/icons/silk/film.png) video/*

AddIcon /icons/silk/application_double.png .exe
AddIcon /icons/silk/application_xp_terminal.png .bin
AddIcon /icons/silk/page_white_zip.png .Z .z .tgz .gz .zip
AddIcon /icons/silk/page_white_code.png .shtml
AddIcon /icons/silk/page_white_acrobat.png .pdf
AddIcon /icons/silk/page_white_zip.png .txt
AddIcon /icons/silk/page_white_c.png .c
AddIcon /icons/silk/page_white_cplusplus.png .cpp
AddIcon /icons/silk/page_white_dvd.png .dvi
AddIcon /icons/silk/page_white_gear.png .ini .conf .sh .shar .csh .ksh .tcl
AddIcon /icons/silk/page_white_actionscript.png .as
AddIcon /icons/silk/page_white_flash.png .fla .swf
AddIcon /icons/silk/page_white_freehand.png .fh3 .fh4 .fh5 .fh7 .ft7 .fh8 .ft8 .fh9 .ft9 .fh10 .ft10 .fhx
AddIcon /icons/silk/page_white_h.png .h
AddIcon /icons/silk/page_white_php.png .php .php3 .php4 .php5
AddIcon /icons/silk/page_white_office.png .docx .docm .dotx .dotm
AddIcon /icons/silk/page_white_powerpoint.png .pptx .pptm .potx .potm .ppam .ppsx .ppsm
AddIcon /icons/silk/page_excel.png .xlsx .xlsm .xltx .xltm .xlsb .xlam
AddIcon /icons/silk/page_white_ruby.png .rb
AddIcon /icons/silk/page_white_word.png .doc
AddIcon /icons/silk/css.png .css
AddIcon /icons/silk/html.png .html .htm
AddIcon /icons/silk/xhtml.png .xhtml
AddIcon /icons/silk/script_code.png .js
AddIcon /icons/silk/database_table.png .sql
AddIcon /icons/silk/script_gear.png .inc .pl .py
AddIcon /icons/silk/help.png .chm
AddIcon /icons/silk/music.png .wav .mp3 .ogg

AddIcon /icons/silk/folder_delete.png ..
AddIcon /icons/silk/asterisk_yellow.png README
AddIcon /icons/silk/folder_add.png ^^DIRECTORY^^
AddIcon /icons/blank.gif ^^BLANKICON^^


DefaultIcon /icons/silk/page_white.png


Followed by a restart of Apache for good measure.
sudo /etc/init.d/apache2 restart


I now have silky icons on my directory indexes, but apparently it's going to need a little tweeking to tell the difference between "text/text", "text/html", & "text/css" somewhere.