On January 2nd, 2008, from the rural town in eastern Taiwan where I was then working, I committed version 1.0 of the Yet Another Related Posts Plugin (YARPP) to the WordPress plugins repository. Since then, YARPP has been downloaded almost one million times and has evolved greatly. It also has been the impetus for my further involvement in the WordPress open source community.

I believe the success of YARPP comes from its simple premise: display related content to visitors and they will stay on your site longer. Your visitors learn more about you and they in turn find more compelling content that they’re interested in.

One oft requested feature for YARPP has been the ability to display related posts from across multiple sites. I’ve thought about what this would involve technically—the infrastructure, the algorithms, the queries—and have chosen not to go down this route. To do so would require a serious investment of time and resources, which doesn’t make sense for me as I continue to be in grad school.

In January, though, I met the team at BlogGlue while in Phoenix for WordCamp Phoenix. It was refreshing to meet a team who has really thought through the technical aspects of calculating “relatedness” of content, as well as the potential social and economic impact that this type of related content suggestion can have. They’ve been working in this space for a couple years and have a patented, scalable algorithm for computing “relatedness” of content from across thousands of different sites. When you sign up with BlogGlue, it displays “related links” on your blog post, just like YARPP does, but from your own site as well as from other content sources that you recommend. In turn, your blog posts may show up on your partners’ sites as well.

What surprised me during that visit was some of the statistics they had gathered from their users: not only does BlogGlue bring in traffic to your site, visitors who come in via the BlogGlue network stay on your site 3–4 times longer than visitors from Google and view 2–4 times as many pages. This makes complete sense to me based on my experience with YARPP: people want to see content that is relevant to them. A search query is one way to find content you’re interested in, but it’s imperfect. With related posts across a network, the system can find related content based on all the content on the current page, which is a much richer “cue” than a simple search query would ever be.

Today I’m pleased to announce that YARPP has officially partnered with BlogGlue. Think of BlogGlue as “YARPP Pro”: it’s the multi-site, social, next generation of YARPP. YARPP helps your visitors discover more of your own content once they’re on your site—BlogGlue additionally helps more people discover your content in the first place, in a rich, meaningful way.

YARPP will of course continue to exist and be supported by me. BlogGlue and I will be advising each other on technical aspects of our offerings, pushing both products forward, as well as cross-promoting our offerings. The latest version of YARPP, 3.3, already has some UI simplifications which came out of a conversation with the BlogGlue team. I’m looking forward to working with them more in the future and I urge you to try out BlogGlue on your own site as well.

Related posts:

1. Keep up with Yet Another Related Posts Plugin with RSS!
2. Yet Another Related Posts Plugin 2.0
3. Yet Another Related Posts Plugin

Related posts brought to you by Yet Another Related Posts Plugin.

A new user of my WordPress plugin tipped me off to a recent Tekzilla segment where WordPress founder Matt Mullenweg recommended Yet Another Related Posts Plugin as one of his personal favorite WordPress plugins. Thanks Matt! ^^

Related posts:

1. Yet Another Related Posts Plugin 2.0
2. Using Templates with YARPP 3
3. Keep up with Yet Another Related Posts Plugin with RSS!

Related posts brought to you by Yet Another Related Posts Plugin.

Warning: preg_match() [function.preg-match]: Compilation failed: unrecognized character after (?< at offset 3 in /…/html/blog/wp-content/plugins/wp-syntax/geshi/geshi.php on line 2132

This seemed to be coming from the version 1.0.8.4 version of Geshi I had installed. A quick google search for “geshi line 2132” gives you over a thousand errors, so this seems to be common issue. Geshi is a fabulous and popular syntax highlighter and is the core component of the WP-Syntax plugin for WordPress.

I did some digging around and realized that the issue was with the compilation of this monstrosity of a regular expression, used (as far as I can tell) to identify PHP code snippets, for example the <?php … ?> keywords:

/(?<start><\\?(?>php\b)?)(?:
(?>[^\"'?\\/<]+)|
\\?(?!>)|
(?>'(?>[^'\\\\]|\\\\'|\\\\\\\|\\\\)*')|
(?>\"(?>[^\"\\\\]|\\\\\"|\\\\\\\\|\\\\)*\")|
(?>\\/\\*(?>[^\\*]|(?!\\*\\/)\\*)*\\*\\/)|
\\/\\/(?>.*?$)|
\\/(?=[^*\\/])|
<(?!<<)|
<<<(?<phpdoc>\w+)\s.*?\s\k<phpdoc>
)*(?<end>\\?>|\Z)/sm

Not knowing exactly where to start in diagnosing this crazy expression, I simply disabled those “script delimiters” in the geshi/php.php file. The sections I commented out are lines 1080-1101. Now the script delimiters like <?php don’t get highlighted nicely, but I feel that’s a small price to pay for eliminating these errors. Another solution for the WP-Syntax users seems to be to downgrade to 0.9.4. Hopefully in the near future an update to Geshi will come out which fixes this issue once and for all.

Related posts:

1. Keep up with Yet Another Related Posts Plugin with RSS!
2. Yet Another Related Posts Plugin
3. Modifiying WordPress plugin activation behavior

Related posts brought to you by Yet Another Related Posts Plugin.

If you have a YARPP support question not directly related to the templating feature, please use the YARPP support forums.

Version 3 of Yet Another Related Posts Plugin is a major rewrite which adds two new powerful features: caching and templating. Today I’m going to show you how you can use templates to customize the look of your related posts output.¹

Previously with YARPP you were relatively limited in the ways you could present related posts. You were able to set some HTML tags to wrap your posts in and choose how much of an excerpt (if any) to display. This limited interface worked great for many users—indeed, these options still exists in YARPP 3.0. However, there’s also a new option for those of you who want to put your PHP skills to work and have complete control over your related posts display. The option will let you choose any files in the templates subdirectory of YARPP.

The structure of a YARPP template

Let’s take a look inside the example template, included with YARPP 3 (yarpp-template-example.php):

5
6
7
8
9
10
11
12
13
14

<h3>Related Posts</h3>
<?php if (have_posts()):?>
<ol>
	<?php while (have_posts()) : the_post(); ?>
	<li><a href="<?php the_permalink() ?>" rel="bookmark"><?php the_title(); ?></a></li>
	<?php endwhile; ?>
</ol>
<?php else: ?>
<p>No related posts.</p>
<?php endif; ?>

There are two basic parts to this (and most all) YARPP template: (a) what you display when there are related posts and (b) what you display when there aren’t. We make this switch with the conditional on line 6. If there are related posts, we introduce an ordered list and use the while loop to loop over all the related posts. For each post, we use the snippet the_post(); to load the appropriate post data, then print the line item.

You’ll notice that we’re using familiar template tags here such as the_permalink() and the_title(). If you’ve ever had to tweak or build a WordPress theme before, you’ll immediately feel at home. I’ll touch on this again later.

The power of PHP

One big advantage of this new templating system is that you can control exactly how the posts are listed, breaking out of all of the previous structural limitations. For example, in the template-list.php template, we put the information for each related post in an array and then concatenate the strings with implode. This way, we produce a comma-separated list for our readers without any stray commas before or after the list, which was impossible until now.

11
12
13
14
15
16

	$postsArray = array();
	while (have_posts()) : the_post();
		$postsArray[] = '<li><a href="'.get_the_permalink().'" rel="bookmark">'.get_the_title().'</a></li>';
	endwhile;
 
echo implode(', ',$postsArray); // print out a list of the related items, separated by commas

You can also run any arbitrary PHP in the template file—even roll another WP_Query, as in the case of template-random.php, where a random post is returned when there are no related posts.

Familiar template tags

As mentioned before, the tags we use in these YARPP templates are the same as the template tags used in any WordPress template. In fact, any WordPress template tag will work in the YARPP Loop. You can use these template tags to display the excerpt, the post date, the comment count, or even some custom metadata. I’ve also written two special template tags which only work within a YARPP Loop: the_score() and get_the_score(). As you may expect, this will print or return the match score of that particular related post.

In addition, template tags from other plugins will also work. For an example, take a look at the yarpp-template-photoblog.php file:

8
9
10
11
12

	<?php while (have_posts()) : the_post(); ?>
		<?php if (function_exists('yapb_is_photoblog_post')): if (yapb_is_photoblog_post()):?>
		<li><a href="<?php the_permalink() ?>" rel="bookmark"><?php yapb_get_thumbnail(); ?></a></li>
		<?php endif; endif; ?>
	<?php endwhile; ?>

In this template’s YARPP Loop, we use some template tags introduced by the Yet Another Photoblog plugin. If you have the Yet Another Photoblog plugin installed, you can use this template to display thumbnails of related posts in lieu of the titles. Notice that here we’re checking first whether each related post is indeed a photo post or not using yapb_is_photoblog_post() and then using the Yet Another Photoblog yapb_get_thumbnail() template tag to get the location of the thumbnail.

Templating in YARPP 3.0 enables the blog admin to uber-customize their related posts display using the lingua franca of PHP and template tags. Feel free to comment here with ideas, comments, and of course links to your YARPP-powered blogs. I look forward to seeing what the WordPress community does with this new feature!

Related posts:

1. Yet Another Related Posts Plugin 2.0
2. Yet Another Related Posts Plugin
3. Keep up with Yet Another Related Posts Plugin with RSS!

Related posts brought to you by Yet Another Related Posts Plugin.

It’s always interesting to see these release spikes in download traffic. Note that this release was on the Wednesday but that was during the day, so Wednesday’s traffic is still higher than the normal ~300/day level, while the big peak (by day) is on Thursday. Too bad wordpress.org doesn’t give me hourly stats, though I guess that would be a little ridiculous.

YARPP is just about at that 35k download mark. I’m looking forward to the next release. ^^

Related posts:

1. Yet Another Related Posts Plugin 2.0
2. Keep up with Yet Another Related Posts Plugin with RSS!
3. Modifiying WordPress plugin activation behavior

Related posts brought to you by Yet Another Related Posts Plugin.

In order to introduce an external ordering source, we need to do four things: 1. create the external ordering source, 2. hook up (read “join”) the external ordering source 3. make sure we use that order, and 4. make it play nice. ^^

By the way, I’m going to assume you, dear reader, are PHP-savvy, proficient in MySQL, and already know a little about WordPress. This how-to is not for the PHPhobic.

The ordering source

For this example, suppose we want to display posts by order of “interestingness.” We’ll just create a table called wp_interestingness with two columns, ID and interestingness and populate it with some data. We’ll even be nice to our database by making sure the ID is the primary key. Easy.

Hook up the external ordering source

When you run a query through query_posts() (or use WP_Query’s query method²), what it’s doing is taking your special request and translating it into a MySQL statement. This means a query like "monthnum=1" is turned into SELECT ... wp_posts.* FROM wp_posts WHERE 1=1 AND MONTH(wp_posts.post_date)='1' .... Every different query introduces something new to the basic SELECT command—in this case, the AND MONTH(wp_posts.post_date)='1'.

We first want to introduce the interestingness for each post and that means joining the new table into the query. We’ll do this using the posts_join filter. This filter lets you add a join statement to the MySQL request.

1
2
3
4
5

add_filter('posts_join','my_join_filter');
function my_join_filter($arg) {
	$arg .= " natural join wp_interestingness ";
	return $arg;
}

Note that here we’re using natural join as wp_posts and wp_interestingness have only one key in common, ID, and that’s exactly the column we want to join them on.

Use the new order

Now that we’ve joined wp_interestingness in, we can refer to wp_interestingness.interestingness in our query. Note now that, by default, an $wpdb->posts.post_date will be used to order the posts. We’ll use another filter here; this time posts_orderby, to patch this part of the query. We’ll search for the default ORDER BY value and replace it with our own interestingness.

1
2
3
4
5
6

add_filter('posts_orderby','my_orderby_filter');
function my_orderby_filter($arg) {
	global $wpdb;
	$arg = str_replace("$wpdb->posts.post_date","wp_interestingness.interestingness",$arg);
	return $arg;
}

By the way, you can now check the resulting MySQL query by echoing $wp_query->request. (If you’re using the WP_Query method I advocated below in footnote (2), you’ll of course have to change $wp_query to the WP_Query object you’re using.)

Learn to play nice ^^

The instructions above do indeed work, but they also cause some major breakdowns in other functions of your blog. Why? That’s because the current code will edit your queries for every instance of The Loop: your index page, your archives, and your RSS feeds. You probably only want to search by interestingness in certain situations. What we need is a way to tell our (admittedly stupid) my_join_filter and my_orderby_filter when they should apply their interestingness magic and when they shouldn’t. There are several ways to set up such a system but here I’ll lay out one that I feel is particularly elegant. We’ll set it up so you can actually use query_posts("orderby=interestingness") and it’ll know what you’re talking about.

One of the first things that happens in query_posts—indeed, way before even the posts_join and posts_orderby filters—is an action hook called parse_query. This lets us look at the initial state of the WP_Query object as it starts to run. In particular, we can look at the orderby query variable and see if we want to order by interestingness. If we do, we’ll set a global variable called $use_interestingness_flag to be true.

1
2
3
4
5
6
7
8

add_action('parse_query','set_use_interestingness_flag');
function set_use_interestingness_flag($query) {
	global $use_interestingness_flag;
	if ($query->query_vars['orderby'] == 'interestingness')
		$yarpp_score_override = true;
	else
		$yarpp_score_override = false;
}

Now we just have to edit our filters so they only run when $use_interestingness_flag == true. We also will make sure to turn the flag back off in my_orderby_filter, as it’s our last filter to run during each query. It’s just like putting the seat back down after using a unisex bathroom.³

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

add_filter('posts_join','my_join_filter');
function my_join_filter($arg) {
	global $use_interestingness_flag;
	if ($use_interestingness_flag)
		$arg .= " natural join wp_interestingness ";
	return $arg;
}
add_filter('posts_orderby','my_orderby_filter');
function my_orderby_filter($arg) {
	global $wpdb, $use_interestingness_flag;
	if ($use_interestingness_flag)
		$arg = str_replace("$wpdb->posts.post_date","wp_interestingness.interestingness",$arg);
	$use_interestingness_flag = false;
	return $arg;
}

This method has a great advantage as you can just set it up once and invoke it whenever you want, even together with other parameters, without any additional code. For example, you can try query_posts("monthnum=1&orderby=interestingness") or query_posts("cat=-529&orderby=interestingness").

Conclusion

Adding an external ordering source to your WordPress post queries can be relatively straightforward if you understand what query_posts does and take advantage of its hooks. This tutorial can also serve as the basis for many other patches to WP_Query, not just the orderby parameter. To better understand the way WordPress builds its MySQL queries and the many posts_* filters which you can take advantage of, go to the source: wp-includes/query.php. Finally, you can use the special parse_query hook and global variables as flags to only apply the filters when necessary.

Related posts:

1. Yet Another Related Posts Plugin 2.0
2. Markdown for WordPress and bbPress
3. Yet Another Related Posts Plugin

Related posts brought to you by Yet Another Related Posts Plugin.

Clicking on a version’s permalink will let you download the plugin. Subscribe now and be the first to find out when the upcoming version 2.1 is released!

I decided to semi-automate this RSS-producing process as well. As a plugin developer using wordpress.org’s plugin hosting, I sync a local copy of the plugin to their server using SVN. I wrote a PHP script to get the modification date information directly from the local files, parse the version log in the read me, and produce the RSS feed. If there’s an interest, perhaps I’ll release this code in the future.

Related posts:

1. Yet Another Related Posts Plugin 2.0
2. Yet Another Related Posts Plugin
3. Modifiying WordPress plugin activation behavior

Related posts brought to you by Yet Another Related Posts Plugin.

This posting is now outdated… for the latest information on YARPP, please visit YARPP’s very own page on my site, or its page on wordpress.org. If you have questions, please submit on the wordpress.org forum. Thanks!

Description

Today I’m releasing Yet Another Related Posts Plugin (YARPP¹) 1.0 for WordPress. It’s the result of some tinkering with Peter Bowyer’s version of Alexander Malov & Mike Lu’s Related Entries plugin. Modifications made include:

1. Limiting by a threshold: Peter Bowyer did the great work of making the algorithm use mysql’s fulltext search score to identify related posts. But it currently just displayed, for example, the top 5 most “relevant” entries, even if some of them weren’t at all similar. Now you can set a threshold limit² for relevance, and you get more related posts if there are more related posts and less if there are less. Ha!
2. Being a better plugin citizen: now it doesn’t require the user to click some sketchy button to alter the database and enable a fulltext key. Using register_activation_hook, it does it automagically on plugin activation. Just install and go!
3. Miscellany: a nicer options screen, displaying the fulltext match score on output for admins, an option to allow related posts from the future, a couple bug fixes, etc.

Installation

Just put it in your /wp-content/plugins/ directory, activate, and then drop the related_posts function in your WP loop. Change any options in the Related Posts (YARPP) Options pane in Admin > Plugins.

You can override any options in an individual instance of related_posts using the following syntax:

`related_posts(limit, threshold, before title, after title, show excerpt, len, before excerpt, after excerpt, show pass posts, past only, show score);

Most of these should be self-explanatory. They’re also in the same order as the options on the YARPP Options pane.

Example: related_posts(10, null, 'title: ') changes the maximum related posts number to 10, keeps the default threshold from the Options pane, and adds title: to the beginning of every title.

There’s also a related_posts_exist) function. It has three optional arguments to override the defaults: a threshold, the past only boolean, and the show password-protected posts boolean.

Examples

For a barebones setup, just drop <?php related_posts(); ?> right after <?php the_content() ?>.

On my own blog I use the following code with <li> and </li> as the before/after entry options:

<?php if (related_posts_exist()): ?>
<p>Related posts:
<ol>
<?php related_posts(); ?>
</ol>
</p>
<?php else: ?>
<p>No related posts.</p>
<?php endif; ?>

Coming soon (probably)

1. Incorporation of tags and categories in the algorithm. I’ve gotten the code working, but I still need to think about what the most natural algorithm would be for weighing these factors against the mysql fulltext score currently used (and works pretty well, I must say).
2. Um, something else! Let me know if you have any suggestions for improvement. ^^

Version log

1.0 Initial upload (20071229)

1.0.1 Bugfix: 1.0 assumed you had Markdown installed (20070105)

Related posts:

1. Introducing Smartdate

Related posts brought to you by Yet Another Related Posts Plugin.