Plugin Support Thread

I need to temporarily discontinue offering personal support for my WordPress plugins. I sincerely apologize for this. I’m putting in time nights and weekend at my job – that and my family have to come first. I don’t expect my situation at work to be like this for more than a few more months. In the meantime, please use the comments thread on this post if you have questions, and I’m hoping you can support each other (there are at least a few savvy users of my plugins who have posted helpful comments in the past). I will try to jump in if I can to offer help, but I can’t promise it right now.

Silent Running

It’s been over a month since I blogged or tweeted. Aside from this post, it’ll be probably another month before I do so again. I’d especially like to apologize to the people looking for help with Shashin and my other plugins, as I have not been responding to support requests (for my plugin users, please see this post).

As I mentioned back in the Spring, I’ve been leading our web team’s transition to scrum. Since then we’ve been working with Agile/scrum training coaches Bob Hartman and Darian Rashid, and they’ve done an amazing job helping us make the transition a successful one.

Before starting with scrum we had poor visibility into our future work – planning was extremely difficult. Now we’re getting better visibility, and it’s something of a “be careful what you wish for” situation. I’ve been working nights and weekends for the past month, getting a handle on all our projects and our schedule, so I can manage expectations for both my team and for our stakeholders. Work is the first thing I think about when I wake up, and the last think I think about before I go to sleep at night. It’s going to stay that way for at least a few more weeks (possibly months), as we get through this transition.

We have several goals: improving quality, teamwork, etc. But our first is to improve our planning: to align our workload with our actual capacity, establish a sustainable pace, and create reliable expectations for our stakeholders. With scrum’s velocity measures and other metrics, my ultimate goal is to clearly demonstrate to our stakeholders what our team already knows: that we do an incredible amount of quality work with a very small staff, and that if we’re expected to do even more, we need more people.

Displaying WordPress Posts by Category, Even If They’re Not Recent

A couple years ago I wrote a post describing how to limit posts on your WordPress home page to ones in specific categories. A limitation of this approach is that it can only show recent posts that are in The Loop. Others have solved this problem as well, but all the solutions I’ve seen have this same limitation. What if posts in the categories you specify aren’t among your recent posts? Your home page would show no posts!

This was a problem for my new home page design. I’ve divided my site’s content into 3 major topics, and I show hyperlinked titles for the 3 most recent posts in each topic. But it’s possible that a topic may not have any posts among the most recent 10 posts, which is all The Loop knows about (10 is WordPress’ default setting for how many posts to show on your home page). So I want to get the most recent 3 posts for each topic, regardless of whether they happen to be in The Loop.

To do this, I created the following function and put it in my theme’s functions.php file. Note that I’m using a straight SQL query, which means this is not guaranteed to work in future versions of WordPress (the WP coders do a good job of maintaining a consistent programming API across versions, but they do change the database sometimes).

function get_top_category_posts ($term_ids) {
    global $wpdb;
    $top_3 = '';

    $results = $wpdb->get_results("select ID, post_title, post_date from wp_posts p
            inner join wp_term_relationships r on p.ID = r.object_id
            inner join wp_term_taxonomy t on t.term_taxonomy_id = r.term_taxonomy_id
            where t.term_id in ($term_ids) and p.post_type = 'post'
            order by post_date desc limit 3", ARRAY_A);

    foreach ($results as $result) {
        $top_3 .= "<li>" . date("M d", strtotime($result['post_date'])) . " - "
            . '<a href="' . get_permalink($result['ID']) . '">'
            . $result['post_title'] . "</a></li>\n";
    }

    return $top_3;
}

I then call it like this from my custom home page (where I’m not using The Loop at all):

<ul>
<?php echo get_top_category_posts('6,12,89,90,98,105,106,115'); ?>
</ul>

I’m passing the IDs for the categories I want. The easiest way to get a category ID is to go to its edit screen and note the cat_ID in the URL. This works for tag IDs also.

There are 3 database tables involved in the query: wp_posts contains your posts, wp_term_taxonomy contains the term IDs for categories (and tags), and wp_term_relationships connects the posts to their categories. Note that a cat_ID or tag_ID you see in an edit screen URL is actually called a term_id within the database. In this case I’m only retrieving the date and title of each post, but you could retrieve the entire post if you want (see the description of the wp_posts table).

If you want to simplify the query to improve performance, you can eliminate one of the table joins if you manually look up the term_taxonomy_id that corresponds to each category’s term_id, and pass those instead:

select ID, post_title, post_date from wp_posts
        inner join wp_term_relationships on ID = object_id
        where term_taxonomy_id in ($taxonomy_ids) and post_type = 'post'
        order by post_date desc limit 3

The downside of doing this is that every time you add a new category or tag, you’ll have to go into your database and look up its term_taxonomy_id. So I don’t recommend doing it this way unless you’re comfortable poking around in mySQL.

(Note to fellow cranky programmers: I know this is poorly abstracted – the need here is simple enough that, to me, abstraction didn’t seem worth the trouble).

A Rumsfeldian Approach to Sprint Planning

UPDATE: A disclaimer to calm some readers who were worried: this post is by no means an endorsement of Rumsfeld. I’m just taking a peculiar statement of his and using it as a way to think about scrum (and I just thought it was funny).


Donald Rumsfeld - Scrum MasterDonald Rumsfeld – Scrum Master

Donald Rumsfeld – Scrum Master

 

“There are known knowns. These are things we know that we know. There are known unknowns. That is to say, there are things that we now know we don’t know. But there are also unknown unknowns. These are things we do not know we don’t know.”

– United States Secretary of Defense Donald Rumsfeld, February 12, 2002

It’s not an accident the name of my department at U Penn’s School of Medicine is Information Services, not Information Technology. We develop software, and for that we are adopting scrum, as I mentioned in an earlier post. But we also provide services, such as meeting various ad hoc reporting needs for our clients. Until now we have tended to be more reactive than proactive, in that we typically wait for service requests to come to us (and they often come on short notice) and then we scramble to fulfill them. Our current workflow is all about multi-tasking and being interrupt-driven, both of which are productivity killers. In a certain sense we are extremely far out on the agile spectrum, in that the idea of locking down requirements and schedules for even as short a duration as a 2 week sprint will be introducing a lot more workflow structure than we’re accustomed to.

To make the change, we need to become more proactive than reactive. Do all those ad hoc requests really need to be quite so ad hoc? Can we do more to anticipate service needs and plan better? This is where Rumsfeld’s seemingly mystifying statement has relevance:

  • A known known is the work we are planning for a sprint. We prioritize user stories, estimate them, etc. and plan our sprint. This is the work we know we will be doing.
  • A known unknown is work we know is out there, but we lack good information on when and/or what exactly it entails. An example is a complex, crucial report we know needs to be run on a certain date, but we don’t know what kind of data issues might come up. The report might turn out just fine and only require a few minutes of effort to run, or the data may contain very subtle errors that take hours to investigate.
  • An unknown unknown is work that cannot be anticipated: a newly exposed bug on an essential system that urgently needs to be corrected; the Dean asking for some new report that no one knew he would ask for (and asking the Dean to get in line is generally not an option…), etc.

A notion we’ll be testing as we move to scrum is whether we can move a lot of our current known unknowns into the known knowns column. To continue with the report example, if we know when it will be run, we could put some slack around it in the schedule, since it’s an important report. We also could do a test run of the report ahead of schedule and review it with the client to see if the numbers add up as expected, to minimize the risk of a last minute surprise. This will require us to be more proactive with our clients, in that we’ll need to seek out and identify upcoming service needs and incorporate them into our sprint planning. We also need to involve our clients more in the overall quality equation, such as asking (and expecting) them to test and review important reports ahead of their due dates. This calls for a culture change for some of our clients, which I expect will be the most significant challenge for us in adopting scrum.

With unknown unknowns such as the Dean asking for unexpected work, we may never be able to control them entirely, but with the tools scrum gives us (such as burndown charts) we can at least more clearly illustrate for our clients how ad hoc requests are impacting their project schedules. It’s also my belief that work requests that are truly impossible to anticipate don’t actually happen very often. The harder cases are bugs that take hours to unwind, or code changes in one place that have unexpected consequences somewhere else. We can reduce these over time by introducing technical practices such as test driven development and regression testing. Both of these will require training and the development of a testing infrastructure, so it will take a while to get there, but we can get there. In the meantime, I’m planning to not have our scrum teams completely fill their sprint time in their sprint planning. We’ll leave some time in the schedule for handling unexpected work.

You might be wondering about unknown knowns – things we don’t know we know. That gets us into the realm of repressed childhood memories and past lives, and I’m hoping that such things won’t impact our sprint planning…

Shashin Support Mea Culpa

Shashin’s popularity has recently spiked, which makes me happy, but it’s also caused a spike in support requests (my last Shashin post has 127 comments). Until the start of this year, I had been diligent in providing support, but my day job recently has been seeping into my nights and weekends, significantly reducing my availability. It’s likely to stay that way for the next several months.

I started on some major enhancements to Shashin early in the winter – most notably flickr and twitpic support – but it’s been on hold for a while now. At this point, my guess is that it’ll be summer before I can get back to it.

Please use the comment section on this post for support questions. I will do my best to provide responses twice per week. I will most likely not be able to delve into problems that involve the particulars of your server configuration, such as problems with connecting to Picasa via curl for unlisted albums (as those kinds of issues can sometimes take hours to work through). But general usage questions and Highslide issues are usually easier to address, and I’ll do what I can to help out.

Thanks everyone for your patience, and please remember I’m doing this for free 🙂 (donations come in very rarely).

UPDATE: I just posted a minor update, version 2.6.3:

  • Shashin is now compatible with WP 3 beta 1 (WP 3 appears to automatically unserialize options, which was causing errors)
  • Added support for child themes
  • Can now uninstall and delete Shashin from the main plugin page (added uninstall hook for existing uninstall function)

Trying Out Scrum

I’ve barely blogged at all for the past several months, and the main reason is that my new job has been keeping me busy, and using up my creative energy. But it’s all good – I’m enjoying the challenges. In case you missed my previous tweets about it, I’m now Director of the 14 person web team in U Penn’s School of Medicine Information Services. I’ve been a member of the team for 5 years, so my role is new but much of the territory is familiar.

Our team has created over 80 web-based applications (including our own development framework), and well over 150 static web sites, and we’re adding new applications and sites all the time. It’s amazing work for such a small group. Check out the portfolio of some of our static sites, and here’s an example of one of our applications: the PennMed Clinical Trials search site (most of our application are for internal PennMed use only, so this is one of just a few that are public).

We’ve started two pilot projects with scrum, which is an “agile” methodology (click the link for a basic definition). Currently, we have a workflow that’s evolved over the years, and is scrum-like in some respects: we expect and allow change, we communicate frequently with our clients. But we’ve held on to “waterfall”-like habits, mainly because of how we were educated as programmers: we try to model as much as possible up front, then we build all the tables, then code all the business logic, etc. Because we are caught between these two approaches, we often run into frustrations: we’ve tended to see refactoring as an indication that we did something wrong up front, we’ve never been sure about the right ways to document our work, and we’ve struggled with scheduling and allocating resources.

I gave a presentation to our team last week on why we’re evaluating scrum, and why we’re doing it now. There’s nothing top secret about it, so I’ve uploaded it to SlideShare, and it’s embedded below.

UPDATE 12/3/2010: I’ve revised and replaced the original slideshow:

Google ClientLogin PHP Examples

The latest version of Shashin (v2.6), which is my plugin for using Picasa with WordPress, adds support for unlisted Picasa albums. Supporting unlisted albums is possible because 1. Picasa does not actually require any authentication to view the photos in unlisted albums if you know their URLs (they simply don’t show you links to view unlisted albums at Picasa, or view their RSS feeds, unless you’ve been authorized), and 2. the Google ClientLogin API is available.

The ClientLogin API documentation has really good prose, but precious little code. There is a freely available PHP library (fron Zend) for interacting with the Google APIs, but I didn’t want to bloat Shashin by several megabytes just so I could use the library’s authentication method (and there are enough interdependencies in the library that I couldn’t easily extract the authentication components).

Shashin uses ClientLogin instead of AuthSub because Shashin has an option for automatic scheduled synchronizing of Picasa albums, so the username and password needs to be saved. I can’t expect Shashin users to type in their Picasa username and password every time Shashin syncs its album data with Picasa.

Below I’ll explain how authenticate to Google services using ClientLogin, and then show a couple examples of interactions with Google services.

Shashin 2.6: Snowpocalypse Edition

Shashin 2.6 is now available for download at wordpress.org. I figure the end of the world is a good time to release a new version. Here’s what’s new:

  • Added support for unlisted Picasa albums (finally!). You must have the PHP curl extension installed to use this feature. I believe most PHP installations include curl, but some hosting providers may need you to ask them to enable it for your account.
  • Added ability to group albums by Picasa user accounts when using the [salbumthumbs] tag.
  • Bug fix: the EXIF support introduced in Shashin 2.4 caused an error when syncing albums in the Windows version of mySQL. Many thanks to MC for letting me debug this on his Windows server.
  • Bug fix: Shashin’s automatic album syncing was interfering with scheduled jobs from other plugins in some circumstances. This is fixed. Note that you also need WordPress 2.9.1 or higher, as this was related to a wp-cron bug in WordPress 2.9.

Also, I’ve installed the Subscribe to Comments plugin, so if you have a support question, you can get an email notification when I reply, instead of having to remember to check back.

UPDATE: I’ve made a couple bug fixes: v2.6.1 fixed another bug related to Windows servers, and v2.6.2 fixed a bug related to Google servers outside the US (see this comment for details).

Shashin 2.5 Now Available

Shashin 2.5 is now available for download at wordpress.org. This actually isn’t the major release I’ve been working on – it doesn’t include Flickr and Twitpic support (that’s still a work in progress). But it does include one very cool new feature: a jQuery based WYSIWYG browser for adding Shashin photos to your posts. You can now browse albums from within the editor, and click on photos to add them to your post (it will write [simage] and [sthumbs] tags for you). To use it, click the Picasa icon in the editor’s media button bar. This great feature was contributed by Sune Pedersen.

A couple usability items to point out with it:

  • If you mouse over a thumbnail on the left hand side, then the right hand side will show you a larger version of it, and vice versa. I found this a little confusing at first, but it’s actually a good way of using the available screen real estate effectively
  • To select a photo, just click it once. Don’t try to drag it. After you click it, you’ll see it appear under the “Selected” heading. If you change your mind and need to remove a photo that you selected, just click it, and it will be removed from the “Selected” area.

The other important new feature is that automatic album syncing is now done every 10 hours instead of once daily. This is because Picasa video URLs now expire every 11 hours, as I described previously. If you have any videos in your albums, then you will want to turn on this feature in the Shashin Settings menu.

There are also two bug fixes:

  • “Next” and “previous” links for browsing album photos now work in Google Chrome and Safari (this is actually a workaround for a webkit bug)
  • Album title links in the album thumbnails sidebar widget now point to the correct URL

Now back to work on version 3.0…

A Problem for Shashin – Picasa Video URLs Now Expire

Sometime over the past month, Picasa made an unannounced change to how it serves videos. Previously, the Picasa RSS feeds provided URLs for videos, and those URLs worked indefinitely. Now the URLs in the RSS feeds expire after 11 hours. So if you add a video to a Picasa album, then sync the album in Shashin, it’ll work fine for 11 hours, and then it won’t work anymore. To make matters worse, re-syncing the album won’t help – Shashin doesn’t check the feed for a URL change when deciding whether to update its records, since the URLs used to never change (Shashin doesn’t just blindly update all its records when sync’ing; it checks certain fields, and then updates only if one of them changes, in order to keep sync’ing relatively fast).

The expiration time is built-in to the new video URLs, and the videos won’t play after that time passes, even if you’re logged in to Picasa. This leads me to believe that using the Picasa API wouldn’t help (even the code used to display the videos on Picasa’s own pages expires). There’s also a signature key built-in to the URL, so there’s no simple hack to the URL that’ll make it last longer.

The ability to embed videos in your own site was never a documented Picasa feature, so I really don’t have grounds to complain. But I made it a feature of Shashin, so my users do 😉 .

I’m traveling over the holidays, but I will try to get a fix released before New Year’s. The only solution I can think of is to change Shashin’s automatic once per day sync’ing feature so that it will sync every 10 hours (an hour before the expiration, to be safe). That way you’ll always have a fresh URL for embedded videos. This won’t be the major new release I’ve mentioned before – it’ll just be a fix for this and a few other minor bugs.

Note: videos added to Shashin more than a month ago still work fine, at least for now. The “old style” URLs in Shashin are still working, and – like I said – they don’t get updated when you sync. It’s only videos you may have added to Shashin within the past month that will give you a problem.