Toppa.com has become a quiet place in recent months. I’ve been spending what time I have on coding Shashin 3, so I haven’t been blogging. I have been applying Agile coding practices to WordPress plugin development, and it’s more than I bargained for. I could write a book about it, but I’ll probably never have time, so I’ll see how far I can get with a series of blog posts instead. Here are some key points I hope to expand on in upcoming posts:

• Testing: a typical WordPress plugin is sprinkled with calls to custom WordPress functions and hooks, so how can you unit test anything? Also, PHPUnit is a command-line utility, so how can you test plugin code that’s meant to run from within WordPress, which is a web app? The answer to the first question is to pursue a combination of isolating your WordPress dependencies, and running them through a facade (my toppa-libs library provides one, but it’s coverage of WordPress functions is still limited). An answer to the second question is SimpleTest, which is designed to run tests through a browser. I’ve written a plugin that runs SimpleTest from within WordPress, so you can do integration testing: SimpleTest for WordPress (which I hope to have up at wordpress.org soon). Update 6/15: it’s now available at wordpress.org
• Which PHP version to use? PHP 5.3 provides a decently comprehensive object oriented toolkit. But WordPress understandably stays at least a couple years behind the current PHP version, given the widespread use of WordPress (not every hosting provider keeps up with the latest PHP releases). The upcoming WordPress 3.2 release will have PHP 5.2.4 as a requirement, so I’ve decided to have the same requirement in Shashin. That means I can use type hinting and autoloading, but not namespaces.
• Code organization: WordPress gives you custom hooks and functions, but how you organize your code is completely up to you. As a result, a lot of WordPress plugins are huge, single files that are a jumble of code for display and application logic. In Shashin I’m using output buffering to create pseudo-templates for display code, and for the application logic, I’m sticking to the single responsibility principle, which means I have small classes and small methods.
• Performance: given the processing power of modern computers, an Agile principle is that you code for readability and maintainability first, and worry about performance only when there is evidence that you may need to sacrifice some of that readability to improve performance. Sticking to this principle is a challenge with WordPress. WordPress sites are deployed in a huge variety of environments, with differences in site configurations, traffic, and the number and type of other plugins in use. This leads to a natural tendency to code defensively for performance. I’ve tried to approach this not by sacrificing readability, but by having a dependency injection model that’s careful to not make unnecessary copies of objects, and by organizing my code to minimize any non-essentials calls when particular hooks are invoked.
• WordPress itself will try to thwart you in writing Agile plugins, as there is nothing Agile in how WordPress itself is written (this is not a criticism of its creators, as only since PHP 5 could you even really consider an object oriented approach with PHP, and WordPress has been around since long before PHP 5 became commonly deployed). A key decision is how far you want to go in isolating your code from direct dependencies on WordPress functions and classes (as they are not written against any interfaces). The more direct your dependencies, the harder it is to unit test anything (and then you can’t use your code outside of WordPress either). There is also no consistent error handling – some WordPress functions may return a WP_Error object, or false, or null, or something else. None throw exceptions. You’ll need to investigate each individual function to find out what variable type its return value will be under different circumstances.

I hope to get SimpleTest for WordPress up at wordpress.org soon, so I’ll have more to say about testing in WordPress next.

Over the past several months I’ve been following Agile coding principles in my work, as described by Bob Martin and others (see, for example, Clean Code, Agile Software Development, and Growing Object-Oriented Software, Guided by Tests). Applying these principles in PHP presents some challenges. Applying them to WordPress plugin development presents even more challenges. This is the first in a series of posts on how I’m dealing with those challenges. My Google searches about PHP and Agile coding don’t turn up much, so I figure I can break some new ground 😉 . But I don’t consider myself an expert – feedback is welcome.

If you follow the Agile principle of small methods and small classes, your projects will consist of a large number of small files. In Java, this doesn’t matter much – the project is compiled before it’s deployed. But PHP is a scripting language, so it’s compiled on the fly. include and require statements can have a moderately expensive performance cost (see here and here). So what’s the best way to include your files for code readability, and for efficiency?

require_once vs require

I always use require_once to include a required file. I don’t use include, which allows the compilation to proceed even if the file is not found (a class file isn’t going to be optional). require_once has a reputation for being slower than require, but this benchmarking indicates otherwise (I wouldn’t consider that an exhaustive study, but it’s enough that I’m not going to lose sleep over it). This way I don’t have to worry if a file has been included somewhere else already. If you have lots of small classes, used in various places, require_once can save you from that headache.

Where to put your require_once calls

There are three options:

1. The simplest option, and worst from both a performance and code integrity perspective, is to include all your project’s class files in the initializing script. Dependencies between classes will not be evident to someone reading your code. That is, if you tried to use one of the classes in a different project, and that class has dependencies on other classes, the class would fail to compile unless you read through the class to find the dependencies and added them to your new calling script (and then you’d have to manage the dependencies in two places). It’s also unnecessarily wasteful. Any given execution path through your project likely only requires a subset of your classes, so there’s no reason to load them all for every single http request.
2. The second and most preferable option is to include all the class files a given class depends on at the top of that class file (before the class line). This is common practice in languages I’m familiar with. From a readability perspective, the dependencies are clearly stated at the top of the class file. From a performance perspective, you’re only loading what that class needs. And from a flexibility perspective, that class can now be used outside the context of any particular calling script (as long as you package it with what it depends on). Note that if you are doing dependency injection, you’ll be passing in already instantiated objects to the dependent class, so your file includes will be one step removed from the class where the objects are actually used. The simplest way to think about this is to include the class file dependencies wherever there are new calls on those classes.
3. A third option is something I was doing, until @speno showed me the error of my ways. I put the require_once statement for a class file on the line before calling new on that class, regardless of where I was in the code. There is no technical problem with doing this – PHP will read the class into memory – it doesn’t matter if you’re inside a method. I did it in pursuit of improved performance: dependent classes are loaded only exactly when they are needed. The problem, however, is that the dependencies are now buried in the code of the class, making the reader of your code do more work to find them (and risking missing them, which leads to bugs…). A tenent of Agile programming is that you optimize for readability first, and that you sacrifice that readability for performance only when there is a demonstrable performance problem to address. What I was doing is an example of premature optimization (and the “gut feeling” optimizations many of us do, like I did, often turn out to not be optimizations at all when you profile the actual performance).

If you do have performance problems, file includes are probably not the first place to look. Database queries are the more common culprit. But if you do need to reduce your hits on the filesystem, you should look at opcode cachers, such as APC (or, for WordPress, the W3 Total Cache plugin) before you contemplate writing hard to read and maintain God objects.

Note: this is a revised version of the original post.

Over the past several months I’ve been following Agile coding principles in my work, as described by Bob Martin and others (see, for example, Clean Code, Agile Software Development, and Growing Object-Oriented Software, Guided by Tests). Applying these principles in PHP presents some challenges. This is the first in a series of posts on how I’m dealing with them. My Google searches about PHP and Agile coding don’t turn up much, so I figure I can break some new ground 😉 . But I don’t consider myself an expert – feedback is welcome.

If you follow the Agile principle of small methods and small classes, your projects will consist of a large number of small files. In Java, this doesn’t matter much – the project is compiled before it’s deployed. But PHP is a scripting language, so it’s compiled on the fly. include and require statements can have a moderately expensive performance cost (see here and here). So what’s the best way to include your files efficiently?

require_once vs require

I always use require_once to include a required file. I don’t use include, which allows the compilation to proceed even if the file is not found (a class file isn’t going to be optional). require_once has a reputation for being slower than require, but this benchmarking indicates otherwise (I wouldn’t consider that an exhaustive study, but it’s enough that I’m not going to lose sleep over it). This way I don’t have to worry if a file has been included somewhere else already. If you have lots of small classes, used in various places, require_once can save you from that headache.

Where to put your require_once calls

The simplest option, and worst from a performance perspective, is to include all your project’s class files in the initializing script. Any given execution path through your project likely only requires a subset of your classes. Loading up all of them for every single http request is unnecessary and may impact performance. It’s also problematic in that dependencies between classes are not evident to someone reading your code. That is, if you tried to use one of the classes in a different project, and that class has dependencies on other classes, the class would fail to compile.

A second option is to include all the class files a given class depends on at the top of that class file (before the class line). This is better, and in PHP code I’ve seen by others, it seems to be a fairly standard practice. It’s also good to see all the dependencies clearly stated at the top of the class file.

A third option is what I’ve been doing, which is to put my require_once statement for a class file on the line before I call new on that class. For example:

class Shashin {

    ....

    public function validateShortcode($shortcode) {
        require_once('Shortcode/ShashinShortcodeValidator.php');
        $validator = new ShashinShortcodeValidator($shortcode);
        $validatorResult = $validator->run();

        ....
    }

    ....
}

I used to think this approach would cause PHP to think the included class was a child of the function where it was included, but that is not the case. The class is simply read and put into memory.

The potential downside of this approach is that you have to read through the file to see the dependencies, but so far I haven’t had a problem with that, because my classes are small 😉 . I like it because I’m making PHP hit the filesystem only when necessary – you don’t always need every possible dependent class for every execution path through a given class. Sometimes there is a trade-off between code readability and performance, but in this case, the impact on readability feels trivial.

In the case of Shashin (my WordPress plugin I’m currently updating), I have a main class where the various WordPress hooks are invoked (which in my experience is the only rational way to deal with WordPress). Therefore, that main class acts as a controller for the application, and ends up including almost all the other classes at some point, from within its various methods. So I do not want to just include them all at the top of the file, as only a handful are needed for any given hook.

The one case where I do include a dependent class at the top of a class file, is when it is a concrete class implementing an abstract class (or interface). I’ll include the abstract class at the top of the concrete class. There’s no reason to make the caller of the concrete class keep track of the dependency on the abstract class.

Lastly, if you do have performance problems, file includes are probably not the first place to look. Database queries are the more common culprit. But if you do need to reduce your hits on the filesystem, you should look at opcode cachers, such as APC before you contemplate writing hard to read and maintain God objects.

Something that set today apart from yesterday is that I had opportunities to talk with several of the speakers. When I arrived, Chet Hendrickson and Ron Jeffries were sitting by themselves at a breakfast table, so I joined them. They were having a conversation about human nature, and whether it doomed us all to failure (heavy stuff for 8AM). By the end of the conversation we were discussing doing Agile in University settings… and unusual experiences with plumbing.

Slide from Eduaro Jezierski’s Philly ETE talk: Architecture and Agility with Lives at Stake

Slide from Eduaro Jezierski's Philly ETE talk: Architecture and Agility with Lives at Stake28-Apr-2011 11:59, Canon Canon PowerShot SD780 IS, 5.8, 17.9mm, 0.05 sec, ISO 640

View at Picasa

 

I had lunch with Eduardo Jezierski, who just before lunch gave a riveting presentation: Architecture and Agility with Lives at Stake. He had many stories to tell. One was about developing, in real-time, a system to save lives after the earthquake in Haiti. He had a product owner in a tent at the airport in Haiti, with a low-bandwidth connection, tweeting user stories as he received calls and people came to him with information. He was right near the runaway for days, with C-130s constantly landing with supplies and taking off again. They were able to broadcast SMS messages to people with cell phones, telling them to text a specific number if they were trapped or injured and needed help, or knew someone who did. His team was developing a system to get those incoming messages into an RSS feed that could be accessed by local responders. It was also fed back to the Haitian diaspora in the US, where volunteers who knew the neighborhoods used Google Maps to help pinpoint locations where people were trapped, based on descriptive information. He said in situations like this, they’ve had to get help for their programmers who develop post traumatic stress disorder. When you receive a message like “I’m pregnant, I’m trapped in a collapsed building, and I’m hurt” and you’re trying to develop code in real time to get that message where it needs to go… well, you can get just a bit stressed. I was fascinated by the work his organization did in Haiti and elsewhere. They have often turned into the facilitators between governments, NGOs, and local communities, as they try to leverage IT to solve local health problems and address emergencies.

Slide from Camille Bell’s Philly ETE talk: Adapting Agile Practices to Fit Your Situation

Slide from Camille Bell's Philly ETE talk: Adapting Agile Practices to Fit Your Situation28-Apr-2011 15:19, Canon Canon PowerShot SD780 IS, 5.8, 17.9mm, 0.05 sec, ISO 400

View at Picasa

 

The other talks I attended were also good. But it’s past my bedtime, so I’ll skip summarizing them for now, and close by mentioning I attended my first Philly WordPress Meetup after the conference. It was a great group. I met the organizer, Brad Williams, and we talked about the next Wordcamp Philly this Fall. It’s still a ways off, but I may give a talk about my current experiences with applying Agile coding practices to WordPress plugin development.

A slide from Say Yes or Say No? What to Do When Faced with the Impossible

A slide from Say Yes or Say No? What to Do When Faced with the Impossible27-Apr-2011 12:12, Canon Canon PowerShot SD780 IS, 5.8, 17.9mm, 0.05 sec, ISO 500

View at Picasa

 

The first day of the Philadelphia Emerging Technologies for the Enterprise conference was great. Molly Holzschlag‘s Keynote gave us some peace, love, and understanding, at least as far as web standards are concerned (she even made references to the Age of Aquarius…). After that, it was a challenge to choose from among the simultaneous sessions throughout the day. I’m there not just for me, but also for my team at SOMIS. They have a variety of interests, so I made sure to attend at least one session on each major theme of the conference (Agile, mobile design, management, languages, and infrastructure).

I’ll focus on my favorite talk, by Chet Hendrickson and Ron Jeffries, who years ago helped define many Agile practices. They presented A Retrospective Rant: 15 Years in the “Agile” Business. They’re both pleased that Agile has become a success and has gone mainstream, but also unnerved that it’s so mainstream that the stodgy Project Management Institute is going to offer scrum certifications (imagine Emperor Palpatine realizing the rebels actually aren’t so bad, and joining forces). Their presentation was very lively, and at times reminded me of Abbot and Costello, as they constantly played off each other. They focused on the most common failings they’ve seen in teams that are trying, but not fully succeeding, with Agile:

• No shippable product increment at the end of an iteration. Chet’s advise: think of every sprint as if it were the last one (not in the sense of stress, but in the sense of focusing on developing complete, functioning, tested features). The idea is that, if you had to go live, you could do so at the end of any sprint (you wouldn’t have the complete feature set, but every feature that you’ve worked on is complete).
• Too many defects: these are “negative features” that handicap the product owner’s ability to plan. They argued for a defect count of no greater than 1. They gave a couple real-life examples of (non-Agile) companies with backlogs of as many as 1,000 defects. Even after stopping all other work for months to fix the defects, after they fixed the 1,000, they had discovered almost 1,000 more.
• Failure to integrate: individual features are not integrated into a functioning set of features before the sprint ends
• System designs that degrade over time: architecture requires constant adjustment as the system evolves
• Partitioned teams: when team members handoff to each other instead of working together, misunderstandings and mistakes proliferate
• Failure to prioritize: saying yes to every project leads to multi-tasking, which leads to frequent context switching for the team, which results in everything taking longer and more mistakes. Work on as few projects as possible at the same time – ideally one.

The other talks I attended were all very good:

• Say Yes or Say No? What to Do When Faced with the Impossible. This took Jeffries and Hendrickson’s point about failing to prioritize projects, and went in-depth on its terrible consequences, and what you can do to prevent it.
• Building Native Apps with HTML, CSS, and JavaScript: a great hands-on presentation about developing for mobile applications. We saw working examples and code for CSS 3 transformations, transitions, and animations, jQTouch, Phonegap, and weinre
• Are You Satisfied With Your Tests? This was a great talk on what I’d call “code smells” for tests – that is, bad habits in testing that you should kick: slow tests, blase attitude towards failing tests, over-mocking, unnecessary complex object builds, gratuitous use of the database, overly “meta” tests, and testing private methods. Here are another attendee’s complete notes.
• Polyglot persistence for Java developers – moving out of the relational comfort zone: the first half of the talk was a comparison of 3 major noSQL databases: Redis, Cassandra, and MongoDB. The last half focused on using them with Java Spring, and assumed a knowledge of Spring that I didn’t have, so I cut out early, but the first part was very informative.

I’m looking forward to Day 2!