If you need to make use of an external program from within a PHP script, then this essay is for you. My example script is for managing an sftp connection (using OpenSSH), but the principals can be applied to any interaction that requires communication between your script and an external process.

One of the Penn Medical School’s business partners recently stopped allowing ftp connections to their servers for retrieving data files. They required us to switch to sftp (secure ftp). Those providing services for transferring sensitive data files over the internet have been steadily moving from ftp to sftp over the past couple of years, and from what I can see, the pace is accelerating. This poses a programming challenge if you have scripts that automate your ftp needs, as they’ll need to be re-written for sftp. This is not a trivial undertaking, especially if you’re programming in PHP. You can’t just swap out your PHP ftp function calls with sftp equivalents. Actually, you can, but you probably don’t want to, as you would have to upgrade to PHP 5 (adoption of which has been very slow across the PHP community) and you would have to install the PECL/ssh2 library, which – as noted on php.net – currently has no stable version.

So we had to roll our own sftp solution, which required using PHP’s program execution functions. The php.net documentation is good on this topic, but much of it is fully comprehensible only if you already know what you’re doing (this isn’t a criticism – it’s a documentation site after all, not a tutorial site). This annotated sample script will help you get started if you’re new to PHP’s program execution functions.

#!/usr/local/bin/php
<?php

$keyPath = 'path/to/your/ssh_key';
$login = 'your_username';
$server = 'your_sftp_server';
$connectionString = "Connecting to $server...\n";

$childPipes = array(
    0 => array("pipe", "r"), // stdin is a pipe that the child will read from
    1 => array("pipe", "w"), // stdout is a pipe that the child will write to
    2 => array("pipe", "w"), // stderr is a pipe that the child will write to
);

# turning off password authentication will avoid getting a password prompt if
# the key fails for any reason
$connection = proc_open(
    "sftp -oPasswordAuthentication=no -oIdentityFile={$keyPath} {$login}@{$server}",
    $childPipes, $parentPipes);

if ($connection === FALSE) {
    print "Cannot connect to $server.\n";
    exit;
}

PHP’s proc_open is a fork by another name. The $childPipes array is for setting up the communication channels from the child process perspective, and proc_open will set $parentPipes to a corresponding set of communication channels from the parent process perspective. Looking at the definition of $childPipes, the logic may seem backwards at first, but it’s not. For example, the parent process will write to the child’s stdin (element 0), which means the child process is reading that channel.

In the user contributed notes on the php.net proc_open page, most folks write out stderr to a file. But for our sftp script we need to see what’s coming through on stderr, so we’re not directing it to a file.

For establishing the connection, we turn off password authentication, which means we won’t get a password prompt if the key authentication fails. This is important, since the script cannot see or respond to such a prompt (the prompt goes directly to the terminal, so you can’t see it on stdin or stdout; you could see it if you want to do TTY buffering, but let’s not go there…).

# The "connecting..." message is written to stderr. Make sure there's nothing
# besides that in stderr before continuing.
$error = readError($parentPipes, TRUE);
sleep(3);
$error .= readError($parentPipes);

if ($error != $connectionString) {
    fclose($parentPipes[0]);
    fclose($parentPipes[1]);
    fclose($parentPipes[2]);
    $closeStatus = proc_close($connection);
    print $error;
    print "proc_close return value: $closeStatus\n";
    exit;
}

I don’t know if this is typical, but the sftp server we’re connecting to returns the “connecting…” welcome message on stderr (we’re reading stderr with a custom function named readError, which we’ll get to below). Having this message on stderr is problematic, since it’s not really an error message. An actual connection error, such as having a bad key, will come through on stderr after the “connecting…” message. This means we first look for the “connecting…” string (the TRUE argument to readError turns blocking on, so we’ll wait for it to appear – more on this below in the readError function), and then we have no choice but to sleep for a few seconds, to see if anything else comes through on stderr. And finally, to see if there was anything in stderr besides the “connecting…” message, we have no choice but to analyze the string 🙁 . This is an ugly solution, but dealing with stderr is difficult, since you never know when an error may or may not appear.

If we detect an error, we close the pipes before closing the connection. This is important for avoiding the possibility of a deadlock.

# gets us past the first "sftp>" prompt
$output = readOut($parentPipes);

After logging in, we’ll get an “sftp>” prompt on stdout. We’ll read from stdout to get past this prompt, using the custom function readOut (which is defined below).

# Get the directory listing and print it
writeIn($parentPipes, "ls -l");
$output .= readOut($parentPipes);
$error = readError($parentPipes);

if (strlen($error)) {
    fclose($parentPipes[0]);
    fclose($parentPipes[1]);
    fclose($parentPipes[2]);
    $closeStatus = proc_close($connection);
    print $error;
    print "proc_close return value: $closeStatus\n";
    exit;
}

print $output;

# close the sftp connection
writeIn($parentPipes, "quit");
fclose($parentPipes[0]);
fclose($parentPipes[1]);
fclose($parentPipes[2]);
$closeStatus = proc_close($connection);

if ($closeStatus != 0) {
    print "proc_close return value: $closeStatus\n";
}

This code just demonstrates getting a directory listing (using the custom function writeIn), printing it, and then closing the connection. You can use this as a template for any sftp commands you want to run.

function readOut($pipes, $end = 'sftp> ', $length = 1024) {
    stream_set_blocking($pipes[1], FALSE);

    while (!feof($pipes[1])) {
        $buffer = fgets($pipes[1], $length);
        $returnValue .= $buffer;

        if (substr_count($buffer, $end) > 0) {
            $pipes[1] = "" ;
            break;
        }
    }

    return $returnValue;
}

readOut loops over the stdout pipe until it sees an “sftp>” prompt, which is how we know that the server has finished writing to stdout. Note that we’ve turned off stream_set_blocking. This lets us define our own controls for reading from the stdout stream. In this case, we want readOut to return when the server has finished responding to a command. The best marker for that is the appearance of the “sftp>” after it finishes processing a command, so we set the while loop to break when it sees the prompt.

function readError($pipes, $blocking = FALSE, $length = 1024) {
    stream_set_blocking($pipes[2], $blocking);

    while (!feof($pipes[2])) {
        $buffer = fgets($pipes[2], $length);
        $returnValue .= $buffer;

        if ((!strlen($buffer) && $blocking === FALSE)
          || ($blocking === TRUE && substr_count($buffer, "\n") > 0)) {
            $pipes[2] = "" ;
            break;
        }
    }

    return $returnValue;
}

function writeIn($pipes, $string) {
    fwrite($pipes[0], $string . "\n");
}
?>

Reading from stderr is more complicated than reading from stdout, because 1. there is no equivalent to the “sftp>” prompt to let us know when the server is done writing to stdout, and 2. at any given time, there may or may not be an error. For most places in the script, we solve this problem by:

1. Calling readError after calling readOut. This is based on the supposition – which has proved reliable – that the server will finish writing to stderr by the time it has finished writing to stdout.
2. Setting stream_set_blocking to false. If we set it to true, the script would wait indefinitely for something to appear on stderr, and most of the time there will be nothing there.

The one situation when this approach doesn’t work is when we first log in, since the server writes to stderr before writing to stdout (as described above, it sends that “connecting…” message on stderr). In this case we turn blocking on, since we know the message is coming.

So far this script has been used with only one sftp server, so you may need to make some adjustments to make it work with your server (particularly with how it reads stderr when logging in). Also, I’d be interested in hearing from anyone who has a more elegant solution to handling the initial connection.

Stepping back from the specifics of sftp, the key thing to take away from this is that you will need to acquire a detailed knowledge of the behaviors of your external process so that your script can interact with it reliably. In particular, you need to test your handling of all the different kinds of errors the external process might throw at your script.