Branding and PR is an increasingly important factor in programming language viability and adoption. Visible instability in the core team is off-putting to large organizations who depend on long term reliability and support and only encourages them to look to languages and tools with more stable and professional core teams.

This RFC proposes that the PHP core team get ahead of the issue and introduce a logo, separate from the public facing project, to provide a sense of professionalism that is lacking. I humbly submit the DramaLlama as the superior candidate.

The Case For The DramaLlama

The Llama is a proud, capable animal that represent the best qualities of the core team. They are herbivorous pack animals that dutifully carry goods across the Andes, much like how PHP powers much of the internet. They can eat almost any vegetation, need little water, and even their excrement can be dried and burned as fuel!

Beyond the comparisons to the actual animal, a side profile of a Llama (of which I propose the logo use, rather than a front profile) with it’s solid body and long neck, represent PHP peaking and achieving new heights.

Risks

By not adopting a logo, the PHP core team risks losing the respect and trust of the end user community. However it could be argued that the core team has survived without this and could do so indefinitely.

Voting

An RFC of this magnitude requires unanimous agreement so as to send a strong signal to the wider community of it’s willingness to buckle down and tackle the real problems PHP faces, rather than executing proxy wars.

The project was generally fairly simple, 3 columns scrolling infinitely in alternate directions. The individual pictures, or “chips” as I called them were to do a 3D rotate transition to keep the visual appeal up.

I covered all the usual suspects to maximize performance, I used as many CSS transitions as possible to harness the GPU, keept everything asynchronous as possible, kept the DOM cleaned up, use requestAnimationFrame. However the biggest problem I encountered was getting the scroll smooth.

Initially the page was fully dynamic and my anmiation code used the scrollTop property (while inserting new chips as needed to prevent stoppage). That proved problematic and terrible performance-wise.

Next I tried inserting chips in the top of their column container with a negative margin and slowly animating the first chip to a margin of 0 (effecitvely constantly pushing all other chips down). This also posed problems and I couldn’t approach what looked like a fluid animation. It would always do a slight, maybe 2-3ms stutter when a new chip was introduced and there was nothing I could do to eliminate it. I tried optimizing my DOM interaction to use DOM DocumentFragment, dropping all jQuery code and going full-native.

Finally I moved to using absolutely positioned chips that I tracked one by one in my animation loop. However, even this was insufficient and I was still getting a stutter.

After much forehead-meets-desk time, I remembered a conversation I had with Tim Caswell a long while back (revolving around the HP TouchPad) where he told me about an interesting quirk of WebKit that turned out to be the cause of my problems:

To make my display flexible I was using the getBoundingClientRect method to calculate when a chip rolls off-screen so I can either remove it from the DOM and insert a new chip at the beginning (as well as to calculate initial position).

The problem is WebKit likes to recalculate the layout of the dom pretty much every single time you use something similar to getBoundingClientRect. (Even getting the window’s innerHeight/innerWidth will force a recalculation). The stutter was coming from WebKit constantly recalculating the layout!

So, a quick refactor to use static, precalculated dimensions for the chips (calculated once on window resize/document load) and BOOM, performance issues gone. So, lesson learned?

All calls to get any calculated dimension from the DOM should be cached or avoided.

Many CLI scripts for web projects consist of just a static .php file which works fine but grow unweildy over time. Thankfully, the aforementioned Symfony Console component is released as a decoupled standalone that can be installed and setup easily and provide us with structure and organization (and some powerful features).

Setup

The Symfony Console component can be found on GitHub at github.com/symfony/console. While installing by hand is doable, I much prefer using Composer to handle my dependencies. If you haven’t used Composer before, I suggest following the Getting Started documentation.

To start things off, we’ll need to setup, or modify, our composer.json file:

1
2
3
4
5
6
7
8

{
    "require": {
        "symfony/console": "2.1.*"
    },
    "autoload": {
        "psr-0": { "": "src/" }
    }
}

And then update our dependencies:

1

$ php composer.phar update

Now that we have Symfony Console and all it’s dependencies downloaded and ready, we’ll need to setup our actual console executable:

1
2
3
4
5
6
7
8
9
10
11
12
13

#!/usr/bin/env php
<?php

date_default_timezone_set('UTC');

set_time_limit(0);

(@include_once __DIR__ . '/../vendor/autoload.php') || @include_once __DIR__ . '/../../../autoload.php';

use Symfony\Component\Console\Application;

$app = new Application('My CLI Application', '0.1.0');
$app->run();

Afterwards a simple chmod +x bin/console will make it executable and you’ll be ready to begin. You’ll notice we placed this in a file named simply console in the bin directory, this is purely a stylistic choice, name it whatever you want. Giving it a test whirl, you should see the output:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

$ bin/console
My CLI Application version 0.1.0

Usage:
  [options] command [arguments]

Options:
  --help           -h Display this help message.
  --quiet          -q Do not output any message.
  --verbose        -v Increase verbosity of messages.
  --version        -V Display this application version.
  --ansi              Force ANSI output.
  --no-ansi           Disable ANSI output.
  --no-interaction -n Do not ask any interactive question.

Available commands:
  help   Displays help for a command
  list   Lists commands

Commands

Syfmony Console commands are simply classes you override and inject into the application instance. The skeleton for any given command looks something like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

<?php
namespace MyApp\Console\Command;

use Symfony\Component\Console\Command\Command;
use Symfony\Component\Console\Input\InputInterface;
use Symfony\Component\Console\Output\OutputInterface;
use Symfony\Component\Console\Input\InputArgument;
use Symfony\Component\Console\Input\InputOption;

class TestCommand extends Command {
    protected function configure() {
        $this->setName("test")
             ->setDescription("Sample description for our command named test")
             ->setDefinition(array(
             ))
             ->setHelp(<<<EOT
The <info>test</info> command does things and stuff
EOT
             );
    }

    protected function execute(InputInterface $input, OutputInterface $output) {
        //...
    }
}

While we have created the class, we haven’t attached it to our console application. To do this we edit our bin/console file and add the follwoing code between the new Application(...) and the $app->run():

1
2
3
4
5
6
7
8

<?php
//...
$app = new Application('My CLI Application', '0.1.0');
$app->addCommands(array(
    new MyApp\Console\Command\TestCommand(),
));
$app->run();
//...

When we run our bin/console we’ll notice the output is now midly different:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

$ bin/console
My CLI Application version 0.1.0

Usage:
  [options] command [arguments]

Options:
  --help           -h Display this help message.
  --quiet          -q Do not output any message.
  --verbose        -v Increase verbosity of messages.
  --version        -V Display this application version.
  --ansi              Force ANSI output.
  --no-ansi           Disable ANSI output.
  --no-interaction -n Do not ask any interactive question.

Available commands:
  help   Displays help for a command
  list   Lists commands
  test   Sample description for our command named test

And we can even use the help command to describe our newly created test command:

1
2
3
4
5
6
7

$ bin/console help test
Usage:
 test


Help:
 The test command does things and stuff

Output

Syfmony Console applications direct all their output through the $output variable provided. What these output objects gives us is easy ANSI coloring utilizing XML-like tags. For example, a $output->writeln("<info>This will be green</info> This will be white"); will print the ecapsulated text in the “info” style which is green. You can add and override styles to your hearts content, just peruse the official documentation.

Input

CLI input comes in 2 flavors: Arguments and Options. In essence, Options are flags using the - or – operators and Arguments are your classic space separated values. To put it visually:

1

$ bin/consle --option=value argument

Symfony Console requires you to be strict and actively provide your definition in the configure() method. For example, we can setup an option and an argument:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

<?php
//...
    protected function configure() {
        $this->setName("test")
             ->setDescription("Sample description for our command named test")
             ->setDefinition(array(
                new InputOption('flag', 'f', InputOption::VALUE_NONE, 'Raise a flag'),
                new InputArgument('activities', InputArgument::IS_ARRAY, 'Space-separated activities to perform', null),
             ))
             ->setHelp(<<<EOT
The <info>test</info> command does things and stuff
EOT
             );
    }
//...

Here we have defined a --flag option which has no value expected (meaning –flag=value would be technically be illegal) and an array argument (which means an infinite series, e.g. bin/console activities1 activities2, etc). When we run our help command, you’ll notice our output has changed to reflect our definition:

1
2
3
4
5
6
7
8
9
10
11
12

$ bin/console help test
Usage:
 test [-f|--flag] [activities1] ... [activitiesN]

Arguments:
 activities  Space-separated activities to perform

Options:
 --flag (-f) Raise a flag

Help:
 The test command does things and stuff

Now when our command is actually running we can check whether or not a flag was issued with a simple $input->getOption('flag'); as well as we can get all of our activities with a $input->getArgument('activities');.

There are many modifiers to be used with both Arguments and Options. For example, you can have required Arguments, Options that require a value to be set, so on and so forth. You can view the official documentation to get an idea of the power available to you.

Integrating With Your Application

The bulk of power you gain from writing CLI apps comes from the ability to share code and resources with your main application.

Symfony Console does not provide an “out of the box” way to integrate with anything, however it is fairly trivial to roll in support on your own.

For example, say you were building a CLI app to complement your Silex web application. You have configured services and database connections you would like to share via the Silex DI container/application object.

A best practice would be to create a base command class that can accept your DI container and have your commands inherit from it:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

<?php
namespace MyApp\Console\Command;

use Symfony\Component\Console\Command\Command;
use Symfony\Component\Console\Input\InputInterface;
use Symfony\Component\Console\Output\OutputInterface;

use Silex\Application;

abstract class ContainerAwareCommand extends Command {
    /**
     * @var Silex\Application
     */
    protected $app;

    public function __construct(Application $app, $name = null) {
        parent::__construct($name);
        $this->app = $app;
    }
}

And update your bin/console file to include the PHP file responsible for creating and configuring the Application object and pass said Application object into the commands:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

#!/usr/bin/env php
<?php

date_default_timezone_set('UTC');

set_time_limit(0);

(@include_once __DIR__ . '/../vendor/autoload.php') || @include_once __DIR__ . '/../../../autoload.php';

//Provides $app, which is an instance of Silex\Application
require_once 'path/to/app/setup.php';

use Symfony\Component\Console\Application;

$cli_app = new Application('My CLI Application', '0.1.0');

$cli_app->addCommands(array(
    new MyApp\Console\Command\TestCommand($app),
));

$cli_app->run();

Another path is to contemplate overloading the actualy Symfony Console Application class. Doing this affords you the ability to override the help message as well as any global option definitions:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

<?php
namespace MyApp\Console;

use MyApp\Console\Command;
use Symfony\Component\Console\Application as BaseApplication;

use Silex\Application;

class Application extends BaseApplication {
    /**
     * @var Silex\Application;
     */
    protected $app;

    public function __construct(Application $app) {
        $this->app = $app;
        parent::__construct('My CLI Application', '0.1.0');
    }

    public function getDefaultCommands() {
        $commands = parent::getDefaultCommands();

        $commands[] = new Command\TestCommand($this->app);

        return $commands;
    }
}

Which has another added benefit of keeping your bin/console file simple:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

#!/usr/bin/env php
<?php

date_default_timezone_set('UTC');

set_time_limit(0);

(@include_once __DIR__ . '/../vendor/autoload.php') || @include_once __DIR__ . '/../../../autoload.php';

//Provides $app, which is an instance of Silex\Application
require_once 'path/to/app/setup.php';

use MyApp\Console\Application;

$cli_app = new Application($app);
$cli_app->run();

Environment

One thing to be aware of, most web applications these days are aware of environments when it comes to what configurations to use. The Symfony Console Application object allows you to manually create your $input and $output options, which allows us to perform a neat little trick and gain early access to the option parsing.

If you look at lines 19 and 20 below you can see us checking the --env option to see if it’s set, if not using a default environment variable MYAPP_ENV from the shell or our fallback ‘dev’ environment designation. We then just go ahead and forward our newly created $input and $output objects through our run() method as shown on line 23.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

#!/usr/bin/env php
<?php

date_default_timezone_set('UTC');

set_time_limit(0);

(@include_once __DIR__ . '/../vendor/autoload.php') || @include_once __DIR__ . '/../../../autoload.php';

//Provides $app, which is an instance of Silex\Application
require_once 'path/to/app/setup.php';

use MyApp\Console\Application;
use Symfony\Component\Console\Input\ArgvInput;
use Symfony\Component\Console\Output\ConsoleOutput;

$input = new ArgvInput();
$output = new ConsoleOutput();

//Determine Environment
$env = $input->getParameterOption(array('--env', '-e'), getenv('MYAPP_ENV') ?: 'prod');
$app['environment'] = $env;

$cli_app_ = new Application('My CLI Application', '0.1.0');
$cli_app_->run($input, $output);

As an added bonus, this same concept can be applied to creating your own Output class. The Composer project actively makes use of this to extend in the ability to do overwrites in the console (the technique used for it’s fancy progress indication).

However, just a tip: In the course of working with flex boxes, you will find that if you given a parent container a flex value, the children can still stretch it out. For example, a long paragraph will push its parent wider and wider to accommodate unless you give it a static width (which kinda defeats the purpose of using flex boxes really).

A nifty trick, however, is if you give an element with a defined box-flex a width: 0px;, the sizing algorithm will ignore said element’s children when sizing the element.

Here’s an example of automatic vertical columns:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

<div id="wall">
    <div id="col1" class="column top">

        <div class="entry">
            Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aliquam eget tristique velit. In ut ligula nibh, a pulvinar lorem. Nam sed elit eget tellus vestibulum
        </div>

    </div>
    <div id="col2" class="column bottom">

        <div class="entry">
            Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aliquam eget tristique velit. In ut ligula nibh, a pulvinar lorem. Nam sed elit eget tellus vestibulum
        </div>

    </div>
    <div id="col3" class="column top"></div>
    <div id="col4" class="column bottom"></div>
</div>

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

#wall {
    display: -webkit-box;
    -webkit-box-orient: horizontal;

    position: absolute !important;
    top:0;
    right:0;
    bottom:0;
    left:0;

    overflow: hidden;
}

#wall .column {
    display: -webkit-box;
    -webkit-box-sizing: border-box;

    -webkit-box-orient: vertical;

    -webkit-box-flex: 1;
    width:0px;

    border-right: 1px solid #333;
}

Our media processor when finished converting videos and creating thumbnails of images publishes said files to our CDN. Only a few hours before our event and our CDN was failing probably 80% of the SSL handshakes for our requests to publish files. There is no need to name names as it was a difficult and obscure bug to diagnose in an extremely short amount of time. We needed to get it working, our CDN provider wasn’t going to be able to resolve the problem in the extremely short timeframe we required, so the decision was clear: we needed to switch providers for the new content.

Normally this is a 2 week job, switching API calls, testing, finding all the occurrences (we process both videos and photos in different styles of batches), etc. We were able to complete it and have it live in under 1.5 hours. Why? We (and really I mean my coworker as this decision was made before I came on) made som really damned good architecture decisions. Primarly:

1. We used OSS, primarily Zend Framework which had well written API libraries for our new CDN

2. All file system operations were abstracted to adapters that all inherited a “FileStore” base class

So, create a file store for our new CDN using the well written, off the shelf API. Change 4 lines of XML configuration, deploy, and we were in business.

Had my coworker succumbed to laziness or listen to some people who claim “oh you’re just making it too complicated” or “just get it up quickly, damn the design decisions” we would have been sunk. Instead he took the time to utilize the adapter pattern (even though it probably added an extra 2 days to his total coding time) despite all signs pointing to us never needing to switch CDN’s (and certainly not within a 2 hour timeframe).

So let this be a lesson to us all. Abstraction and design patterns sometimes feel like an “enterprisey overcomplication” but they aren’t there for everyday needs. They exist for those days when a service provider gets hit by an obscure bug and you have to swap out a component under extreme time schedules. So suck it up, use them, and thank yourself when you find yourself in such a situation.

The code used the more-compatible-with-SQL-Server-2000 technique of modifying the query to SELECT TOP (offset + limit), reverse the ORDER BY clause and SELECT TOP (limit), then finally reversing the returned dataset.

As ugly as this technique is, it works. The problem is it requires an extreme amount of intelligence or an extreme amount of simplicity in the query in order for an automated system like Doctrine to be usable. The biggest caveat with this technique is good goddamned luck paging your query if it doesn’t have an ORDER BY. And sometimes queries that are complex enough break the modified Zend_Db code.

There exists an easier MSSQL paging technique. Using features first available in SQL Server 2005, with only 1 subquery you can mimic MySQL’s LIMIT clause with ease.

Basically, Microsoft provided the following special feature to determine the row number in your final resultset:

1

SELECT Row_Number() OVER (ORDER BY column) AS RowIndex FROM table

Notice the OVER (ORDER BY column) segment? This is provided as this query will most often be used in a subquery. Given that MSSQL does not allow ORDER BY statements in subqueries, just move them into the OVER (...) section.

To borrow Ralph Varjabedian’s example, the following MySQL query:

1

SELECT * FROM users LIMIT 15, 15

Becomes functionally equivalent to the following MSSQL query:

1
2
3
4
5
6
7
8
9
10
11
12
13

SELECT
    *
FROM
    (
    SELECT
        Row_Number() OVER (ORDER BY userID) AS RowIndex,
        *
    FROM
        users
    ) AS sub
WHERE
    sub.RowIndex > 15
    AND sub.RowIndex <= 30

Now all I needed to do was add the ability for Doctrine to parse a generated MSSQL query and reform it like the one above!

I’ve provided a copy of the Doctrine connection adapter I wrote. Simply add the following line to wherever you setup Doctrine:

Please note: This was the result of about 6 hours of hacking today. There are certainly places in the code where it can be more robust or improved, especially in my little parser. Use at your own risk (though I have yet to encounter any errors in my application).

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284

<?php
class CaseMan_Doctrine_Connection_Mssql extends Doctrine_Connection_Mssql
{
    /**
     * @var string $driverName                  the name of this connection driver
     */
    protected $driverName = 'Mssql';

    /**
     *
     * @var boolean $is2005OrBetter             cached result determining if server is SQL Server 2005 or better
     */
    protected $is2005OrBetter;

    /**
     * the constructor
     *
     * @param Doctrine_Manager $manager
     * @param PDO $pdo                          database handle
     */
    public function __construct(Doctrine_Manager $manager, $adapter)
    {
        parent::__construct($manager, $adapter);
    }

    /**
     * Adds an adapter-specific LIMIT clause to the SELECT statement.
     *
     * @param string $query
     * @param mixed $limit
     * @param mixed $offset
     * @return string
     */
    public function modifyLimitQuery($query, $limit = false, $offset = false, $isManip = false)
    {
        if ($limit <= 0 && $offset <= 0)
            return $query;

        if( !$this->is2005OrBetter() ) //Not at least 2005
            return parent::modifyLimitQuery ($query, $limit, $offset, $isManip);

        //SETUP FIELD ALIASES
        $inner_query_name = '_inner_query_';
        $row_count_name = '_inner_query_row_count_';

        //PREPARE TOKENIZER REGEXES
        $escaped = "\\\.";
        $sections = "SELECT|FROM|WHERE|GROUP[ ]+BY|HAVING|ORDER[ ]+BY";
        $open_delimiters = "[\[\(]";
        $close_delimiters = "[\]\)]";
        $string_delimiters = "[\"\']";

        //TOKENIZE QUERY
        $_split_query = preg_split(
            "#({$escaped}|{$sections}|{$open_delimiters}|{$close_delimiters}|{$string_delimiters})#i",
            trim($query),
            -1,
            PREG_SPLIT_DELIM_CAPTURE | PREG_SPLIT_NO_EMPTY
        );

        $query_parts = array();
        $_state = 'BEGIN';
        $_current_part = 'SELECT';
        $_stack = array();
        while( $_token = array_shift($_split_query) )
        {
            switch( $_state )
            {
                case 'BEGIN':

                    if( trim(strtoupper($_token)) == 'SELECT' )
                    {
                        $query_parts['SELECT'] = '';
                        $_current_part = 'SELECT';
                        $_state = 'SECTION';
                    }
                    else
                    {
                        throw new Doctrine_Exception("Invalid query passed to modifyLimitQuery, must begin with SELECT");
                    }

                    break;
                case 'SECTION':

                    if( preg_match("#^({$sections})$#i", trim($_token)) )
                    {
                        $_section = strtoupper(trim($_token));

                        $query_parts[$_section] = '';
                        $_current_part = $_section;
                        $_state = 'SECTION';
                    }
                    else
                    {
                        $query_parts[$_current_part] .= $_token;

                        if( preg_match("#^({$string_delimiters})$#i", trim($_token)) )
                        {
                            array_push($_stack, trim($_token));
                            $_state = 'STRING';
                        } else if( preg_match("#^({$open_delimiters})$#i", trim($_token)) )
                        {
                            array_push($_stack, trim($_token));
                            $_state = 'DELIMITED';
                        }
                    }

                    break;
                case 'DELIMITED':

                    $query_parts[$_current_part] .= $_token;

                    if( preg_match("#^({$close_delimiters})$#i", trim($_token)) )
                    {
                        $_prev_delimiter = array_pop($_stack);
                        switch( trim($_token) )
                        {
                            case ']':
                                if( $_prev_delimiter != '[' )
                                    throw new Doctrine_Exception("Mismatched ]");
                                break;
                            case ')':
                                if( $_prev_delimiter != '(' )
                                    throw new Doctrine_Excpetion("Mismatched )");
                                break;
                            default:
                                trigger_error("FATAL ERROR: UNRECOGNIZED CLOSE DELIMITER TOKEN '{$_token}' IN " . __CLASS__ . '::' . __METHOD__, E_USER_ERROR);
                        }

                        if( count($_stack) == 0 )
                            $_state = 'SECTION';
                    }
                    elseif( preg_match("#^({$open_delimiters})$#i", trim($_token)) )
                    {
                        array_push($_stack, trim($_token));
                    }

                    break;
                case 'STRING':

                    $query_parts[$_current_part] .= $_token;

                    if( preg_match("#^({$string_delimiters})$#i", trim($_token)) )
                    {
                        if( trim($_token) == end($_stack) )
                        {
                            array_pop($_stack);
                            $_state = 'SECTION';
                        }
                    }

                    break;
            }
        }
        unset($item, $_current_part, $_token, $_stack, $_state, $_section);

        //DIVIDE UP THE SELECT STATEMENT TO PREPARE TO INSERT THE ROW_NUMBER() SELECT FIELD
        $_select_split = array_map('trim', preg_split(
            "#^(DISTINCT|)[ ]*(TOP[ ]+[0-9]+|)#i",
            trim($query_parts['SELECT']),
            -1,
            PREG_SPLIT_DELIM_CAPTURE | PREG_SPLIT_NO_EMPTY
        ));

        $_select_details = array_pop($_select_split);

        $query_parts['SELECT'] = array_merge((array)implode(' ', $_select_split), (array)$_select_details);
        unset($_select_split, $_select_details);



        //SETUP OUTER QUERY SELECT STATEMENT
        $outer_select = array();
        foreach( array_map('trim', explode(',', end($query_parts['SELECT']))) as $_select )
        {
            $matches = array();
            if( preg_match('#AS[ ]+(?<alias>.*)$#i', $_select, $matches) )
            {
                $outer_select[] = "[{$inner_query_name}]." . trim($matches['alias']);
            }
            else if( preg_match('#^(?<table>(\[[^\]]+\]|[^\.]+)\.|)(?<field>.*)#i', $_select, $matches) )
            {
                $outer_select[] = "[{$inner_query_name}]." . trim($matches['field']);
            }
        }

        //SETUP ROW_COUNT OVER() SEGMENT
        if( isset($query_parts['ORDER BY']) )
        {
            $row_count_select = "Row_Number() OVER (ORDER BY " . $query_parts['ORDER BY'] . ") AS [{$row_count_name}]";
            unset($query_parts['ORDER BY']); //ORDER BY NOT ALLOWED IN SUBQUERY, OVER(...) TAKES ITS PLACE
        }
        else
        {
            $row_count_select = "Row_Number() OVER (ORDER BY (SELECT 0)) AS [{$row_count_name}]";
        }

        $query = implode(' ', array(
            'SELECT',
            count($query_parts['SELECT']) > 1 ?
                $query_parts['SELECT'][0] . ' ' . implode(', ', array(
                    $row_count_select,
                    $query_parts['SELECT'][1]
                )) :
                implode(', ', array(
                    $row_count_select,
                    $query_parts['SELECT'][0]
                )),
        ));

        unset($query_parts['SELECT']);
        foreach( $query_parts as $section => $parameters )
            $query .= ' ' . $section . ' ' . $parameters;


        $outer_query = "SELECT " . implode(', ', $outer_select) . " FROM (" . $query . ") AS [{$inner_query_name}]";

        if( $limit || $offset )
        {
            $outer_where = array();
            if( $limit )
                $outer_where[] = "[{$inner_query_name}].[{$row_count_name}] <= " . ($limit + $offset);
            if( $offset )
                $outer_where[] = "[{$inner_query_name}].[{$row_count_name}] > " . $offset;

            $outer_query .= ' WHERE ' . implode(' AND ', $outer_where);
        }

        return $outer_query;
    }

    public function is2005OrBetter()
    {
        if( !isset($this->is2005OrBetter) )
        {
            $version = $this->getServerVersion();

            if( $version['major'] >= 9 )
                $this->is2005OrBetter = true;
            else
                $this->is2005OrBetter = false;
        }

        return $this->is2005OrBetter;
    }

    /**
     * return version information about the server
     *
     * @param bool   $native  determines if the raw version string should be returned
     * @return array    version information
     */
    public function getServerVersion($native = false)
    {
        if ($this->serverInfo) {
            $serverInfo = $this->serverInfo;
        } else {
            $query      = 'SELECT @@VERSION';
            $serverInfo = $this->fetchOne($query);
        }
        // cache server_info
        $this->serverInfo = $serverInfo;
        if ( ! $native) {
            if (preg_match('/([0-9]+)\.([0-9]+)\.([0-9]+)/', $serverInfo, $tmp)) {
                $serverInfo = array(
                    'major' => $tmp[1],
                    'minor' => $tmp[2],
                    'patch' => $tmp[3],
                    'extra' => null,
                    'native' => $serverInfo,
                );
            } else {
                $serverInfo = array(
                    'major' => null,
                    'minor' => null,
                    'patch' => null,
                    'extra' => null,
                    'native' => $serverInfo,
                );
            }
        }
        return $serverInfo;
    }
}

If you’re using Zend Framework and NetBeans, you may be like me and bemoaning the lack of code completion in your Zend Views. As you may know, essentially what Zend_View does is includes your view within a method that belongs to a Zend_View object. This gives your view some nice variable encapsulation as well as access to the $this object (which is how Zend_View provides access to all the ViewHelpers and other functions).

Unfortunately NetBeans can’t figures this out (such is the problem with static analysis on a dynamic language) without help. If you’ve been using NetBeans and its code completion you’ll have already noticed that the PHPDocumentor syntax for @var or @return is how NetBeans figures out much of its code completion information, but that syntax doesn’t work in a view script.

Thanks to Mystic at tiplite.com I now know that to have code completion in your Zend_View scripts in NetBeans, add the following to the top of your view script:

1
2

<?php
/* @var $this Zend_View */

Obviously if you’re using custom Zend_View objects you can pass in their class name instead.

Assuming you have access to the Doctrine CLI tool, the sequence you need is:

1. Make the changes to your YAML schema files
2. Run $ doctrine generate-migrations-diff to generate the migration deltas
3. Run $ doctrine migrate to determine if your migration deltas are working (a concern if you’re using SQL Server)
4. Run $ doctrine generate-models-yaml to update your models to reflect the status of your database

The reasoning behind this, and if I’m wrong someone can correct me, is the “generate-migrations-diff” task compares the differences between your YAML schema files and your current models. If you have updated your models before generating the diff, obviously Doctrine will find no difference and generate no migration delta.

My first ever professional conference (ignoring BarCamps) was last year’s ZendCon. I showed up in Santa Clara not knowing anyone except for a few brief appearances on #phpc (freenode).

What made ZendCon absolutely enjoyable was indeed what everyone is suggesting to you: the hallway track.

I could have easily attended sessions then hid in my room until the next day, yet I chose to go out to the social events and thanks to that decision, ZendCon ‘08 remains currently one of my fondest memories.

It’s one thing to sit down and listen to talks from famous PHP developers, those that I read about all the time (or read their books in many cases). It’s a whole other beast to find yourself sharing a beer with them in some random “English Pub” while getting your ass kicked at trivia (I still have the beer glasses that were given to me by the real winners).

And these events didn’t happen because I’m some modest uber-genius (I am by no means modest), I’m famous, or even particularly well respected (“He’s not stupid” is as gracious as a comment as I could want in a programming community).

They happened because someone yelled “Pub Crawl!” and, dammit, I was there.

They happened because I struck up conversations with random people.

They happened because I got in religious flame wars with random people. (Ask me about the time, thought not at ZendCon, where I launched into a tirade against CakePHP with who I later found out was the founder of CakePHP)

As it has been reiterated time and time again, this conference is not really about the sessions (it’s a success if there’s just 1 talk that you go to). It’s about the people you meet and the friendships you foster.

In summation, I have a list of goals for you, my loyal readers that are attending ZendCon:

1. Start a conversation with a random person in a hallway
2. Find and say hi to an author of one your PHP books. Tell them how awesome the book has been
3. Attend at least 1 UnCon session
4. Attend at least 1 post-session social (keep track on Twitter and ask your fellow attendees for info)
5. Find me and introduce yourself

Sure I knew the syntax (C based, not terribly hard), understood closures (LISP will do that to you), understood the prototype approach to Object Orientation (oddly enough playing with Python caused it to finally click).

Well around a month ago The Daily WTF posted one of their weekly programming puzzles.

Lo and behold, it’s the classic “Eight Puzzle” that I had to solve in LISP using several different algorithms!

Feeling particular motivated I decided to finally get around to really learning JavaScript (by learning I mean really knowing the language inside and out, like I know PHP and to a lesser extent Python).

The first thing I had to tackle was: Which algorithm would I choose? There are several to choose from, DFS (depth-first search), BFS (breadth-first search), Greedy, what have you. Well, a testament to my schooling, I do remember the most optimal I could remember was the A* algorithm.

The A not terribly complex once you look at it properly. Most examples discuss the A, like most navigation algorithms, as a tree structure which, while academically correct, never actually touches a tree data structure.

The A* algorithm works by examining the current node, the possible exit directions, and how much closer the new states after exit bring you to your goal. It arranges these new nodes in a priority queue and recurses on all the new nodes until such time that the current node is the goal node.

In simpler terms: examine the paths that bring you closer to the goal first.

The Eight-Puzzle is an excellent situation in which to first play with this algorithm as all the pieces we need for the algorithm are easy to conceptualize. It’s just a very simple navigation puzzle: How do we move the blank spot to it’s final destination (and making sure the other pieces are set too).

By pieces we need for the A* algorithm, I mean:

• Heuristic (or “weight” of a current path, how far are we from the goal, etc.)
• Expansion Function (from our current node, what are the paths we can take?)

The Expansion function is easy. It’s literally which way can you move the blank spot. If you’re in the exact center of the board, the expansion function will return UP, RIGHT, DOWN, LEFT. If you’re in the top left hand corner, the expansion function will return RIGHT, DOWN.

The Heuristic is a little trickier. If we knew the exact distance (number of moves) a certain state in the puzzle is to the finish, we would need a searching algorithm. Instead we estimate (hence this value being called a heuristic).

2 decent estimations would be counting the number of tiles that are not in their final position as well as calculating the manhattan distance between the blank space and it’s final spot. The manhattan distance is the actual “walking” distance as opposed to the straight line distance. Think of the grid street system that is famously used in New York City. While drawing a line from point A to point B may be 2.5_ish_ miles, in reality you walk maybe 4 miles as you must walk north about 2 miles and east about 2 miles.

Even better is combining the two values. As excellent as manhattan distance sounds for an estimation, you run into inconsistencies like an unsolved board where the blank spot is in the correct position being given more weight in searching than a board 1 step away from being solved with the blank spot only 1 block away from the correct (and final) position.

In Part 2 I will step away from the algorithm and step into JavaScript to build our basic utility function (namely creating and managing a board).

NOTE I never finished part 2 however my code can be found on github of the final solution.

Enter hMailServer, a free, full-featured SMTP/POP/IMAP server for Windows with a pretty, shiny GUI configuration interface. All I really want to do is setup SMTP for localhost only but one could easily use hMailServer as a production mail server.

To start things off, lets make sure our php.ini directives are setup correctly. By correct, I mean PHP needs to be looking for our SMTP server on my local machine (localhost) at the default SMTP port (25), and, just in case, set a default from address:

1
2
3
4
5
6
7

[mail function]
; For Win32 only.
smtp = localhost
smtp_port = 25

; For Win32 only.
sendmail_from = user@domain.tld

Looks good. Second step is to download and install hMailServer. This tutorial is working from version 5.2 build 356, but you should be able to use the latest stable or unstable version that you might desire.

Once you’re installed and configured, run the administrative application. If you click over to status you will notice a message to the effect of “You haven’t specified the public host name for this computer in the SMTP settings.” That means we have some more configuration to do!

Now mosey on over to Settings » Protocols » SMTP, navigate to the “Delivery of e-mail” tab. We set our “Local host name” to “localhost” and click “Save”.

Next, for securities sake (as well as to ensure we don’t have to waste time configuring account as this is only a dev machine), go to Settings » Advanced » IP Ranges » My Computer. Un-check the POP3 and IMAP checkboxes under the “Allow Connections” as we don’t intend to receive mail, nor use the IMAP protocol. Then, un-check everything under “Requires SMTP authentication” as we want our PHP applications to have full reign. Finally, click “Save”. Remember, this is a dev box.

UNDER NO CIRCUMSTANCES SHOULD YOU USE THESE SETTINGS ON A BOX THAT WILL BE EXPOSING ITSELF TO THE INTERNET OR ON A PRODUCTION SERVER! However, since our local machine will obviously have internet connectivity, lets remove all access from outsiders!

Navigate to Settings » Advanced » IP Ranges » Internet. Un-check EVERYTHING from “Allow connections” to deny outsiders any services, and, just in case, check EVERYTHING under “Require SMTP authentication” and click “Save”.

And voilà! We have a fully functioning SMTP server that our local PHP environment can use to send test email messages.

The nice thing about our administrative interface is if we go to Status, navigate to the “Logging” tab, and click “Start,” we can capture the dialogue between hMailServer and your PHP app when sending an email address (very useful for debugging).

I’M GOING TO BE SPEAKING AT ZENDCON 09!

I will be giving my talk on automated deployment techniques with Phing.

To say that I’m excited is quite an understatement. Nervous now probably tops excitement as now I have to make sure I give a damned good talk and not make Eli White look like a fool for having picked me for a slot.

Time to go bury my nose in Power Point and research!

Awkward date calculations if you don’t have the magical strtotime() function in PHP. strtotime() is able to do relativistic time conversions from the common “+1 hours” to the more complex (and more relevant) “Second Sunday March 0”.

Why do we do the ‘March 0’ in the above string to calculate the start boundary? Doing some tests with strtotime() reveals some behavior you should be aware of.

strtotime("March"); doesn’t actually give you the first of March, it gives you the current day in the month specified (date('D, F j, Y, g:i a', strtotime("March")); returns Tue, March 10, 2009, 12:00 am at the time of this post). Doing a similar test but substituting April for March results in the 10th of April (at 12 am).

Now in most cases doing a strtotime("March 1") will suffice (date('D, F j, Y, g:i a', strtotime("March 1")); results in Sun, March 1, 2009, 12:00 am). However, as you can tell, this month is going to be awkward because the first day of the month is a Sunday. strtotime(), when calculating the fuzzy “Second Sunday,” doesn’t include the current day. So calculating date('D, F j, Y, g:i a', strtotime("Second Sunday March 1")); will actually return the 3rd Sunday (Sun, March 15, 2009, 12:00 am).

So our solution is actually to take a step back and return the PREVIOUS day to the first day of March and then calculate the Second Sunday from there (since we know that starting from the next day will account for the first). Normally, dealing with the last day in February is a headache, BUT again thanks to PHP magic we don’t have to figure out if it’s February 28th or 29th, we merely do a “March 0” which steps us back a month (date('D, F j, Y, g:i a', strtotime("March 0")); returns Sat, February 28, 2009, 12:00 am). From there we can calculate the Second Sunday with ease and eventually determine that the second sunday is March 8th, which is confirmed by the above USNO website.

Applying these lessons to calculating the first Sunday in November we come up with the following snippit:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

<?php
$remove_hour = strtotime("Second Sunday March 0");
$add_hour = strtotime("First Sunday November 0");

$time  = time();

if( $time >= $remove_hour && $time < $add_hour )
{
    var_dump("Lost an hour");
}
else
{
    var_dump("Gained an hour");
}

Cheers!

The system spawns off a small script that manages mplayer/ffmpeg/lame into the background and holds on to the PID to track the conversion process. HOWEVER, there was a problem in the system where large files were being ignored and no error messages were popping up.

What ended up happening was I forgot to take into account PHP’s file upload behavior in the conversion process.

When PHP accepts a file upload it creates a temporary file (usually in /tmp/). The caveat is PHP frees (read: deletes) the temporary uploaded file at the end of script execution.

So, while the converters could convert a small file in the same encoding as the desired output, a large file (say, a 10mb MPEG) that needed to be converted to a different format (say, a FLV), the converter would begin working in the background but when the PHP script finished executing, PHP would delete the file right out from under the converter’s feet.

So, a tip to anyone doing anything similar, go ahead and at least rename/move the file (I created a unique MD5 hash from the current time plus a few other things and use that as the new file name) and pass that to any background processing functions so PHP won’t delete the file at the end of processing.

It will save a WHOLE lot of headaches for you in the future.

Also, if anyone is interested, I will probably write up a tutorial on how to write a PHP script that will convert your MP3’s in the background.

There are a multitude of options available but currently I’m liking Mumbles. Unlike, say, Specto, which does the monitoring itself, Mumbles provides a DBus interface, a command-line app named mumbles-send, and (I’m not sure if it’s implemented in the current stable download) libnotify support. Woutc, from the Specto project, commented below explaining that Specto is not intended to be a Mumbles competitor, but a package to easily monitor system internals (I quote “…the purposes from mumbles and specto are different…specto is monitoring what happens outside your desktop, mumbles monitors what happens on your desktop (or in your network)…”) Apparently he has plans to build a Mumbles plugin so that one can optionally have Specto send its messages to Mumbles for display.

I decided the best easiest route is to access the internal DBus API, however the forums and other resources on the Mumbles site… well… just plain suck. And by suck I mean tell you that something exists and… thaaats about it.

Well in my Google quest I discovered the existence of D-Feet, a DBus debugging tool (on Ars of all places). Thanks to a quick sudo apt-get install d-feet I found the existence of a public interface in info.growl.Growl that allows for a Notify(title, message) signal to be passed.

Well, with a place to access all I needed was DBus integration with PHP (because I want to start sending debug notification via Growl/Mumbles like this idea on the Zend Framework incubator).

Luckily, GREE Labs provides a DBus C extension for PHP that was easily downloaded and installed. Once installed I read the documentation to get an idea on how to use the API (which is a 1:1 mapping to the DBus API).

After a bit of hacking I finally came up with an alpha product:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

<?php
function null_callback()
{
        var_dump(func_get_args());
}

$dbus = dbus_bus_get(DBUS_BUS_SESSION);

$m = new DBusMessage(DBUS_MESSAGE_TYPE_SIGNAL);

$m->setPath('/info/growl/Growl');
$m->setInterface('info.growl.Growl');
$m->setMember('Notify');
$m->appendArgs('Hello World!');
$m->appendArgs('Lorem Ipsum Dolor Sit Amet. (' . time() . ')');

$r= $dbus->send($m);

After running the script in the console, did I see success?

Damn right I did!

Though with a caveat:

1
2
3
4
5
6
7

Warning: dbusconnection::sendwithreplyandblock(): dbus_connection_send_with_reply_and_block() failed (Traceback (most recent call last):
  File "/var/lib/python-support/python2.5/dbus/service.py", line 643, in _message_cb
    (candidate_method, parent_method) = _method_lookup(self, method_name, interface_name)
  File "/var/lib/python-support/python2.5/dbus/service.py", line 244, in _method_lookup
    raise UnknownMethodException('%s is not a valid method of interface %s' % (method_name, dbus_interface))
UnknownMethodException: org.freedesktop.DBus.Error.UnknownMethod: Unknown method: Notify is not a valid method of interface info.growl.Growl
) in /home/daniel/test.php on line 21

1
2
3

Call Stack:
    0.0002      64496   1. {main}() /home/daniel/test.php:0
    0.0024      66372   2. dbusconnection->sendwithreplyandblock() /home/daniel/test.php:21

Apparently Mumbles seems to be throwing an Exception that it’s failing to to find the Notify signal (though everything works correctly). I guess I could ignore this, use the error suppressor (@), or even yell at GREE Labs to have the objects throw exceptions so I can catch them… I guess I’ll hack on it some more and report back. I should probably spend more time learning the DBus specs since this is my first project playing with DBus…

Update: Though it’s not listed on the API page for PHP DBus’s API, there is a method send() that takes only a single argument (the message object).

Edit:

I saw a question on Reddit asking what is the use of this technique if it’s limited to the desktop it was called on and PHP is primarily a server side language. Why not do this in Python or Perl?

Well, Mumbles was written in Python so there’s no point in me doing this in Python: it’s already been done.

However, the primary use case of a technique like this is having a web app post notifications and errors for the developer. When I work on a site I have a local copy running on my desktop and/or laptop, so when it posts Growl/Mumbles notifications, I get them on my desktop. It’s great for situations where, say, I have a page in my PHP app that never displays its contents because it processes data then redirects to another page. If a warning or other non-fatal error that I should really fix occurs, then I would normally have to dig through the system wide PHP error log (if you even have error logging enabled). However, if I wrote a custom error handler that posts errors to Growl/Mumbles as they happen, when I visit a transitory page like I described I get little Growl/Mumbles notifications showing that I screwed up!

He talked about how when BMC Software picked him up to head up their open source efforts, he battled a little bit the developers. They wanted to make sure the code was perfected, polished, and at its best whenever they started dumping it on the public. Of course they’d want to do this, releasing code as OSS is almost equivalent to deciding to present yourself to the work naked and without makeup. They wanted their friends who could see the code to be in awe, and their code base to be a shining example of a project done right.

Whurley wanted them just to release it. Not because he wanted it now, not because of impatience or deadlines or anything else. He wanted them to just release it because, and I’m paraphrasing him, releasing open source thats a little buggy will actually help grow and foster the project. Granted, you do not introduce bugs, you just open and display the code and start making releases even though there are still tickets in your bug tracking system.

He gave the hypothetical example of the stereotypical arrogant OSS developer finding a bug in BMC’s FooBar software, calling the BMC devs dumbasses, fixing it (because it was a small and easy bug), and now being invested in the software. You don’t really lose much reputation, but you gain an invested and loyal following.

While this sounds good and true, and it makes a lot of sense (if there’s nothing to do, OSS developers will never get involved), it never really clicked for me until I associated another story with whurley’s principle.

I meet Elizabeth Marie Smith (aka auroraeosrose) at ZendCon08 and had good times talking with her and many of the other #phpc members. At one point in the conference, at an opening for one of her talks (a brief overview of PECL extensions), she told the audience how she got into programming PHP and PHP C extensions. She wasn’t a computer science major, she only started in the early 2000’s (prior to that not much computer science experience).

What had happened was, to the best of my recollection and her seven things post, in the early days of 2001-ish she was really big into Anime and wanted to start a fan site. Her dad helped her install a forum (written in PHP) on a server he was running out of his house on a cable connection.

Well, as with many of the forum projects, there were bugs to be had in the software. Elizabeth was dedicated to the idea of her site and decided to break open the forum and dig around to fix the bugs. Fix them she did but everything kinda snowballed from there. In fact it was bugs in the PHP-GTK extension that caused her to learn C (not an easy feat) and become the maintainer of PHP-GTK as well as the Windows port of PHP maintainer.

It was bugs and problems in the software that caused her to be where she is now, it was bugs in the OSS world that caused her to latch on to PHP-GTK and the Windows PHP port and get them to where they are today. Though she told this story a long time ago, and whurley’s presentation was even further back yet, it only just now clicked that I had seen a real life example of whurley’s principle of “just publish it, flaws and all.”

The lesson from all this can even be applied outside of OSS development. It’s a lesson on perfectionism and can be applied towards my personal development practices. I know that agile teaches to publish early and publish often, fixing bugs and adding features on the way, but perfectionism still plagues me. I have maybe 10 drafts on this blog on small little PHP and Python topics I could publish that I never do because I feel it’s not good enough, or too simple. I have had code at work that I could have released a week ago but instead I obsess over making sure its “perfect” when a simple staging deployment will keep everyone happy, keep my moral up, and show me where the REAL bugs are.

Recently at work I had the need to run a script before every single controller (namely to add a plugin folder to Dwoo) for a specific module that I did not desire for any other modules.

I could have subclassed all my controllers to extend a custom action controller that handled this in the init() method, however I’m lazy so I wrote a quick Zend Controller Plugin to handle this for me.

What it does is grabs the module name from the request object on routeShutdown (the routeShutdown happens after the route has been parsed and the request object has been generated, but before the action controller is instantiated and executed). From there it uses Zend_Font_Controller::getInstance()->getModuleDirectory(...) to grab the literal path to the module directory, tacks on our init file name (in our case “init.php”), and if it exists execute it in its own clean environment.

The init.php file has access to $this, where $this is an instance of the controller plugin (in particular we can do $this->getRequest() inside the init script).

The code is as follows:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41

<?php
/**
 * This checks the current request module's directory for an initFile (defaults
 * to init.php) and runs it before the controller is loaded.
 * 
 * @copyright 2009 Daniel Cousineau
 * @license http://opensource.org/licenses/mit-license.php MIT License
 * @version 0.1.0
 */
class My_Controller_Plugin_ModuleInit extends Zend_Controller_Plugin_Abstract
{
  public static $initFileName = "init.php";
  
  /**
  * @param Zend_Controller_Request_Abstract $request
  * @return null
  */
  public function routeShutdown(Zend_Controller_Request_Abstract $request)
  {
      $moduleName = $request->getModuleName();
      
      $moduleDirectory = Zend_Controller_Front::getInstance()->getModuleDirectory($moduleName);
      
      //Trim the paths and filenames to prevent any problems
      $initFile = rtrim($moduleDirectory,'/\') . '/' . ltrim(self::$initFileName,'/\');
      
      $this->runInitFile($initFile);
  }
  
  /**
  * Run the file in its own cleaned scope
  * 
  * @param string $_initFile location of the input file
  * @return null
  */
  protected function runInitFile($_initFile)
  {
      if( file_exists($_initFile) )
          include_once $_initFile;
  }
}

As always all code I post on this blog is covered under the MIT license.

While there already exists a Python set of classes that serializes (and serializes) Python data, it (a) didn’t handle objects and (b) was licensed under the GPL (rant to follow later), meaning I couldn’t use it for it’s intended purpose due to the closed source nature of my project.

I decided this was a good enough time to play around with git so I created a github repo here.

It could probably still use some more TLC in the object area, maybe a few more supported types, and unserialization functions would be nice.

I do feel it wasn’t bad for a few hours of hacking if I do say so myself.

The code for the module (should be in a phpserialize.py file) is as follows:

Python Code

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135

"""
Collection of functions to serialize Python variables to the native PHP format

@version 0.1
@author Daniel Cousineau <dcousineau@gmail.com>
@copyright Copyright (c) 2008 Daniel Cousineau
@license http://www.opensource.org/licenses/mit-license.php MIT License
"""

from types import *

def serialize(var):
  """
 Serialize var to the native PHP serialization format
 """
  if type(var) is IntType or type(var) is LongType:
      return serialize_int(var)
  
  elif type(var) is FloatType:
      return serialize_decimal(var)
  
  elif type(var) is BooleanType:
      return serialize_boolean(var)
  
  elif type(var) is StringType:
      return serialize_string(var)
  
  elif type(var) is NoneType:
      return serialize_null()
  
  elif type(var) is ListType or type(var) is TupleType:
      return serialize_array(var)
  
  elif type(var) is DictType:
      return serialize_dictionary(var)
  
  elif type(var) is InstanceType:
      return serialize_object(var)
  
  else:
      raise TypeError("Invalid Type %s" % (type(var)))

def serialize_string(string):
  """
 Format: s:L:"STRING";
     L: Length of the string
     STRING: The string itself. No need to escape quotes (")
 """
  return "s:%d:"%s";" % (len(string), string)

def serialize_int(int):
  """
 Format: i:D;
     D: the integer
 """
  return "i:%d;" % (int)

def serialize_decimal(decimal):
  """
 Format: d:D;
     D: the decimal (accepts expontent notation)
 """
  return "d:%s;" % (decimal)

def serialize_boolean(boolean):
  """
 Format: b:B;
     B: 1 for True, 0 for False
 """
  return "b:%d;" % (boolean)

def serialize_null():
  """
 Format: N;
 """
  return "N;"

def serialize_array(array):
  """
 Format: a:K:{VALUES}
     K: Number of entries (number of keys for dictionaries)
     VALUES: serialized key appended to serialized value. Do not worry about ;
             if a value is an array
 
 DOES NOT END WITH A ;
 """
  values = []
  for index, value in enumerate(array):
      values.append(serialize_array_key(index))
      values.append(serialize(value))
  return "a:%d:{%s}" % (len(array), "".join(values))

def serialize_array_key(var):
  """
 Decimals and Boolans are automatically converted to integers for key values.
 Arrays and Objects throw errors if they are set as a key
 """
  if type(var) is IntType or type(var) is FloatType or type(var) is BooleanType:
      return serialize_int(int(var))
  
  elif type(var) is NoneType:
      return serialize_int(0)
  
  elif type(var) is StringType:
      return serialize_string(var)
  
  else:
      raise TypeError("Invalid Key Type %s" % (type(var)))

def serialize_dictionary(array):
  """
 See serialize_array()
 """
  values = []
  for index, value in array.iteritems():
      values.append(serialize_array_key(index))
      values.append(serialize(value))
  return "a:%d:{%s}" % (len(array), "".join(values))

def serialize_object(obj):
  """
 Format: O:L:"CLASS":K:{MEMBERS}
     L: Length of class name
     CLASS: class name (string)
     K: Number of class members (ignore 'static' members)
     MEMBERS: Members as an associative array (dictionary) (ignore 'static' members)
 """
  objClass = obj.__class__.__name__
  
  values = []
  for index, value in obj.__dict__.iteritems():
      values.append(serialize_array_key(index))
      values.append(serialize(value))
  
  return "O:%d:"%s":%d:{%s}" % (len(objClass), objClass, len(obj.__dict__), "".join(values))

I also wrote up some tests in PHP:

PHP Test Code

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130

<?php
echo "BEGINNING TESTS:n";
echo "----------------nn";

//STRING

$test = "String";
$python = <<<HEREDOC
import phpserialize
print phpserialize.serialize("String")
HEREDOC;

print_results("STRINGS", $test, $python);

//INTEGER

$test = 52;
$python = <<<HEREDOC
import phpserialize
print phpserialize.serialize(52)
HEREDOC;

print_results("INTEGERS", $test, $python);

//DECIMAL

$test = 9.8;
$python = <<<HEREDOC
import phpserialize
print phpserialize.serialize(9.8)
HEREDOC;

print_results("DECIMALS", $test, $python);

$test = 1.52*pow(10,-5);
$python = <<<HEREDOC
import phpserialize
print phpserialize.serialize(1.52*pow(10,-5))
HEREDOC;

print_results("DECIMALS (exponential format)", $test, $python);

//BOOLEAN

$test = true;
$python = <<<HEREDOC
import phpserialize
print phpserialize.serialize(True)
HEREDOC;

print_results("BOOLEANS", $test, $python);

//NULL

$test = null;
$python = <<<HEREDOC
import phpserialize
print phpserialize.serialize(None)
HEREDOC;

print_results("NULLS", $test, $python);

//ARRAY

$test = array(9, 8, 7);
$python = <<<HEREDOC
import phpserialize
print phpserialize.serialize((9, 8, 7))
HEREDOC;

print_results("SIMPLE ARRAYS", $test, $python);

$test = array("key" => "value", 5 => "another value");
$python = <<<HEREDOC
import phpserialize
print phpserialize.serialize({"key":"value",5:"another value"})
HEREDOC;

print_results("ASSOCIATIVE ARRAYS", $test, $python);

$test = array("key" => "value", 5 => array("values"));
$python = <<<HEREDOC
import phpserialize
print phpserialize.serialize({"key":"value",5:["values"]})
HEREDOC;

print_results("COMPLEX ARRAYS", $test, $python);

//OBJECTS

class Test
{
  public $i = "PUBLIC";
  public $j = "B0RKED";
}

$test = new Test();
$python = <<<HEREDOC
import phpserialize

class Test:
  def __init__(self):
      self.i = "PUBLIC"
      self.j = "B0RKED"

print phpserialize.serialize(Test())
HEREDOC;

print_results("OBJECTS", $test, $python);



function print_results($test_name, $var, $python_code)
{
  echo sprintf( "TESTING %s:nn", $test_name );

  echo sprintf( "Expecting:  %sn", serialize($var) );

  echo sprintf( "            %sn", implode("n            ", explode("n", print_r($var, true))) );

  echo "n";

  $recieved = exec("python -c " . escapeshellarg($python_code));

  echo sprintf( "Recieved:   %sn", $recieved );

  echo sprintf( "            %sn", implode("n            ", explode("n", print_r(unserialize($recieved), true))) );
  
  echo "n---nn";
}

Which gives us the following output:

PHP Test Results

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152

BEGINNING TESTS:
----------------

TESTING STRINGS:

Expecting:  s:6:"String";
            String

Recieved:   s:6:"String";
            String

---

TESTING INTEGERS:

Expecting:  i:52;
            52

Recieved:   i:52;
            52

---

TESTING DECIMALS:

Expecting:  d:9.800000000000000710542735760100185871124267578125;
            9.8

Recieved:   d:9.8;
            9.8

---

TESTING DECIMALS (exponential format):

Expecting:  d:1.5200000000000001853270141516372149226299370639026165008544921875E-5;
            1.52E-5

Recieved:   d:1.52e-05;
            1.52E-5

---

TESTING BOOLEANS:

Expecting:  b:1;
            1

Recieved:   b:1;
            1

---

TESTING NULLS:

Expecting:  N;


Recieved:   N;


---

TESTING SIMPLE ARRAYS:

Expecting:  a:3:{i:0;i:9;i:1;i:8;i:2;i:7;}
            Array
            (
                [0] => 9
                [1] => 8
                [2] => 7
            )


Recieved:   a:3:{i:0;i:9;i:1;i:8;i:2;i:7;}
            Array
            (
                [0] => 9
                [1] => 8
                [2] => 7
            )


---

TESTING ASSOCIATIVE ARRAYS:

Expecting:  a:2:{s:3:"key";s:5:"value";i:5;s:13:"another value";}
            Array
            (
                [key] => value
                [5] => another value
            )


Recieved:   a:2:{i:5;s:13:"another value";s:3:"key";s:5:"value";}
            Array
            (
                [5] => another value
                [key] => value
            )


---

TESTING COMPLEX ARRAYS:

Expecting:  a:2:{s:3:"key";s:5:"value";i:5;a:1:{i:0;s:6:"values";}}
            Array
            (
                [key] => value
                [5] => Array
                    (
                        [0] => values
                    )

            )


Recieved:   a:2:{i:5;a:1:{i:0;s:6:"values";}s:3:"key";s:5:"value";}
            Array
            (
                [5] => Array
                    (
                        [0] => values
                    )

                [key] => value
            )


---

TESTING OBJECTS:

Expecting:  O:4:"Test":2:{s:1:"i";s:6:"PUBLIC";s:1:"j";s:6:"B0RKED";}
            Test Object
            (
                [i] => PUBLIC
                [j] => B0RKED
            )


Recieved:   O:4:"Test":2:{s:1:"i";s:6:"PUBLIC";s:1:"j";s:6:"B0RKED";}
            Test Object
            (
                [i] => PUBLIC
                [j] => B0RKED
            )


---

The easiest way to provide a downloadable file is by altering the headers and echo’ing the file content. In our case:

1
2
3

header("Content-type: text/csv");
header("Cache-Control: no-store, no-cache");
header('Content-Disposition: attachment; filename="filename.csv"');

We want to set our applicable Content-Type so that the browsers associate the file properly. Just relying on the extension doesn’t work, even in Windows. The magic is in the third header setting, “Content-Disposition,” which informs the browser to download as a separate file (don’t open a new window and display a blank page, just display the file download box) and tell the browser the filename is “filename.csv”. This way rewrite rules like http://localhost/export/csv/ will result in a download box that declares the file “filename.csv” rather than a randomly assigned name or whatever the current url is.

Into the meat of the CSV export. At the very beginning we need to open up a stream to the PHP output (the same place where echo sends its string content, which is NOT stdout):

1

$outstream = fopen("php://output",'w');

Next we’re going to assume you already have your data packed nicely into an array (or array of arrays) so long as we have a single array per row/line.

The magic comes into play using the build in PHP function fgetcsv(). fgetcsv() takes an array for a single row and outputs it, automatically escaping output according to column and enclosure delimiters!

fgetcsv() requires a file resource as its first parameter and the magic of PHP streams is they act like a file resource (actually a file resource is just a file stream), so we give it $outstream to make fputcsv() echo its output. We fill in the rest of the parameters according to the php.net documentation and voila we have:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

header("Content-type: text/csv");
header("Cache-Control: no-store, no-cache");
header('Content-Disposition: attachment; filename="filename.csv"');

$outstream = fopen("php://output",'w');

$test_data = array(
  array( 'Cell 1,A', 'Cell 1,B' ),
  array( 'Cell 2,A', 'Cell 2,B' )
);

foreach( $test_data as $row )
{
  fputcsv($outstream, $row, ',', '"');
}

fclose($outstream);

For more output stream craziness, Timothy Boronczyk from the #phpc IRC channel on freenode shared a code snipped that outputs CSV either to the output buffer OR will return it as a string using some clever streams hackery:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

function exportCSV($data, $col_headers = array(), $return_string = false)
{
    $stream = ($return_string) ? fopen ('php://temp/maxmemory', 'w+') : fopen ('php://output', 'w');

    if (!empty($col_headers))
    {
        fputcsv($stream, $col_headers);
    }

    foreach ($data as $record)
    {
        fputcsv($stream, $record);
    }

    if ($return_string)
    {
        rewind($stream);
        $retVal = stream_get_contents($stream);
        fclose($stream);
        return $retVal;
    }
    else
    {
        fclose($stream);
    }
}

So ZendCon turned out to be completely worth it, even if only for the chance to meet in person a lot of the core PHP community and then drink semi-copiously with them.

The talks were good, though most of them were either more basic than I would have liked or really indepth (/me thinks back to Sara’s PHP Extension Writing talk, though that was more “few php developers gets this deep” than indepth/complicated).

The uncon was nice, which of course I’d like it given my love for barcamps, refreshes, and generally community driven and adhoc events.

The Yahoo party started out awkward given that one expects an open bar after paying $500+ for tickets to a conference, however from what I gathered it really wasn’t so much Yahoo’s fault more the hotel’s. I won the raffle for the “Day with Rasmus” prize and felt bad ‘cause when one of the Yahoo girls came to me and to have me verify my information, I told them my email (which was a gmail account) was correct but if they still have trouble to just google my first and last name since I dominate the first page for my name.

My only problem with ZendCon is that it ended…