Blog Post

Introducing Acceptance Testing

GravCart has some basic Unit testing in place, for both PHP and JavaScript code. This allows to test the functionality in isolation, one method at a time. It's pretty useful to predict low-level behavior, and to keep constant track of possible issues when changing a method implementation. Not everything is covered, but the aim is to cover all it's sensible to cover over time.

Today I worked on doing the groundwork for allowing automated Acceptance Testing for the plugin, and its addons.

What does it mean?

The most immediate level at which code quality, and the whole product, is reflected is the user-facing part of the plugin.

The user experiences the process of checking your site, choosing one of the products you sell, add it to the cart, check some options if any are setup, and proceed to the checkout.

Nothing can go bad in this flow, or the user will be prevented a purchase, and bad things will happen, in the worst case leading to a missed sale.

Your role

Your role, as Shop owner, into this, is to make sure you carefully followed the docs and filled the information where it was required. Like setting up the correct Stripe / PayPal account information. And make sure you run the site on a good server, in good shape, up to date, etc etc.

My role

My role, as the Plugin developer, is to make sure that every change I ship in a release will never break this flow. Every option, every workflow change, every setting changed should be tested, on all the major browsers and operating systems, to make sure everything will work as expected once shipped. Possibly on different PHP versions, on different Web Servers, on different hosting setups.

It's a pretty hard job, right? Especially working solo, when my already limited resources should be focused on building the product, not testing every single possible thing that might go wrong, on all possible cases I might think of.

So how can this be achieved without duplicating myself? With automated testing.

Automated testing is amazing.

It's like having an army of minions at your fingertips doing what you require, at the time you want.

Enter Codeception.

Codeception is a BDD style testing framework for PHP, and allows Unit Testing and Acceptance testing (and Functional, which is not yet a thing for GravCart) with the same tool.

Used in Grav core already, for Unit Testing, I already baked it in GravCart for Unit Testing, and today I decided to write a base Acceptance Testing automation that will, moving forward, expand to cover all possible scenarios the Plugin will support.

How does it work?

Conceptually it's pretty simple. There is an Actor, which I called FrontendTester, who simulates a customer of yours purchasing a product. One of the existing workflows tested is picking a product, adding it to the cart, and purchasing it using Stripe.

So I created a simple, sample skeleton and added a couple products and entered my Stripe test keys.

And then I instructed my tester to execute some actions. Here's the code:

$I = new FrontendTester($scenario);

$I->wantTo('buy a product with Stripe');
$I->lookForwardTo('conclude the purchase process');


buyProductWithStripeCheckout is a method which my actor knows how to do as it's stored in its own class. Here's its implementation:

    public function buyProductWithStripeCheckout()
        $I = $this;
        $checkoutPage = new CheckoutPage($I);

It's a very eloquent syntax, which is perfectly readable.

Everyone of those lines is a separate method, implemented using the Acceptance testing syntax offered by Codeception. For example addProductToCart is:

     * Add a product to the cart
    public function addProductToCart()
        $I = $this;
        $I->see('Geek Toys');
        $I->seeLink('Geek Toys');
        $I->click('Geek Toys');
        $I->see('Anime, Gaming, Movies, Comics, we have all your toys');
        $I->see('Stuffy Turret');
        $I->see('Brick Mug');
        $I->seeLink('Add to cart');

which again is a pretty simple description of the actions to take on the page.

The nice thing about this kind of testing is that, once properly setup, it will also execute the JavaScript code. This is done via PhantomJS WebDriver, which simplifies things a lot over using Selenium, the alternative platform that offers such possibility.

Once all is setup, I can now type

./vendor/bin/codecept run acceptance --env chrome

and just wait for the minions to do the work and report it to me.

I can also watch them doing every step with

./vendor/bin/codecept run acceptance --env chrome --debug

Or even see every screenshot of the steps by enabling the Codeception\Extension\Recorder Codeception extension.

Taking this to the next level

The next level is testing everything on every browser, on every platform. There are various services that allow this, I chose Open Sauce from Sauce Labs, because they offer an unlimited free tier for Open Source projects.

I have setup a couple platforms in my configuration, with

                    browser: 'chrome'
                        platform: 'Windows 10'

                    browser: 'firefox'
                        platform: 'Windows 10'

                    browser: 'MicrosoftEdge'
                        platform: 'Windows 10'

so I can test browsers on Windows, by running ./vendor/bin/codecept run acceptance --env edge_win10 --debug I can quickly check if my scenario works in Edge, without even having a computer that can run it.

Wrapping up

I can only imagine the amount of time this will save me while developing, and also the amount of bugs which this toolset will allow me to find before reaching your sites.