How the Xmas Sales Back End was Built

November 29, 2012

Even though most of what makes the SitePoint Christmas Sale 2012 fun happens in the front end, we still need a back end serving the data. It doesn’t play a huge role, but it’s still an important one.

Before starting the development of it, we talked about what this role should be exactly, and came up with the following requirements:

Serve sales data (deals information) and story data (images and animation) in JSON format
The data needs to be different for every day: Every day contains not only the current day, but also the previous days (with minor modifications such as disabling the sale)
Admin interface to edit sales and swap them easily if needed

The last task, especially, would require a lot of work if it had to be written from scratch, so we decided to use a framework. In our case, as we use PHP, we chose Symfony2.

The following will describe how the basic setup works and some of the advantages of using Symfony2.

Setup

Symfony2 is a MVC framework based on many components wired together. The components are managed by composer, a dependency manager for PHP. Using composer, it only takes a couple of command lines to get Symfony2 running.

Once the framework is ready to use, I usually start by defining the entities (the model part of MVC). Normally this would mean writing a lot of boilerplate code (classes with getters/setters for all database fields), but Symfony2 provides handy CLI commands to generate entities. This is both less error-prone and a lot faster.

Admin interface

The second tedious task would be to write an admin interface with forms to create and edit the entities. Luckily, Symfony2 allows us to create a CRUD (create/read/update/delete) interface automatically with another command. To make things look nicer and more usable, I styled the interface using Twitter Bootstrap.

After a few adjustments like adding validation and ordering the lists, we get an admin interface which is already quite usable. Ready to serve content to the client!

Serving JSON

It would, of course, have been possible to store the sale and story data statically, but then it would also have been possible for you to get a sneak peek of the following days in the source code :). So, we decided to serve the JSON needed for the current day from the back end.

On each request, our Symfony2 app gets the current day and based on that determines which sales and which part of the story should be sent to the client. This is quite easy to achieve by querying the database, assembling the retrieved data into an array, running it through json_encode and sending it back to the client. Done.

Caching

Actually, implementing this procedure every time a request comes in is a huge waste of resources. The response is the same for all clients for each individual day. On any given day, it is necessary to generate the response only once, store it in a cache and then reuse that for the rest of the day.

Caching is great, but cache invalidation is one of the two hardest things in computer science. Symfony2 comes with a built-in caching mechanism (modelled after the HTTP cache specification), making it a lot easier to handle.

To start with, we need to choose which invalidation model to use. There are two possible models: expiration and validation.

Expiration is the easier one and should always be preferred if possible. The developer defines how long a resource should be valid and, if a request is made past that time, a new cache is generated. Unfortunately, that won’t work for us as we don’t know when the first request is made, so we don’t know how long exactly the cache should be valid. We need to use the second model.

When using the validation model, you calculate just enough data to see if the data in the cache is still valid. If it is, serve the cache; if not, regenerate it. This procedure means, however – as opposed to the first model – that you have to start your app and retrieve some data to determine if the cache is still valid. Luckily, the information we need in our case to see whether the cache is still valid is just the current day which of course is not very complicated.

Now, when generating a cache we set the last modified value to the current day. Then, on every following request, we compare the last modified value of the cache with the current day. If they match, we serve the cached response. With this code in place, we only access the database once per day (instead of for every request!).

Testing

Of course, we want to be sure our code works and every day will actually have the correct information in the JSON before the sales start. We definitely don’t want to sit in front of our computers anxiously watching as the clock-hand passes 12pm Melbourne time and reload the page to see if the new sale has been served correctly …

For this task, we use functional tests. Functional tests are very similar to unit tests, but their focus is to test that the output of the app is correct. Every test simulates an HTTP request and allows the developer to run various assertions on the returned response. This is really straightforward in Symfony2 as the framework is modelled around the request-response model of HTTP, so it is very easy to simulate the requests.

In our use case, I decided to create additional routes (accessible internally) which allow the developer to pass in a desired day. The response will then be what the response of the publicly accessible route will look like when it is requested on the actual day. In our tests, we call these routes and check that the returned JSON contains the information we expect. With this setup, we are able to simulate the complete Christmas Sale in advance and check that all the data will be exactly as it should be – allowing us to let the sales begin with complete peace of mind.