So, it’s crunch time again… Time to dig into those last frantic hours (days) before uploading a new app to the AppStore, making sure everything is correct, that all the texts and images that comes with the release are in their proper place, all signatures and little iTunes oddities are sorted and, of course, that the App actually works as it should.
TempusCura complicates matters further by being a hosted multi-user App, introducing a variation in UI perspective defined by different user-roles and a server backend that needs to be tested for vulnerabilities and scalability as well. I am using Google App Engine which introduces a third backend problem: Optimizing for cost: Sometimes the best performing solution is not the cheapest, and at the end of the day, I *am* trying to run a business so numbers need to add up.
But that’s a topic for a different post – right now I wanted to share my pre-release test procedure for TempusCura.
During development I use TestFlight to manage testers and while that works for distribution of test binaries, it does not really help validate anything. Unit testing sorts out bugs in core functionality and is especially useful for backend testing, but as anyone who ever worked on an App (or any other UI intensive piece of software) can tell you, there’s a scary bug potential hiding in the UI which no amount of Unit testing will find.
So at the end of the day – or the end of development, rather – boring as it is, you do need to go through that end-to-end systems test. I guess there are many ways to do this, but for an iOS app, I like to sit down with InterfaceBuilder as a guide and the app running on a device and then go through every screen and every button while writing up a test protocol.
The test protocol is basically a check list of what should happen in various parts of the app when the users performs certain actions under some given circumstances.
To give an idea of how that may look, I’ve attached the TempusCura test protocol in its first revision as an example.
Some of the things that I try to keep in mind when writing the protocol are:
- Write each test as a simple one-line question/statement that can be easily validated as true or false.
- Try to group tests so that they follow the flow in the app naturally – this makes testing so much quicker.
- Make sure all UI paths are included.
- Make sure the expected result is obvious from the test description.
One of the interesting things with a test protocol is how many bugs you find just writing the document. For TempusCura I found a handful of small and large issues when writing the document, and 17 bugs (quite a few serious ones) the first time I ran the test.
Remember to always take the entire test from the start with the binary you intend to upload – we all know what usually happens when we “fix” bugs in the 11th hour 😉