Acceptance testing in the land of the startup
Download session PDF
Add to calendarSongkick is a web based music startup which operates in a new market place full of unknowns, developing a free service. Focusing on user experience as one of the most critical features of the product. Continuously iterating and experimenting with features based on user observations and metrics. Unusually we adopted Acceptance tests and BDD while still in the early startup phase. Having spent more than 2 years growing our system and learning the pain points both technically and culturally we have lots of interesting lessons we would like to share about startups and Acceptance testing.
The session will start out by highlighting what makes acceptance testing different in a startup environment, setting the scene for the world Songkick operates in. Then the session will through presentation slides work through an example of a real feature passing through our development workflow. Using photos from Songkick of the cards, the people/roles and the tools involved. Following the progression across the Kanban board (with the focus being on testing).
The session will end with a brainstorming activity focusing around a mind map representing Songkicks current experiences and lessons. Building on this mind map with suggestions from the audience of relevant lessons or experiences they want to share.
As an output of the session a more detailed mind map will be produced and then shared to help promote discussion and fuel ideas in the wider startup community.
Timings
| Duration | Summary | 5 minutes | Setting the scene of startups and Acceptance testing. |
| 30 minutes | Working through a real example with Songkick’s flow. |
| 20 minutes | Brain storming, outputting a mind map. |
| 5 minutes | Questions and summing up. |
Background
The session has been evolving from:
- An experience report I gave at the Agile Testing UK user group (http://agiletesting.org.uk/october_2010.php)
- A session I ran at XPDay 2010 London on ‘XP and Startups’.
Songkick.com is also used as a case study in Gojko Adzic’s (http://gojko.net/) upcoming ‘Specification by Example’ book: http://specificationbyexample.com/.
- Some of the challenges and failings in the disjointed connection between the automated tests and QA.
- The importance of Acceptance tests as documentation when rapidly scaling the size of the company and development team.
- Understanding the cost that Acceptance testing can have on a startup due to a rapidly changing, volatile environment (for example throwing away a number of features and the effort spent working on their acceptance tests).
- How Acceptance testing and QA fit within a rapidly growing startup (and the many ways we got it wrong).
- Helping build trust in the Acceptance tests by making them easily accessible by everyone (including non-technical roles) and when creating the Acceptance tests pairing with different roles (especially between the QAs and the developers).
- Demonstrating the value of recording test metrics (such as failure rates) and using them to feedback into how the tests are run. For example prioritising the tests that are most likely to fail.
