I want to process incoming xml payloads with an encrypted block of elements contained within the unencrypted main body.
And lets suppose when we look at what that entails it’s something like this:
- Edit DTD to accommodate encrypted block (which will be JSON containing key value pairs to then be substituted into well formed XML as if the encryption hadn’t been used).
- Identify whether this payload is of the encrypted variety or not.
- Check that the client is allowed to use this new partially encrypted service.
- Obtain encrypted private key from DB, decrypt it, decrypt the AES key in the payload, use the AES key to decrypt the rest of the encrypted block.
- Basic validation of elements that have been decrypted (probably just that they are all present).
- Create a well formed XML document by putting the decrypted JSON key value pairs into their correct places within the XML document.
- Process the XML document as we normally would do before the new encrypted service.
Now lets suppose that this all looks too big to fit into 1 sprint. We should break it down into smaller user stories. Fine.
- Lets pull out the decryption tasks into their own user story.
Er… ok. How are you going to go about that?
I want a decryption component to decrypt stuff that our new service consumes.
Noooooooo! This sucks – big time. Who wants a decryption component? No one. I don’t care about a decryption component, the PO doesn’t want a decryption component, and the customer doesn’t want one either. What we all really want is a solution – a decryption component is of no value to anyone. Now the other steps in our breakdown above don’t really make any sense because there’s a big hole in the process flow.
I want a component to parse JSON key-value pairs and to place them into an XML template in order to generate a full & well formed XML document.
I want a story to mop up everything else not already covered by the previous 2 stories.
Now there is no hole in the process flow because each step is in it’s own discrete story. Huzzah!
Kill me now. Why do we even try to fit stories into sprints in the first place? Why do we have sprints at all? Because we don’t want ‘nearly done’ but non-demonstrable code floating around for extended periods. We want to show progress frequently in order to get rich and timely feedback, rather than having to wait until the last piece of the puzzle is put in place before anything actually works, and because the concept of a story bundles functional requirements, code & tests, all into a neat little package of context.
The naively crafted stories above are well on their way to dictating implementation details, rather than specifying problems that need solutions. Especially when you start adding acceptance criteria to them.
What are the QAs going to do on the decryption story? The programmer will unit test it, expecting the QA to help him ensure it works properly, and makes sense, in the wider context of the solution to the specified problem. In reality they know that they want to test this decryption implicitly as part of the problem solution. They are going to be writing automated tests which don’t actually map to this user story – as if the test actually belongs to an epic which contains that story. But that means they can only complete their tests once we’ve implemented enough of the other stories such that we actually have something that resembles a solution. They can’t really do much with the story itself, that the programmer probably can’t handle himself with his unit testing. Moreover, even if they did it would have nothing to do with the problem solution – they’d just be testing an arbitrarily chosen implementation detail of the forthcoming solution.
What will we demo to the PO & stakeholders? Are they going to care that your language of choice can do decryption? Will they care that you crafted your decryption class in a certain way? Of course not. So then if we have nothing to demo, why did we bother breaking down the original story in the first place? We’re now, practically speaking, in the same position we would have been with one monster story broken into several technical sub-tasks – except that would at least still retain it’s user story integrity, albeit a user story that would need a longer sprint to complete. The list of issues goes on.
Having some meetings every 2 weeks does not make a series of sprints. Arbitrarily splitting work into 2 week chunks does not make a series of sprints. Labelling a unit of work a ‘User Story’ does not make it any more valuable, nor does it make anything easier. — Me
So what should we have done? I won’t suggest ‘the’ way it should be done. Instead I’ll suggest the very first method that jumped into my head, and suggest that with some thought an even better solution might present itself. Even so, I believe that this alternative would be MUCH more desirable than what we’ve discussed so far. Make the problem easier to solve, and deliver that solution. Then flesh out the solution to solve the full problem in a later iteration. That is to say we should maintain user stories that actually serve the purpose of a user story. We should maintain that little package of problem/solution wrapped up in context…
I want to process incoming xml payloads which contain an embedded JSON payload of key-value pairs, held within the main document body in place of regular Xml elements.
This is the whole original problem, but a little simpler. The JSON isn’t encrypted. We can implement the entire solution, with proper agile testing – possibly with some nice BDD style test specs to serve as living documentation – and have something complete, if not yet fully functional, to demonstrate at the end of the sprint. The more holistic nature of implementing a solution, as opposed to a component, helps to leverage the benefits that Scrum offers when it comes to providing team context & cohesion. It also means that we have a “potentially shippable” product increment (although perhaps it’s worth noting that the Scrum Guide now says ‘releasable’ rather than ‘shippable’); The salient point here is that a decryption class is not a product increment as it is effectively dead code until the solution as a whole is ready to make use of it.
The selected Product Backlog items deliver one coherent function, which can be the Sprint Goal. — Scrum Guide
Now that we have a working solution that we can demonstrate in it’s current form, we can add an additional user story to the backlog:
I want the new service to be able to accept and process encrypted JSON in it’s payload, instead of unencrypted JSON.
Now in a further 2 weeks at the next sprint demo, we can show off our all singing & dancing version. The existing tests can be tweaked and we can now integrate with whatever it is that is producing encrypted JSON in the first place. More importantly no kittens get harmed in the process, which is more than can be said of the original effort. This activity (generating and breaking down stories) should very rarely be difficult. As soon as it is, stop – breathe… ask yourself why you’re even generating user stories in the first place. Then ask yourself why you care if it fits into a sprint or not. Then carry on.