Test Driven Requirements

{Disclaimer – this is a cross-posted article appearing both on the Triton-Tek blog and my personal blog Budding BA}

My job at Triton-Tek is not an easy one to define, but predominantly I am a Business Analyst and Project Manager on Agile web development projects, see my previous articles here and here.  However there is one major project on which I have been working since day one that has required a lot of Quality Assurance (QA) support from me so that the developers get quick feedback on their work.  This work as a QA, has been incredibly beneficial to me as a BA in crafting the requirements for the project.  In an Agile environment, the approach is very much “just-in-time”, and the same applies to requirements gathering.  We will have a set of stories for the release, and a plan for the current iteration, but the specific ins-and-outs of the stories are still very foggy.  So as a BA I will add details to the stories as they come up, but often it is an iterative feedback loop that defines the requirement.

Agile User Story completion cycle

Because of this emphasis on QA in my role I started to read about how to do this in an Agile fashion, and the results are fascinating.  The connection between the QA role and the BA/requirements role in Agile projects is very strong, and the Agile testing techniques I have learnt have helped my requirements analysis and definition as a result.

This mindset change starts with the “Agile Testing” book by Lisa Crispin and Janet Gregory.  It is quite a large book, but very practical and describes in detail the daily hands-on role a tester must play on an Agile project.  I will not review the book here, that is another post on its own, but I am adding the Agile Testing Quadrant below as it gives a good overview of the different areas of testing.

Agile Testing Quadrant from "Agile Testing" by Lisa Crispin & Janet Gregory

Essentially the book drills home the many different angles from which the product must be tested in order that you can be sure of as solid an end result as possible.  Critically the developers must practice Test Driven Design (TDD) for this all to work.  This is accomplished by starting with failing unit tests (because the functionality does not yet exist) and then coding the solution so that the tests now pass.  So for example, they would start with a test titled “User can enter a username and password to login”, and then code the user login functionality so that the test passes.  I learned of TDD upon starting at Triton-Tek and understand what its purpose was, however since reading this book my eyes have been opened to how the same approach can be used when gathering requirements, and how that can feed directly in to the code, simultaneously creating automated unit tests and setting the acceptance criteria for when a story can be considered “done”.

Now, before I continue, I want  to stress that this is not a new discovery on my part, there is a community around Acceptance Test Driven Design and Behavior Driven Design (BDD) that talk about exactly this process.  However, as a BA with no coding background, the personal progression in understanding of how this all ties together is one from which I think a lot of BA’s can benefit.

Going back to the quadrant above, the area we are focused on is Q2, the functional and story tests that can be automated.  The book specifically talks about using the FitNesse tool to automate functional testing.  This tool allows you to create understandable “business” phrases, using a certain format, that the developers can then take and use a fixture to directly test their code to make sure the functionality works as the business expected.  The great thing about this approach is that it expands on the quickly jotted down acceptance criteria on a story card so that the developer is immediately presented with the list of tasks he or she needs to complete in order to call the story done.

From a BA perspective, this approach is hugely helpful.  Since reading about FitNesse, I spoke with the team at Triton-Tek and Matt Hidinger recommended SpecFlow as that uses the more business and domain friendly BDD approach, building on the Cucumber syntax, and integrates easily with TFS and the .NET world in which we work.  In BDD, the tests are created using the following syntax:

Given [condition]

When [action]

Then [result}

So an example would be:

Given the user has already registered

When the user enters the correct username and password on the login screen

Then the user will be logged in to the system

These "Given .. When .. Then" statements can then be used by the developer as the basis for their unit tests ensuring the functionality of the story meets the business / user requirements.  This is a very simple, but powerful tool for a number of reasons critical to a BA's work:

1. The language is easy to understand, so it can form the basis of supporting requirements documentation.
2. Anyone can write them, so you can pull in stakeholders and product owners to directly contribute.
3. They encourage analytical and practical thinking about the requirement.
4. Reduces the lost in translation problem of transferring business requirements to the developer.

So now with my BA hat on, when I am expanding on the requirements for a User Story, I can start walking through the many business rules, user scenarios and use cases using the "Given .. When .. Then" statements.  This always end up uncovering hidden problems or complexities, that may not have been discovered until development had begun.  It is also a fantastic way to keep the product owner or client representative involved, the statements are easy to understand so any business user can create them, meaning that they are setting their own acceptance criteria.  This is very important in a consulting environment such as ours where any misunderstanding of what functionality a user story should actually achieve can have large monetary and client/contractor relationship ramifications.

Now with my QA hat on, the biggest bonus to this approach is that I have simultaneously created automated functional tests, and encouraged TDD behavior from the developers.  This should allow me to focus on the more fuzzy (and in my opinion, interesting) exploratory and usability testing, which is where I can add the most value to the product and the team.  Overall, everyone wins, and most importantly the requirements are delivered seamlessly from the business to the development team.

Resources on this topic:

• Agile Testing: A Practical Guide for Testers and Agile Teams by Lisa Crispin & Janet Gregory
• Lisa Crispin's blog and follow her on Twitter @lisacrispin
• Brian Marick's blog - an Agile tester and developer, strongly influenced Lisa Crispin
• Behavior Driven Design (BDD) - the original article by Dan North
• SpecFlow - the .NET BDD tool
• Cucumber - the language behind SpecFlow
• Acceptance Test Driven Development - also see the deck below
• Yahoo Agile Testing Group

When User Stories are not enough

I recently attended an interesting webinar by MKS on why User Stories are sometimes an inadequate representation of requirements.  I thought it raised some excellent points, and certainly got me thinking.

Starting with a refresher on what User Stories were, the presenter Colin Doyle emphasized how they are NOT a requirements document.  Instead they are a placeholder for continued conversation on that particular piece of functionality and used especially for prioritization.  They are the tool through which requirements are communicated and discussed rather than delivered in the form of hefty requirements documents (see this excellent article from Mike Cohn on User Stories).  As Agile prescribes, in teams where face-to-face, daily communication is possible and the Product Owner and key stakeholders are readily available, the User Story format works very well.  Each story follows the INVEST criteria and are first and foremost independent of other stories, and the non-functional requirements or constraints do not dominate.  With each iteration, the product develops incrementally with the gradual evolution and folding back in to the project of the inevitable changes that will arise as the product comes to life.  If there is a need to review the requirements and see the “big picture”, then the User Stories in combination with Test Driven Design (TDD) or Behavior Driven Design (BDD) practices, should provide all the details you need.

Colin Doyle argued the point however that there are 3 key issues with this method, all centered around the scalability:

1. Too many customers for the product resulting in conflicting requirements and a significant effort to understand customer environments and needs.
2. Complex products with significant interdependencies, lots of exisiting requirments and the lack of a way to see the big picture easily.
3. Large organizations where “scaling of face-to-face conversations is an N2 function (Brooks)” and may include offshore or off site teams.

Therefore he recommended the careful introduction of some traditional requirements engineering practices to help solve these issues.  In particular using modeling techniques such as use cases, decision

Use Case Model example

trees or event-response tables to get a handle on the big picture.  This will help the project in number of ways.  Firstly the developers will understand the full scope more clearly and so guide their TDD/BDD test cases and hence their code.  It will also help the Product Owner who may be unable to remain fully informed for large, sprawling products.  These requirements artifacts will also form the basis for a high level dependency analysis from which relatively independent User Stories and backlog items can be derived.  This should ensure that you arrive at a position with nicely traceable project, from the high-level product owner level down to the actual code.  The presenter was not arguing for a bastardization of the Agile philosophy by doing this requirements analysis upfront, he stressed it should be performed in a just-in-time, iterative manner, and the requirements artifacts changed and adjusted as the product develops.

I am receptive this approach.  I don’t think that the Agile community should unconditionally shun traditional requirements engineering practices as “old-school” and no longer relevant.  My concern with the approach is how easy it would be to slip in to a gather requirements up-front process.  The models and artifacts could become weights around the neck of the project requiring constant revision, whilst at the same time being the excuse why something was done a certain way.  I believe that the Agile methodology, with it’s primary emphasis on conversation over documentation, promotes an environment of “ask before you act”, to ensure that the correct path is chosen.  If there is too much documentation available, this can be viewed as the definitive answer, and reduce the amount of communication within the team and clarification with the product owner.

But that is not say I am ruling this out, and in fact I think I will try and incorporate this approach in to the next complex project I work on.  I think that so long as the culture of the team remains inquisitive and conversation intensive, these models can be good aides to discussion, just like user stories.

Failure and the Agile solution

Failure.  Not a great word, and something that genuinely strikes fear in to people’s hearts.  Over the past few of months I have listened to a couple of podcasts that have offered differing views on failure.  This is a topic that has always been at the forefront of my mind, as I always wanted to be someone who was not afraid to fail.  Obviously no-one wants to fail, but if you have the concept ingrained in your psyche that failure is an acceptable part of success, then you are more likely to listen with an open mind and try new things.  But life and work is competitive, and human mentality leaves little room for failure.  I work for a consultancy where the Owners are working every day to win new projects.  Once the projects are won, they must in turn be successfully implemented.  The web application or web site must be completed on time, on budget and meet, if not exceed, the requirements.

So where can failure fit in to the software development process?

Before answering that I will quickly summarize the two podcasts that inspired this article and their differing points of view.

I.A. Podcast – Failure: The Foundation for Success

Jeff Parks interviewing Brad Nunnally

This podcast really emphasizes a need to embrace failure as part of a design and development process.  They cover three key parts to failure recognition:

1) The importance of retrospectives and documenting warning signs to learn from past mistakes.

2) Not to shy away from assigning blame and highlighting the success of others.

3) The importance of embracing failure and keeping an open mind, realizing that everyone makes mistakes

Through a combination of these ideas, Jeff and Brad believe the quality of your work will increase and you will garner more respect and a closer working relationship with your colleagues and customers. Furthermore, the very act of embracing failure will leave you more open-minded.

37 Signals Podcast – Jason Fried’s speech at BIG Omaha 2009

Jason Fried is someone I respect hugely.  He is an entrepreneur who brings a forceful clarity of vision and thought to his work.  His blog articles and podcast episodes are always thought-provoking and centered around forthright, positive action, and the distillation of problems in to simple concepts.  During this podcast he references a previous article he wrote on Signal vs. Noise titled Learning from failure is overrated, which takes a very different view of failure to Jeff Parks and Brad Nunnally.

Jason does not agree with a culture of celebrating failure that he thinks has developed over the last few years.  Critically, Jason’s view on this topic is from an entrepreneurial / business perspective, not a development team perspective, and from this vantage point he see’s embracing failure as absolutely the wrong approach.  If you are embracing failure then you are focusing on the wrong thing.  Jason says that you should focus on your successes predominantly.  If you tried 10 things and 8 failed, you should focus on the 2 successes and try to replicate that, rather than analyze and emphasize the failure.  He is not suggesting you ignore your failings, and obviously lessons should be learnt, but for progress to be made you need to be successful, so that should always be your focus.

So to go back to my earlier question, where does failure fit in to the development process?  Agile would be my answer.  The Agile process encapsulates many of the points highlighted by these two podcasts, and my own beliefs on failure, and enshrines it in a procedural guideline that failure will happen and is just part of the process.  The key tenets of this philosophy in relation to failure are captured in these points.

1) Iterations

The very reason for iterations is that there is an implicit understanding that the first time around not everything will be correct.  The best we can do is strive to complete a requirement as we understand it at this point in time.  At the end of the iteration it will be reviewed and tested as it will be a working piece of functionality.  Feedback will come from the customer and it is either signed off as done, or that changes need to be made, or new stories added to that theme.  There are no failures as such, just a cycle of build and review that will eventually result in the finished product.  This both embraces failure as part of the process but also focuses on the positives and what is right in that first release, and builds from there.

2) Iteration retrospectives

Iteration retrospectives are such an important part of the Agile process and yet one that does seem to overlooked at times.  We were guilty of this in one of our recent projects and I think we repeated mistakes as result.  A retrospective is where the cards need to be laid on the table and a open discussion needs to take place.  Each team member needs to be honest about what they did well and what they struggled with, whether that be a personal issue or one with another team member.  The failings of an Iteration are clear for people to see, but what is most important, is laying out a plan for how to mitigate these problems going forward.  Often you can do this by looking at what has worked well and using those lessons learnt.

See this great article on retrospectives also: http://www.agilegamedevelopment.com/2008/06/when-we-talk-about-agile-we-use-phrase.html

3) Team work

Brad Nunnally in the IA Podcast mentions the flat team structure of Agile projects as a negative when embracing failure because there is no one in a position of overall responsibility to make owning up to failures possible.  I believe this to be an issue with the team in which Brad is working rather than an issue with Agile teams.  Your team in an Agile project is a team of equals.  You should all be pulling in the same direction, and are responsible to yourself and each other to complete your tasks.  There is no wall over which you can throw work.  If work did not get done, whether it was your task or not, it is the teams failing, not the individuals.  Whoever was assigned responsibility for the task will have to be the one that announces the failure, in the daily stand-up or retrospective, but it is the team who bears the responsibility.  Everyone must rally round to dissect why the problem happened and how it can be addressed.  A failure occurs because it was not raised early enough, or it was not addressed when it was raised.  If the problem was never raised by an individual, that can be a case where individual failure has occurred, and is most likely a personal trait that needs to be worked on, such as talking up when you need help, something that can be difficult for some people to do.  Obviously this becomes more difficult if there are disinterested or unmotivated individuals on the team, and if that is the case then we are moving in to the area of HR rather than Agile self-organization.  Overall, working in a self-organizing, flat team of project oriented individuals, should result in a culture of union of purpose where failings are embraced by the team, along with the successes.

So shall I just go out a start screwing things up?

Well, yes and no.  I think there is value in both Jeff Parks/Brad Nunnally’s embracing of failure and Jason Fried’s embracing of success.  I think it is important to have a healthy fear of failure, so I do cringe when using the word “embrace”.  That is being a little too familiar and comfortable with that outcome.  Failure should, and always will be, a prickly and uncomfortable topic, and I don’t think that is a bad thing.  We do not want to fail, we want to succeed.  If you are too comfortable with failure, you may start to confuse progress with consistent failings.  Lets not forget, failure does have a consequence, whether that be a monetary loss, or a disgruntled customer.  Positive reinforcement is needed to make sure that what is working is captured and repeated, and demonstrable progress is occurring.  I think this quote from Fred Brooks (author of “The Mythical Man-Month“) in the most recent Wired (August 2010) says it well:

You learn more from failure than success. In failure you’re forced to find out what did not work. But in success you can believe everything you did was great, when in fact some parts may not have worked at all.  Failure forces you to face reality.

I like that last sentence, “Failure forces you to face reality.”  I absolutely agree, but the nature of reality is that we need to succeed or else we may not get another chance at failing!  So I would say, lets embrace our successes but make sure to face our failings head-on, taking positive action to ensure the same mistakes are not repeated.

The Agile BA / PM – First Impressions

So I am 12 weeks in to my new role as an Agile Business Analyst, so I wanted to give a summary of my impressions, and how the role combines with being a Project Manager and QA part-timer.

1) Documentation

No more paperwork!

This is the biggest area of change for me, and from my perspective a welcome relief.  Gone are the bulky Business Requirements Documents that I slaved over for days, only for the customer not to read, and for the team not to look at again once the review is over.  Agile is agile because it is nimble.  You do not weigh yourself down with hefty tomes, documenting every in and out of the project.  Instead, stories, and scenarios are used.  Work is done firmly at the business level.  What does the business need?  Let the Technical Lead do the interpretation.  Yes you need to be grounded in technical reality when the implementation starts,  but you don’t know why you are doing it unless you have captured the business level goal.  This can normally be captured in a sentence, or short story, and the simpler the better.

2)  Requirements Gathering as a Conversation

Requirements are a continuous dialogue.

The convention of Agile requires that the customer be heavily involved in the project throughout the lifetime of development.  Although we cannot incorporate the customer to the level we would like, we are still delivering working functionality and demos every week or two, so a constant stream of feedback is available.  This dramatically changes the requirements gathering process.  Working from iteration to iteration, as the BA I work ahead of the developers as much as possible, reviewing the stories and clarifying requirements.  However more typically I am working in the moment with developers as requirements questions come up, because they don’t know the roadblocks or lack of understanding until they really get in to working on a story.  So on these occasions, we are having ad hoc meetings with the customer, discussing their needs and working through use cases, that will result in working functionality very quickly, typically no longer than 2 weeks.  The discussion therefore remains fresh in their mind, the decision is captured and implemented quickly, and the presentation of the result is fast.  I have found this to be a very effective way to seek requirements and verify their understanding.  Nothing is ever captured and documented, never to mentioned again until weeks or months later when the product is delivered.  This is why requirements gathering now feels like a real dialogue, a never ending conversation stream connecting the developers to the business, and vice-versa, through myself and the Tech Lead.

3) Project Management

Not Project Manager, Project Co-ordinator.

This is the completely new area for me as I have never been a Project Manager before.  However, once again, Agile to the rescue! The PM role in our Agile projects is very limited, and in fact we call it Project Co-ordinator to stay true to the team ethic of Agile.  We are all in it together and not under the instruction and watchful eye of a manager per se.  The small teams, and heavy involvement I have as the BA means that I have my finger on the pulse of the projects progress.  But more importantly, the iteration development cycle means that we are never falling behind schedule or stressing about keeping Microsoft Project up to date.  We assign stories to be completed in an iteration, based on the number of points we estimate can be completed.  One or two weeks later, we know if we have hit that or not, and we estimate the next iteration based on the last.  Apart from that, I do some burn down charts, however not for all projects, it depends on the customer, as does whether they need some thorough updates as we progress, but really the PM responsibility is limited.  Most critically in this role I feel the need to stay Agile.  Don’t get bogged down in facts and figures and work that does not have business and customer value.  It is a powerful motivator to stay on track, and follow only those critical metric required for the project to succeed and to keep the customer happy.

4) QA as Usability Testing

What is the customer experience?

I have found this area to be the most challenging and perhaps, dare I say it, the least interesting part of the process.  It is a vital aspect to the process, no doubt, however my lack of training or formalized approach to this function means that I maybe create more work and more repetitive actions as I feel my way through the process.  However, I have found it is an excellent opportunity to do some rudimentary usability testing.  I put on my “I am the user or customer” hat and dig in to the fledgling software, asking the basic or simple questions of the developers to see if there is a better or simpler way to do things.  So it is not just Use Cases, bugs, issues, report, it is once again a more fluid dialogue of not only the issues found but maybe a better way to organize a screen or function during the next iteration.

Overall, I am thrilled with my new role. I am learning so much, and I am involved in so many things that I have wanted to be doing for so long now.  I will try and keep the articles coming as I truly define my place as a Business Analyst / Project Manager in the Agile software development world.

Thanks again for reading!

Great slide shows from Pathfinder

I came across these two great presentations about Agile on the Pathfinder blog, which is something I check from time to time being an Agile development shop in Chicago.

The first slide show is about integrating design in to the Agile process.  Something that is not really addressed in the Agile Manifesto, and is something I have posted about before.  It is critical area for Agile BA’s to address because the dive in and deliver functional code approach of Agile is great, but can lead to a very haphazard design with no clear vision.

The second one on Writing User Driven Requirements just really helped bring me back again to always thinking about the user.  I am certainly prone to thinking about business needs, or developer/technology constraints only, but the user should always stay front and center.

The Integration of Design and Agile Development

View more presentations from Pathfinder Development.

Writing User-Driven Requirements

View more presentations from Pathfinder Development.

Agile and User Centered Design

Nielsen Norman Group

This new article from Jakob Nielsen was an interesting read. It relates back, in a roundabout way if you really squint at it from a distance, to my previous posts (here and here) on where the Business Analyst fits in to the User Centered Design (UCD) process. Nielsen is looking at the problem from the perspective of how UCD fits in to the Agile development lifecycle. The focus on a rapid sprint style of development leaves little room for user engagement and testing. Furthermore the concentration on small sprints to achieve functionality can be at the expense of a unified, user-focused design and information architecture. Unless of course that UCD is integrated in to the project from the beginning. This is Nielsen’s point.

The BA gets involved in many of the areas that UCD preaches (requirements gathering, wireframing, information structure, process flow etc.) but he/she is not a designer, so my query was how they fit together? Nielsen makes this comment:

“it’s important to designate a gatekeeper to track requirements and communications between the UX team and the other project teams to keep everybody on track (even though those tracks are parallel).”

This would be a natural starting point for a BA to again be that bridge between two groups. Although it sounds like a Project Manager type role at first glance, I see greater scope for engagement and influence in the project, and so more of a BA type role. To be the “gatekeeper” between the UCD team and the development team. To integrate the user and design requirements from the UCD side, and working within the information architecture and design principals, bringing that to bear on the functional requirements and delivering that to the development team.

Just a thought.