 The purpose of this Evaluate Solution Performance (BABOK 7.6 – the last task included in the BABOK Guide) is:
“Evaluate functioning solutions to understand the value they deliver and identify opportunities for improvement.”
This task involves understanding the business value devlivered by the solution and whether it is over- or under-performing, validating that value with metrics, and deciding whether the elimination or replacement of the solution is necessary.
Many BAs read this and think, “I’ve never done that. As soon as I’m done with the requirements, I’m assigned to another project. I never have time to go back and evaluate the performance of the solution.” And you might be right.
I read it and think, “I do this all the time, just rarely at the end of a project; always at the beginning.”
Let me share an example with you from my real-world experience:
When I started working as the first true BA at a company that had recently purchased, but not yet consolidated, 5 smaller companies, the first thing we did was complete an assessment of each company’s business processes and systems. After weeks of work and lots of travel, the result was 5 individual reports and one consolidated report showing overlapping functionality, strengths, and issues. This report contained a whole lot more than a solution assessment (much of it would be considered enterprise architecture by the BABOK) and was not quite so formal as the BABOK describes because I don’t recall including too many metrics, but it did begin to make a case for replacing the 5 independent technology stacks with a single consolidated technology stack. By learning about the key challenges each organization faced and the limitations of their technology systems, we had the seeds of a plan to begin enterprise analysis for a multi-million dollar initiative.
Ideally, you’ll evaluate the performance of the solution before the project is declared “done” (something we talk about in the business analysis process), but it doesn’t always work out that way. This is another good reminder that the BABOK is not a methodology, or a process, so it’s perfectly OK to start at the end.
This post is one installment in our Journey Through the BABOK with BA Stories series.
>> Learn the Business Analysis Process
An essential element of succeeding in a business analyst job role is understanding the business analysis process. We walk you through an 8-step business analysis process in the 4-week self-study session of the BA Essentials Master Class. You’ll learn a step-by-step business process that you can customize to meet your organization and project situations, how to create a timeline for a new business analyst assignment, and be prepared to handle the more common issues BAs face on new projects.
Click here to learn more about the BA Essentials Master Class
Laura, Thank you so much!
Michelle,
That print copy is going to be yours. I’ll get it in the mail next week. Cheers!
Laura
Laura
My experiences have been pretty similar to what you mentioned, as they have been centered around evaluation of new acquisition functionality and then performing a compare/contrast exercise against the enterprise system. This highlights a couple of things: a)what each system can or cannot do and which does it better, and b) a general measurement of what each system is capable of handling. We are then able to assess whether to migrate functionality into the enterprise fold or convert the acquisition using the enterprise functionality. Also, we are able to use measurements of non-functional requirements for the acquisition in their current environment (e.g., number of transactions processed) as a baseline to ensure that we are improving performance with whatever solution is decided.
We also undertake a similar exercise when evaluating processes in the acquisition, which actually occurs before the technical assessment. In this case, we are seeking process inefficiencies and deficiencies to work with.
So I suspect that when a lot of people comment that they are performing gap analysis, that technique might actually be broken down into gap analysis and some degree of solution evaluation…at least on the front end of requirements.
I also agree with your statement about not always having the opportunity to evaluate a solution after the implementation. It always seem like there is a more pressing need with the limited resources we all work with these days.
Doug,
It sounds like despite our disparate domains we share some similar experiences.
Great point too that many gap analysis activities likely include an element of solution assessment. That makes perfect sense to me. Does this trigger any bells for anyone else reading? Perhaps what you’ve been calling gap analysis relates back to this BA task too?
Hi Laura,
Yes I worked with metrics to measure the performance of the system and the orders. Our deliverable was to reduce the order cycle time from 240 days to 90 days or less. We also worked to ensure that they key strokes involved in creating and maintaining an order were reduced and were measurable.
For example, we heard from the business that the system was causing them more time due to lack of training but also due to the workflow of the order. I worked on training and delivered the training, then we worked on the workflow to ensure that the workflow made sense in the order process.
Sounds like a great use of metrics Michelle!
I consider myself very fortunate with my first experiences as a BA. I didn’t even realize the process that I followed was not the same in each company. Each project I received or identified was worked on by me from inception to the solution, training, testing, warranty and follow up and evaluation.
My best example involves a system that was developed by the ‘quick wins’ team. This system was developed quickly (agile methods) by functional analysts during a work stoppage situation. The team had the buy in from the executives to move forward with the solution. The system purpose was to create and track data orders for the pipeline of order fulfillment – to give all areas of the business visibility of each order at any given time.
It was brilliant – however, as with all systems, once the real workers arrived back to learn it – many deficiencies were discovered. I had the opportunity to evaluate the system and work with the business and the programmers to enhance and change the system so that it worked more effectively.
When I started the project I began working with the programmers to understand their purpose for choosing this solution. I then worked with the business to understand what concerns they were having and what could work better. So, a lot like elicitation.
I then learned the system – quickly so that I could discuss it with intelligence and basically became a super user. As I tested orders and understood the purpose of the system and the ways we could enhance it – I was very excited. This solution was amazing, developed in house by an amazing team and in such a short time.
Working with the business and the developers we met face to face and then as we were all in different locations in the country, teleconference meetings occurred. The evaluation showed that this system was working and could be more efficient if we listened to the business.
The difficulty of developing a solution without the business is that when you roll the solution out – you discover all the deficiencies. The evaluation process presented me with a road map to help the developers to understand the business point of view and to help the business see the value in the system and to let the system evolve with them. It was amazing! We made so many positive changes and did a great deal of training to deliver an enhanced solution that was bought into by the business.
What a great story! Thanks for sharing Michelle. This does indeed seem to be an example of evaluating solution performance. When you say you were able to work with the business stakeholders to understand their needs, were there any metrics you were able to gather to use in the evaluation? Are there any specific opportunities for improvement that stand out in your memory?