Influence of stakeholders on testing

Wikipedia defines Software testing as "an investigation conducted to provide stakeholders with information about the quality of the product or service under test."

A stakeholder is someone who has a 'stake' or 'interest' in the product being developed. The stakeholders of a project influence functionality being tested, non functional aspects of the product, resources (tools & technology) used for testing and even time and team size. Somethings like the time and team size aside, understanding expectations of stakeholders from any product can really help testers add value to the quality of product.

For adequate coverage the testing team should ‘think’ like a stakeholder while writing and executing test cases. I’ve seen issues discovered by the end users after the product went live, which should have been caught by the team who tested requirements. One of the primary reasons for this is not ‘thinking’ like a stakeholder.

Let's take an example to understand this. Consider a scenario where ABC company is launching a website to manage their sales orders and delivery. This website will allow customers of ABC to place orders on the website, ABC users to track deliveries of goods and payments received. The management reports section will allow senior managers to view revenue specific reports. SuperDevs company has been awarded this development project and they are further outsourcing the testing to BestTest company.

Some groups to look for while identifying stakeholders in this case would be:
End users - People who'll actually use this product. The customers of ABC Inc., staff and senior management. Most often these are the people who’ll give project requirements. People from this group influence core functionality of the product. More often than this this the only stakeholder group whose needs are validated by the testing team in their testing.
Customers would need to

1. Search for products and place orders. Might involve logic for discount calculation, shipping charges calculation etc.
2. Make payments over the internet securely
3. Edit details like shipping address etc., sometimes after placing the order
4. Track production status and delivery schedule (expected date of delivery)
5. Cancel an order. This might involve cancellation fees to be levied on the customer

Staff members from various departments would have different needs like

1. Production dept staff would need to know open orders requirement to plan production schedule
2. Marketing people would need to know which items are in demand and which are not, so they can create discount schemes etc to move the slow items
3. Customer support people would need to know the status of open orders and product details of orders that have been delivered so they can answer support queries, handle returns etc.

Executive staff would need special reports to give them revenue details as well as forecasts about orders so they can take intelligent decisions regarding budget and strategy.

Project Sponsors – Project sponsors have a value-for-money attitude towards a project. They influence timelines, team strength, budget and quite often tools & technology used by the dev and testing teams.

They do have an interest in the competitiveness of the product in their own niche market, though. This area is often overlooked while testing. It is a good idea to have someone research the available market tools or ideas so that the product can have solid requirements and better features. A good way to log those ‘feature requests’ during requirements testing phase is to do this home work and come up with ideas. Wear an entrepreneurs hat while you do the research, so you can get most of your feature requests approved! :)

ABC company’s IT staff – These people would typically co-ordinate between ABC company, SuperDevs team and BestTest team. They’ll have a stake in making sure the project runs smoothly and is delivered on time. They might not influence the functional requirements as much as they would lay down the non-functional ones.

Go talk to some of the IT team members before you write non-functional test cases like expected response time, number of concurrent users etc. They’ll also be your best guides in setting up test labs. Another overlooked area of testing. Several projects run into major / critical issues because the end users’ environment is different from the ones used by testing folks to test.

The development team (SuperDevs in our scenario) – Yes! They have an interest in the quality of product delivered, therefore they are also project stakeholders. Though they do not influence the functional aspects of a product, they’ll definitely make a difference to it.

Be it requirements testing, design & code reviews, buddy (pair) testing - there are several ways you can work alongside developers to improve quality of your product.

The testing team (from our example, the BestTest folks) – Are they one of the stakeholders? What do you think?

Are there any other stakeholders you can think of?

I’ll leave these questions open for you to mull over. Do share your thoughts in the form of a comment so we can discuss and debate.

This article enlists some of the areas which a testing team can better cover. Testers and test managers, next time you are assigned a project you can create a similar list of stakeholders and make sure you cover everyone’s interests. You’ll surely see some reduction in issues that escape to the production environment.

Happy testing :)

Enjoyed reading this article? You’ll like receiving more tips and testing related information in your e-mail. Please subscribe by clicking here. You’ll also be helping me in reaching my goal of having 1000 subscribers by Dec 2011. You have my word for no spam and complete privacy of your information.

Use what you have

In one of the development projects where I was the test manager, I faced with a challenge. My team size varied through out the development cycle. Here’s how:

Between three test engineers and myself we analyzed requirements, tracked issues thereof, designed test scenarios and test cases. Three additional test engineers joined us during execution and release phase. Since the dev and test were independent business units, the test engineers were typically loaned resources who were assigned on availability basis.

We were faced with the challenges of frequently changing test engineers besides the package deals that come with testing a data intensive, high impact financial auditing software.

Here is how we two tools available in our development environment to help us improve productivity:

1. SQL Server:

    Our software accepted data input in the form of XML files. We created bulk test data (based on some sample data provided by our clients) in SQL server tables. Then used simple “For XML” clause to generate basic XML file. Minor find-and-replace tweaks generated our actual input file. We then attached these test data files with each test case in our test case management tool for use during the execution cycle. Now the newbie engineers joining us had simply to use this data to verify the expected results.

2. Share point server

    We were given a project website on SPS for our internal collaboration. We created lists on this site:
      - Queries list: Queries regarding requirements or a behavior were logged and tracked here. Each query had a user to whom it was currently assigned to. Test manager monitored aged queries and made sure they were resolved satisfactorily. Often these queries ended up simplifying requirements that were conflicting or ambiguous, or helped improvise technical design. Sometimes the query turned out to be a genuine issue, so it was transferred to our defect tracking tool. We made sure to add the test case / defect id from other system into this list so we could establish traceability.

    - Test scenarios list: Before we created detailed test cases, we nailed down the top level test scenarios. This helped us gauge the extent of testing needed and get a feel of changes being done to the system in each iteration. This list also served as the introductory chapter for new members.
       Each item in this list had 3 owners assigned to it – the business analyst owner, the dev owner and the QA owner. The QA owner was typically the one who created this scenario. The dev owner was the one who would use this scenario during his / her preliminary testing before marking a piece of code as ready-to-test. The BA owner would come into the picture is anyone from the team asked clarifications on an item or if the requirements document needed updates due to a scenario.

The advantage SPS provided over .xls sheets was that we did not have to juggle shared workbooks and private copies. All info was kept in one location for every one to see. We also had our team members set up alerts on each of these lists so as to get notified of any changes.

These are just two of the ways we used tools in an unusual way to meet our specific needs. Do you have any more of such examples you’ll like to share with our readers? Feel free to add your comments.

Enjoyed reading this article? You’ll like receiving more tips and testing related information in your e-mail. Please subscribe by clicking here. You’ll also be helping me in reaching my goal of having 1000 subscribers by Dec 2011. You have my word for no spam and complete privacy of your information.