SLA based model for testing
Paper Presentation at STeP-IN SUMMIT 2009
Ramanath Shanbhag, Deputy General Manager - Testing Practice, Aztecsoft (a MindTree Company)
Customers’ confidence in our capabilities and accelerate trust in our relationships with first-time customers. Giving a customer the comfort feeling early in the relationship, providing assurance about what they expect, and a general commitment on how we would be conducting business can be leveraged through SLAs.
Committing to SLAs is like gambling; wagering on how the skills and competencies of our testers will address customer needs. If we have an objective understanding of our testers’ capabilities with respect to customer needs, we can consider taking the calculated risk of offering an SLA, and assuring the customer both that what we have committed is achievable and that we have a plan for cases when we may not meet the first-tier objective. The end goal of the SLAs is customer trust and agreement on the work to be done in greater detail than if only top level goals are specified. For example, using SLAs like number of test cases executed per person day (or resource day/time unit), number of defects found per person over unit time, number of test cases automated per resource time, defects found in test automation code, or test case effectiveness- defects found through test passes Vs. ad-hoc method can give a customer better visibility on the service offered.
Risk / reward balance is key. Good SLAs are specific, quantifiable (a known maximum exposure), and measurable. The best SLAs are all these and we internally are confident that we (not the customer) will be the first to notice failure to meet the agreement so we can address the issue proactively. It is also important to create a culture of awareness of SLAs in which all internal and external stakeholders and project participants understand the SLAs and report potential failure to achieve them.
In this presentation, we’ll describe a method we use to create SLAs from before customer engagement, during the RFP process, and through project execution and completion.
Committing to SLAs is like gambling; wagering on how the skills and competencies of our testers will address customer needs. If we have an objective understanding of our testers’ capabilities with respect to customer needs, we can consider taking the calculated risk of offering an SLA, and assuring the customer both that what we have committed is achievable and that we have a plan for cases when we may not meet the first-tier objective. The end goal of the SLAs is customer trust and agreement on the work to be done in greater detail than if only top level goals are specified. For example, using SLAs like number of test cases executed per person day (or resource day/time unit), number of defects found per person over unit time, number of test cases automated per resource time, defects found in test automation code, or test case effectiveness- defects found through test passes Vs. ad-hoc method can give a customer better visibility on the service offered.
Risk / reward balance is key. Good SLAs are specific, quantifiable (a known maximum exposure), and measurable. The best SLAs are all these and we internally are confident that we (not the customer) will be the first to notice failure to meet the agreement so we can address the issue proactively. It is also important to create a culture of awareness of SLAs in which all internal and external stakeholders and project participants understand the SLAs and report potential failure to achieve them.
In this presentation, we’ll describe a method we use to create SLAs from before customer engagement, during the RFP process, and through project execution and completion.