I’ve recently been looking at how to build a reasonable test case management solution (good!=word documents) for our company. I quickly learned this is not a very developed field. Mercury TestDirector seems to dominate the commercial field, with the other QA product companies (CompuWare, IBM-Rational) following suite, but not there yet.
Test Case Management as I understand it deals with the following objects:
* Test Plans
* Test Cases
* Test Labs
* Test Schedule
The main deliverables expected from a Test Case Management solution are:
* It should provide visibility into the testing process – both the plan and the actual execution.
* Is risk-oriented, focusing on the riskier aspects of the system first.
* Facilitates day to day management of the testing team.
* It should present a coherent picture connected to the Issue Tracking tool (what bugs are blocking tests, what tests need to be ready for a certain release, etc.)
* It should present a coherent picture connected to the Automated testing frameworks (If the smoke test / sanity failed during the night, the Test Case Managment should reflect that without requiring the QA engineer to manually copy+paste the information).
* Tracking pass/fail status for each test
* Tracking pass percentage per module per version per milestone requirements
* Dynamic priority management which affects to do list for testers
* Coverage of requirements by test cases (each change request should be linked to at least one test case ? Closing a change request requires successful test run ? )
* Management of test beds relevant for each test case
* Manage test cases that are blocked by other change requests (bugs/enhancements)
* Accessibility to test case information from each bug / change request.
* Accessibility to test logs from the relevant test case instance
* Manage the relationship between different test cases – its quite useful to create dependencies – e.g. run the login test case then run the change password test case.
* MANY test case instances for ONE test case in ONE version
* ONE requirement can be tested by MANY test cases
* ONE test case may test MANY requirements ?
* ONE test script may be used in MANY test cases
* ONE test case may run in MANY configurations
* Time estimate for coverage of a version
* Last time a specific test was run, by who, what results
* specific test case results across builds
* Ability to share the test cases with an OEM or remote team
* Version control for test scripts (link to the SCM)
* What version of the test script was used for each test case instance ?
* What version of the test case was used for each test case instance ? (if we added a sequence, and suddenly tests started to fail, it doesn’t mean regression in software)
My understanding of this is based on some resources I’ve been monitoring (see the full list on ).
Some of the noteworthy ones are:
* StickyMinds.com : Article info : Reengineering Test Management
* StickyMinds.com: Bringing Your Test Data to Life
* Rhonabwy writes about his own experience with the open source test case management tools from time to time
* OpenSourceTesting – Test Management Tools is the list of test case management tools everyone refers to.
Based on the information I found I’ve been looking at TestMaster, TestLink and QATraq, but didn’t install any of them yet. The other ones really seem either dead or not ready yet.
I’m still trying to understand whether the correct approach is to get a test management tool, and try to connect it to your issue tracker, or to get a really customizable issue tracker (e.g. JIRA) and build what you need of a test management tool there. I’m still contemplating the pros and cons, and trying to understand how much of test case management is actually an issue tracking type activity, and what are the parts that are not. This is quite uncharted ground from what I’ve found so far, and I understand that part of having a “reasonable” solution is to skip some of the requirements and vote for simplicity.
A good friend which knows what he’s doing when it comes to managing QA efforts repeatedly tells me to aviod the bells and whistles and the complex reports metrics and processes, and go for simple worthwhile metrics, the reports and flows that are necessary to support them, and to focus on the substance. Thats a big part of what I consider to be a “reasonable” solution.
You’ve certainly got a good understanding of what a test case management system should provide. I was in your shoes ~18 months ago and took the development route. I’ve since created a tool (Qualify) that meets most of the requirements you specified in your post. I’ve started selling the product via my new company – SQA Design. Check us out at http://www.sqadesign.com. I’d be happy to provide you with some free licensing for your feedback.
– Charles West
I would gold für wow cultivate courage.buy wow gold “Nothing is so mild wow gold cheap and gentle as courage, nothing so cruel and pitiless as cowardice,” syas a wise author. We too often borrow trouble, and anticipate that may never appear.”wow gold kaufen The fear of ill exceeds the ill we fear.” Dangers will arise in any career, but presence of mind will often conquer the worst of them. Be prepared for any fate, and there is no harm to be freared. If I were a boy again, I would look on the cheerful side. life is very much like a mirror:sell wow gold if you smile upon it,maple mesos I smiles back upon you; but if you frown and look doubtful on it,cheap maplestory mesos you will get a similar look in return. Inner sunshine warms not only the heart of the owner,world of warcraft power leveling but of all that come in contact with it. “ who shuts love out ,in turn shall be shut out from love.” If I were a boy again, I would school myself to say no more often.billig wow gold I might cheap mesos write pages maple meso on the importance of learning very early in life to gain that point where a young boy can stand erect, and decline doing an unworthy act because it is unworthy.wow powerleveling If I were a boy again, I would demand of myself more courtesy towards my companions and friends,wow leveling and indeed towards strangers as well.Maple Story Account The smallest courtesies along the rough roads of life are wow powerleveln like the little birds that sing to us all winter long, and make that season of ice and snow more endurable. Finally,maple story powerleveling instead of trying hard to be happy