As I mentioned earlier, before we were able to finalize our new development environment we were gobbled up (acquired) by another company, about 4 times the size of our group.
In the area of issue tracking, the bigger company was using TestDirector, with some customizations, but actually their processes were quite simplistic, and weren’t enabling a truly effective R&D; process.
For Test case management, they are using plain old Word/Excel but are now open to other options.
In the area of SCM btw both companies were using good ol’ CVS and were quite sick of it. more on that later…
With this as the baseline, upcoming posts will try to describe the process for integrating SCM, Issue Tracking and choosing a Test Case Management solution agreeable and effective to all.
At some point we understood we must have a robust test automation harness which can at least cover our smoke test and regression. This will help us feel more confident in our releases in less time, and allow meeting the business needs.
As we are an appliance based file-system product, in essence an IT infrastructure product, all of the commercially available harnesses from CA, Mercury and the like are useless, as they focus on GUI/Web automation, and we need API automation and the ability to run and control file system operations and file system testing tools.
We considered home-grown approaches but decided that the time-to-market is too long for our needs.
We considered adopting STAF/STAX (http://staf.sourceforge.net/index.php ) but again the custom work needed around it was estimated to be too long and required human resources and expertise we didn’t have, and weren’t available in the neighbourhood.
What we eventually chose was a testing
automation harness called Aqua
(http://www.aquasw.com/) and we are very satisfied with it.
Its still requires significant customizations/development in a project mode, rather than an off-the-shelf product, but for some situations its the best and only available approach today, and is much better than developing on your own from scratch, or testing manually.
About 8 months ago we chose TD (TestDirector) for Test Case Management as we needed to employ something fast and the Director of QA I brought in had experience with it, while the open source tools we looked at didn’t convince us at that point, and seemed a risk we didn’t want to take, considering the other work necessary in other QA areas at the time.
NOTE: I wonder how much money Mercury made just from people in this situation…
Anyhow we are now documenting test cases and managing testing progress with the tool, with moderate satisfaction. The lack of integration to our issue tracker (both Bugzilla and JIRA) is a concern, as well as the high admission price per user, and the feeling that we are working with a glorified MSAccess inside a web browser (I wonder why…).
Lack of support for Linux machines, Firefox, and in general the fact that Mercury (sorry, HP now… http://www.mercury.com/us/company/pr/press-releases/072506-hp-acquires-mercury.html) is evidently considering itself more of an IT Governance / BTO company, and outgrew its QA roots, doesn’t make this move seem very strategic.
I’m not saying TD is bad, its just not the right solution for small dynamic groups, which want to integrate new solutions when they solve business problems, and where much of the environment is open-source or small vendors, not the gorillas of the Enterprise Tools segment.
We made a tactical move knowingly, but there might be room for change, as we are not fully invested in the tool at this point.
Sometime before the acquisition, we decided on JIRA as the issue tracker and started evaluating it, including history migration, a bit of customization, and trying to understand what processes need to be in place in order to be effective and deal with the complexities we were seeing regarding real world software development and maintenance.
This was an easy decision, as we really really like JIRA the more we look at it. The main advance in the JIRA space this year was in my view the plethora of plugins that became available, and provide functionality we needed, as well as convince us of the strategicness of the JIRA/Atlassian ecosystem.
Another easy decision was migrating to Confluence, the atlassian enterprise Wiki, for our knowledge base and documentation platform.
Another thing we are seeing is commercial vendors in the development tools space (SCM, Build management, etc.) integrating with atlassian. This is very positive and not surprising, considering the openness of the platform, the reach into many customers, and I guess the BizDev efforts of the atlassian guys, which I think try to be Best of Breed for issue tracking and wiki, while relying on very strong partnerships with every sexy major player/community in the space (not necessarily the heavy-weight guys like Mercury/CA) to provide other functionality people expect.
Long time no post.
I’ll try to recap where I stand regarding the issues I started talking about a year ago, at least for a bit of closure regarding the issue tracking and test case management areas.
Lets see if I can keep it up this time…
In any case, my company was gobbled up by another software company, and so somewhere in the middle of upgrading our development environment infrastructure, our plans were disrupted, but one cannot complain… Now the new company R&D; department needs to have an issue tracking and test case management, with the added challenge of integrating two methodologies, histories, and views on the world. I was tasked with this project. in the following posts I will try to address several aspects of this project.