Software today is full of features. Especially where I work. The product has more features than those of anyone of our competitors in the industry.

In pursuit to keep up with the new feature demand in the industry, I think we lost track of the usability portion. Our tool is great for regression testing, but the sales figure does not come up to our expectation. With regression testing, developers are looking for red flags that are worth paying attention to.

Being able to use a feature is definitely great.  However, being able to use the feature to save developers’ time is another concept altogether.  This is usability.

For example, we have a reporting tool that provide data on all the tests that have run previously.  Some customers use it for Sarbane-Oxley compliance.  However, the report presents literally “data”, not “information”.  The report shows date/time of a specific test being run, the results of the test, verification (pass/fail).  If you have months of data in the database, the report will shows a lot of data (very likely a mix of passed and failed tests).

A report such as this has too much noise. No one, unless circumstances demand, will go through the list of tests and really examine every fail test.  The point to provide the report is for a test engineer at first glance, able to tell what type of tests passes, what type of tests fails.  It should at least be able to tell how many percentage of priority 1 tests passes, and etc.  If you have 20% passing rate on your priority 1 tests, you know you have some serious issues with your system.  If you have 100% pass on your priority 1 tests, you can leave office on time today.  This is “information”.  A piece of knowledge on the overall, high level outcome of the regression test.

Unfortunately, it will take a few more release before we can get it right.