October, 2008

Role of Software Reliability Estimation in Software Design

Recently, I had to decide about a design issue. A new functionality had to be implemented. Two modules were candidates for that. So, I had to chose either to implement it in one or another of these two modules. Obviously, such a design decision has to observe certain criteria. I have listened to my team leaders debating about this issue during a design meeting. Some of them have chosen one module while the others have chosen the other module. The most of their arguments were objective. And all of them had in mind real-time constraints and future maintainability of the whole system. Finally, I decided that the functionality had to go in one of those modules. In this article, I give account of how to deal with such a software design process.

My role as leader of both development and quality assurance at the same time allows me to put a bridge between the development and the test activities. In design meetings, I often come with more information about the quality of the modules we want to change and this helps to take the right decisions. As I said above, my team leaders have focused on software capacity (real-time) constraints and long term software maintainability. This is a good practise. But what I was able to figure out as another important decision criterion was the software reliability. And I had this information because I am always involved in the testing process.

Let’s now abstract the problem: how to put together the estimated software capacity, maintainability and reliability in order to solve such a software design problem? I would say that it depends on the software development context. The software production I’m personally involved in has objectives and constraints that are different from those which rule the software development in other organizations. That’s why another software architect in another workplace would not order those three criteria as I did. In regard of the nature of those two modules I had to choose among and the degree of capacity and maintainability they have already reached, reliability was more important than maintainability and capacity. Moreover, from what I know about those two modules’ maintainability, one among them was more maintainable but less reliable due to the specific environment (third party libraries ) it has to interact with.

My software design problem needed those three criteria to be addressed but other design situations may involve other software quality elements like robustness, for example. Whatever the criteria are, the designer has to keep all of them in mind. I personally prefer to write down criteria instead of relying on my unconscious reflexes to carry them.
Let’s now focus on software reliability estimation. As I said above, I got that information from my software quality process. Yet in the latter, reliability estimation is not often straightforward. Depending on the workplace’s goals and constraints, the software quality manager has to decide about the meaning of the software reliability conceptual elements which are the following:
- Failure’s severity (major vs. minor).
- Failure’s impact (system wide vs. local).
- Failure’s probability.
- Likelihood of recovery, time to recovery, extent of recovery (complete vs. partial).

Having well defined all those conceptual elements, the manager has to organize them into the software quality process and integrate them in his/her software quality metrics. The first two variables (failure’s severity and impact) could be available in the bug tracking system. The two other variables could need to be extracted from the system’s logs, usage statistics, and customers’ or beta testers’ reports.