When is firmware good enough?

May 5th 2018

The question came up recently about when is firmware good enough to ship a product? 

This can be a difficult question but ultimately boils down to your product's tolerance for failures. For example an anti-lock brake controller is something that most people want to work every time needed with zero tolerance for failures. While a kids toy car that stops working and needs a power cycle would be more tolerant of a firmware bug. 

So to help people make this decision here are a few questions to ask about your product.

What is the cost to you or customer if the product fails?

In the anti-lock brake example it could cost the life of the customer, so if the cost is high firmware quality must be high as well. 

Can firmware update be done in the field and what is product cost? 

This is a compound question in that if you sell a toy flash light for $3 that has a bug in firmware, they are so cheap that it is cheaper to throw the product away and get a new one than do firmware updates.  However if you make a data logger that cost a few hundred dollars it might be acceptable to have a firmware bug on the first release as long as customer can upgrade firmware latter to fix issues. 

What is the cost of releasing bug fixes?

So not all bugs are created equal, some are easy to find and fix and some are very hard to find and fix.  However often even the easiest bugs to fix have a heavy cost. Specifically many organizations require products to go through an extensive test process before shipping.  Note, if you don't require some testing then you need to.  Some of the test are time and resource consuming. So even a simple bug in the firmware might take weeks of testing before it can be released to production.  If the cost of releasing new firmware is high then the quality of the firmware needs to be higher. 

For example if you are designing a medical product then often the cost to customer if it fails is high, the cost to test new firmware will most likely be high and so even if the firmware can be updated in the field the firmware has to be robust.   

Another example might be a GPS tracker for a Dolphins which are attached to the Dolphins and then latter retrieved to get data off the unit. So the cost of losing the data for the customer is high, and the firmware can not be update while it is in use. So the firmware needs to be good. 

Another question that comes up along the same lines is how good is the firmware?