A language agnostic debugging list
Updated: 4 days ago
Checklists are nice cognitive tools. This one can also work as a reminder when designing code. Please come with feedback and suggestions for improvement.
It is not always the case that we immediately spot the reason for an error in a codebase. Sometimes it is hard to find due to the design. In other cases the definition of correct behaviour is the problem. It might also be a lack in understanding when it comes to the technology used. This checklist is useful regardless of your knowledge of the language you are debugging. Depending on domain knowledge, technical knowledge and the state of the code base, how long the steps take and what information they give, will vary. This is, as most knowledge work, best done together with someone else, or with an entire team.
Write a test, of some kind, that captures the unintended behaviour. In worst case it is a description of steps to take written on a piece of paper. Best case is a unit test. This will define where we see the problem, with its setup and the state it leaves the program in.
Do we know for sure that this is a problem, not just a state where the behaviour is not clearly defined? Make sure that we know what the code is supposed to do in this particular case.
Are we sure that this is where the problem is caused? Check that the indata is correct, and that the data out is not as expected. Try to narrow it down, define as little code as possible where the error might be.
Has it ever worked? If it has, what has changed since then? Version control is your friend here.
Is the IDE saying anything? A real development environment uses partly the same algorithms as the compiler/transpiler/interpreter, to be able to give hints and syntax highlighting. There is also often static code analysis tools that can highlight practises that make the code fragile. Use tools.
Are there inputs, not stated as such, in the code we are looking at? Some code use state that is external in some way. One way to check this is by trying to write a unit test, instead of an integration test. What do we need in our setup to make it work? The state could be a global variable used by a collaborator, todays date, or some settings in the system. Make sure that we know all the actual input data to the code we are investigating. Try to make the implicit input explicit when testing the code.
If there is comparing going on, is it done correctly (for that language)?
Do we know what could be null/nil/empty? Keeping null-checks at the boundaries of code makes this easier. Use the tools the language provide to deal with missing data.
Are data-structures used as intended? If iteration happens, are the boundaries the right one? Do the data-structures have any quirks in this framework?
Are there other language features that could be a problem? In Python, default arguments can turn into singeltons. In ObjectiveC a method call on a deallocated object will return 0, or nil, but succeed. Try to rewrite the code, even if it turns it bulkier, just to see if misdirected "cleverness" might be the problem. Then read the documentation to understand what happens if the behaviour changes.
Is the flow of execution as we think it is? Use a debugger to step through the calls, following the control flow. Test cases can also be run in debug-mode. Make sure that there are no code skipped. One example is if code is executed as part of a condition, that is already evaluated as true, and therefore skipped.
Are we working with asynchronous code? If we are, then make sure that it is done correctly. Test ways to make sure that all of the suspect code is run synchronous, to eliminate logic errors that are not timing related, even if the code can never run like that in production.
Take a break, if you have not already. At this point I might start searching the internet or try writing a question at Stackoverflow (often the solution is in writing the post, not posting it.) I might also call a friend. My experience is that the harder the answer is to find, on the internet, the more likely it is that the misstake made is really basic.
Hope this list helps. I will use it the next time I get stuck on code not working as I intended.
How do you find your mistakes? Do you have anything to add? Have I missed anything language specific that is not included in the steps above?