CheckPointer compared to other Memory Safety checking tools
The SD CheckPointer tool finds memory management errors in programs with explicit allocation, deallocation and pointer dereferencing operations. Here we compare this tool with other tools that accomplish related analyses. For each tool, the union of Errors Detected and Errors Missed are intended to cover all possible memory safety errors to enable effective tool comparisons.
The table should make it clear SafetyCheck is stronger than all the dynamic checking tools, and will detect any statically-detectable error that occurs in some actual execution run.
| Tool Name | Language Analyzed | Analysis Type | Errors Detected | Errors Missed | Status/ Availability |
|---|---|---|---|---|---|
| CheckPointer | C (ANSI, MS, GNU) C99 (ANSI, GNU) | Dynamic using compile-time instrumentation | All runtime memory safety access errors including outside globals, locals, heap, strings, arrays or struct elements. Memory leaks | Statically-detectable errors in un-executed code | Commercial Windows Linux (under Wine) |
| CCured | ANSI C with restrictions; structs containing pointers change layout breaking compatibility | Dynamic using compile-time instrumentation | Runtime access errors outside globals and heap-allocated storage; dangling pointer errors | Errors in structs whose layout changes due to "fat" pointers; statically-detectable errors in un-executed code | Research Linux? |
| SoftBound | Clang (ANSI?) C | Dynamic using compile-time instrumentation | Runtime access errors including outside globals, locals, heap, strings, arrays or structs | Temporal errors (double free); errors involving C library block-memory moves (e.g. memmove); statically-detectable errors in un-executed code | Research Windows, Linux |
| MSCC | ANSI? C | Dynamic using compile-time instrumentation | Runtime access errors including outside globals, locals, heap, and temporal errros (e.g., double free) | Problems with bad casts on dynamically allocated blocks statically-detectable errors in un-executed code | Research Only applied to small programs Linux? |
| SafeCode | ANSI? C | Dynamic using code transformations | Prevents, not detects, memory access faults for dangling pointers | Statically-detectable errors in un-executed code | Research/Linux? Clang variant? |
| Insure++ | ANSI? C | Dynamic using compile-time instrumentation | Statically detection of some uninitialized pointers Runtime access errors including outside globals, locals, heap; memory leaks | Erroneous access to recycled heap memory; statically-detectable errors in un-executed code | Commercial Windows?, Linux, AIX |
| Bug Fighter | C (GNU, MS) | Dynamic using compile-time instrumentation | Most runtime array subscript errors; null pointer dereferences | Errors for certain array access syntax; other pointer-related errors; some array index errors; statically-detectable errors in un-executed code | Breaks some larger programs Freeware Windows |
| Bounds Checker | Windows C, C++ | Dynamic using compile-time instrumentation | Runtime array subscript errors | Errors for certain array access syntax; all pointer-related errors; statically-detectable errors in un-executed code | Commercial Windows |
| Purify | Object code: several machine architectures; not Visual Studio C++ | Dynamic using load? time instrumentation | Runtime accesses outside global, stack or heap-allocated storage; accesses outside program, stack or data segments; free bad pointer; double free; memory leaks | Runtime access errors into reused heap storage; access errors to local variables, or within a struct; statically-detectable errors in un-executed code | Commercial Windows, Linux, Solaris |
| Valgrind | x86 object code | Dynamic using runtime traps | Runtime access outside of heap allocated variables; accesses outside of program, data or stack area | Accesses outside of global or local variables; accesses outside of elements of structs; statically-detectable errors in un-executed code | Open Source Linux |
| DieHard | ANSI(?) C | Dynamic replacing malloc | Statistically, most runtime accesses outside heap-allocated variables | Runtime accesses outside globals, locals or struct elements; statically-detectable errors in un-executed code | Research Linux? |
| Electric Fence | Object code: x86 instruction set | Dynamic using load? time instrumentation | Runtime accesses outside global, stack or heap-allocated storage; accesses outside program, stack or data segments; free bad pointer; double free; memory leaks | Runtime access errors into reused heap storage; access errors to local variables, or within a struct; statically-detectable errors in un-executed code | Freeware? Linux? |
| glibc MALLOC _CHECK_ | GNU C | Dynamic using malloc-internal checks | Runtime erroneous free operations double free | Runtime accesses outside of heap-allocated variables statically-detectable errors in un-executed code | Open Source Windows, Linux |
| PC-Lint | ANSI, MS, GNU C | Static using heuristic flow analysis | Some pointer access errors; many false positives | Runtime access errors | Commercial Windows, Linux |
| Blast | ANSI(?) C | Static | All CCured-detectable problems; some specified temporal safety properties | Any unspecified temporal property | Research Linux? |
| Cyclone | Cyclone: Type-safe variant of C | Dynamic | Null dereference; uninitialized pointer dereferences; dangling pointers | Statically-detectable errors in un-executed code | Research Linux? |
| Prefix/ Prefast | Visual C | Static | Many memory safety errors; problems with false positives | Runtime access errors not detectable by static analysis | Commercial Windows |
| Coverity Prevent | C, C++, Java | Static | Many memory safety errors; problems with false positives | Runtime access errors not detectable by static analysis | Commercial Windows, Linux |
| Fortify | C, C++, Java | Static | Many memory safety errors; problems with false positives | Runtime access errors not detectable by static analysis | Commercial Windows, Linux |
We believe we have characterized these other tools fairly. If you think a tool is missing, or a description is inaccurate, or could be improved, please send a note to [email protected]
