Archive for May 6, 2009

10 Code Coverage Tools for C & C++

Code coverage is a measure used in software testing that describes the degree to which the source code of a program has been tested. It a form of white box testing as it is a form of testing that inspects the code directly.

Software_testing

Image Source : http://www.zeeteenet.com/images/sw_testing.jpg

Recommended Reading:

To measure how well the software is tested by a test suite, one or more coverage criteria are used. There are a number of coverage criteria and here are the important ones:

Function coverage – that says if each function in the program been executed or not
Statement coverage – that says if each line of the source code been executed or not
Decision coverage (also known as Branch coverage) – that says if each control structure such as an if statement has been evaluated both to true and false
Condition coverage – says if each boolean sub-expression has been evaluated for both true and false
Path coverage – that says if every possible route through a given part of the code been executed or not
Entry/exit coverage – that says if every possible call and return of the function been executed or not

In this article, we cover the popular Code Coverage tools to measure Code Coverage.

1. Testwell CTC++

CTC++ is a powerful instrumentation-based test coverage and dynamic analysis tool for C and C++ code. As a coverage tool, CTC++ shows the coverage all the way to the Modified Condition/Decision Coverage (MC/DC) level. As a dynamic analysis tool, CTC++ shows the execution counters in the code, i.e. more than plain boolean coverage information. You can also use CTC++ to measure function execution costs (normally time) and to enable function entry/exit tracing at test time

2. CoverageMeter

CoverageMeter is a complete code coverage tool chain for C/C++ programs available under Linux or Microsoft Windows. It analyzes the performance of a software validation and permits to measure the performance and optimizes the testing process of a C or C++ applications by:

  • Finding untested code sections.
  • Reducing the amount of tests by finding redundant tests - With CoverageMeter it is possible to find which portion of the source code is covered only by one execution of a test, and to detect if a new test does not cover more source code line than existing tests.
  • Finding dead code trough displaying the code parts which are never executed.
  • Specially useful for manual testing - CoverageMeter is able to calculate the optimal execution order of tests which maximize the overall coverage after each run.
  • Also, CoverageMeter is able to perform its analysis on a difference of two applications.
  • This permits to find which tests are impacted by a source code modification and permits to measure the test quality of a patch or a hot fix.

It is composed of 4 tools:

With CoverageScanner you can analyzes, instrument and generate the C/C++ application. CoverageBrowser displays and manages the results of the coverage analysis. Execution Agent is a small free tool which is dedicated to manual tests and specially for black box interactive tests. It also has an optional Microsoft Visual Studio 2005 & 2008 Add-In which permits to generate code coverage configurations from every C/C++ projects created by Microsoft® Visual Studio.

3. BullseyeCoverage

BullseyeCoverage is a code coverage analyzer for C++ and C that tells you how much of your source code was tested. You can use this information to quickly focus your testing effort and pinpoint areas that need to be reviewed. Here are some of the features of BullseyeCoverage:

  • Function coverage gives you a quick overview and condition/decision coverage gives you high precision
  • Works with everything you can write in C++ and C, including system-level and kernel mode
  • Short learning curve
  • Integrated with Microsoft Visual Studio
  • Include or exclude any portion of your project code
  • Run-time source code included, for custom environments

4. GCT

GCT is a C Code Coverage Tool. GCT was the third coverage tool by Brian Marick. It instruments C code in a source-to-source translation, then passes the instrumented code to a compiler. It is suitable for measuring the coverage of embedded systems as it was first used on a Unix kernel. In addition to branch and multiple-condition coverage, it also has boundary-condition and loop coverage. For more details on this tool check the tutorial at testingcraft.com

5. CppUnit

CppUnit is a unit testing framework module for C++, described as a C++ port of JUnit. The library is released under the GNU Lesser General Public License

6. Dynamic Code Coverage

Dynamic Code Coverage identifies untested portions of your software code, so your team can focus attention on those undiscovered areas, and improve your software quality immediately. Dynamic Code Coverage builds a detailed coverage analysis, covering every function, line, decision, and branch. Coverage files from subsequent multiple runs get added in. You can look at the information you gather in a variety of ways. And you can annotate each source file with function, line, decision, and branch information.

7. TCAT C/C++

TCAT C/C++ for Windows measures the Effectiveness of Tests, Test Suites. It identifies untested code and improves test efficiency.

It provides a combined branch and call-pair coverage for C-C++. It provides annotatable calltree displays and digraph displays with access to source.

8. COVTOOL

COVTOOL is an open source test coverage analyzer for C++ programs. It lets you dynamically instrument your source code as you compile. An instrumented program keeps track of the lines of code that were executed during its run and produces a log of the same upon program termination. Multiple program runs will produce multiple logs.
You can then use the many log files generated during a suite of tests to analyze test coverage percentages. Most importantly, you can use the coverage information to annotate your source. Thus you can find which lines in which source files did not get executed during your entire suite of tests. Code that has been tested may still be buggy, but you should have almost no confidence in code that is never tested.

9. gocv

gcov is a tool you can use in conjunction with GCC to test code coverage in your programs. You can use it with along with GCC to analyze your programs to help create more efficient, faster running code and to discover untested parts of your program. You can use gcov tool to help to understand where your optimization efforts will best affect your code. You can also use gcov along with the other profiling tool, gprof, to assess which parts of your code use the greatest amount of computing time.

10. xCover

xCover is a Code Coverage library for C and C++, written in C. xCover itself is a platform-independent library. It has been tested and used on both UNIX and Windows - in principle, it should work on any operating system for which a compatible C/C++ compiler exists.

5 Free and Open Source Desktop Project Management Tools

Project Management Software applications help Project Manager to track Schedule, Performance, Cost and Resource utilization. Microsoft Project is the industry leader and a proprietary software that costs significant amount of money. If money is a constraint, then don’t think again because there are equally good Project management software applications that cost nothing!

Project_Management_Page-23

Image Source: TrueSolutions (Project Management Services)

1. GanttProject (Alternative to Microsoft Project)

We had already featured GanttProject last month, which is a cross-platform desktop tool for project scheduling and management. It runs on Windows, Linux and Mac OS X, it is free and its code is open source.

Ganttproject-house-building-sample

2. KPlato

KPlato (part of KOffice on K Desktop Environment-KDE) is a project management application for Linux. It supports resource allocation, work breakdown structure, critical paths, Gantt chart and many more. A complete project management application for Linux

gantt_small_2006_01

3. OpenProj (Alternative to Microsoft Project)

OpenProj is a free, open source project management solution. OpenProj is a complete desktop replacement of Microsoft Project and other commercial project solutions. OpenProj shares the industry’s most advanced scheduling engine with Project-ON-Demand and provides Gantt Charts, Network Diagrams (PERT Charts), WBS and RBS charts, Earned Value costing and more. It is very popular and crossed more than a million downloads in the few months since its launch.

OpenProj_big

4. Open Workbench (Alternative to Microsoft Project)

Open Workbench is an open source desktop application that provides robust project scheduling and management functionality. Already the scheduling standard for more than 100,000 project managers worldwide, Open Workbench is a free and powerful alternative to Microsoft Project. It runs on Windows platform with Sun JRE 1.3.1 or higher.

OpenWorkbench

5. TaskJuggler

TaskJuggler is a modern and powerful, Open Source project management tool. Its new approach to project planning and tracking is more flexible and superior to the commonly used Gantt chart editing tools. It has already been successfully used in many projects and scales easily to projects with hundreds of resources and thousands of tasks. TaskJuggler runs on Linux/Unix. However, there are ports for Windows and Mac OS exist. For more info, please click here.

Task Juggler

12 Source Code Profilers for C & C++

Code Profilers are very distinct from traditional debuggers. They are able to catch the trivial and non fatal coding errors which are often hard for humans to catch. These trivial bugs later claim weeks of debugging and extra cost to fix. Code profilers will save you hours of debugging time. These tools can automatically detect many memory management and threading, etc bugs. These tools will give you confidence that your programs are free of many common bugs, some of which would take hours to find manually, or never be found at all. You can find and eliminate bugs before they become a problem.

As usual, there are a lot of code profilers available for C/C++. Here is a list of very prominent such players.

1. Insure++ (For Windows, Linux, UNIX like)

image

Insure++ is a runtime memory analysis and error detection tool for C and C++ that automatically identifies a variety of difficult-to-track programming and memory-access errors, along with potential defects and inefficiencies in memory usage. Errors such as memory corruption, memory leaks, access outside of array bounds, invalid pointers, and the like often go undetected during normal testing, only to result in application crashes in the field. Insure++ will help you find and eliminate such defects in your applications to ensure the integrity of their memory usage.

During testing, Insure++ checks all types of memory references, including those to static (global), stack, and shared memory — both in user’s code and in third party libraries.

Errors that Insure++ detects include: Corrupted heap and stack memory, Use of uninitialized variables and objects, Array and string bounds errors on heap and stack, etc.

2. Rational Purify (For Windows and Linux/UNIX)

image

IBM Rational Purify is a runtime analysis solution designed to help developers write more reliable code such as C++ and C. Reliability is ensured via two crucial functions: memory corruption detection and memory leak detection. Rational Purify packages support for these two runtime analysis capabilities in a single product with a common insta ll and licensing system.

3. Valgrind (For UNIX like)

Valgrind is an award-winning instrumentation framework for building dynamic analysis tools. There are Valgrind tools that can automatically detect many memory management and threading bugs, and profile your programs in detail. You can also use Valgrind to build new tools.

The Valgrind distribution currently includes six production-quality tools: a memory error detector, two thread error detectors, a cache and branch-prediction profiler, a call-graph generating cache profiler, and a heap profiler. It also includes one experimental tool, which detects out of bounds reads and writes of stack, global and heap arrays. It runs on the following platforms: X86/Linux, AMD64/Linux, PPC32/Linux, PPC64/Linux.

Valgrind works with programs written in any language. Because Valgrind works directly with program binaries, it works with programs written in any programming language, be they compiled, just-in-time compiled, or interpreted. The Valgrind tools are largely aimed at programs written in C and C++, because programs written in these languages tend to have the most bugs! But it can, for example, be used to debug and profile systems written in a mixture of languages. Valgrind has been used on programs written partly or entirely in C, C++, Java, Perl, Python, assembly code, Fortran, Ada, and many others.

Valgrind is Free and Open Source Software. Read here for more on Valgrind.

4. Zoom (For Linux)

image

image

Zoom is a graphical, low-overhead, sampling profiler for Linux from RotateRight which diagnoses performance problems related to CPU and memory-intensive code. Zoom supports Call Tree Visualization, Code Visualization etc. It is really easy to correlate assembly and actual source, bottlenecks in the code etc. Zoom supports DWARF (debugging data format).

5. Code Analyst (For Windows and Linux)

image

AMD CodeAnalyst Performance Analyzer is a suite of powerful tools that analyze software performance on AMD microprocessors and help developers optimize application performance. It uses a profiling technique to identify performance areas and issues within the system. Using this suite of tools, developers can get visibility into overall system performance, and can navigate into different modules and functions to locate hotspots within the target application.

CodeAnalyst allows you to profile applications using various performance events available on different AMD processors. You can specify performance events to target various functional units of the processor, or profiling granularity to adjust the accuracy of the result.

CodeAnalyst is designed to profile the performance of binary modules, including user mode application modules and kernel mode driver modules. Timer-Based Profiling and Event-Based Profiling collect data from multiple processors in a multi-processor system. It can support any language as it works with the binary.

6. Sleepy (For Windows)

image

‘Sleepy’ is a non-invasive profiler for the Windows platform that allows statistical profiling of C++ applications through sampling the instruction pointer register. Sleepy presents results in a GUI built with wxWindows.

The Sleepy profiler uses a technique where the profiler runs in a different thread from the target program. Every 1ms or so, the profiler thread suspends the target thread, and pulls out the current instruction pointer register value from the thread context. These mem addresses are resolved into procedure names and line numbers using debug information. This allows line-level resolution, without making any changes to the target program. The only requirement is that the target program is compiled with (MS) debug information

Sleepy uses the MS Debug Help (dbghelp) library to read debug information, and is therefore limited to the debug info formats that dbghelp can understand. As far as I know, debug information produced by non-MS compilers such as GCC will not be understood, and therefore such code cannot be profiled.

An improved version Very Sleepy is also available.

7 Intel VTune (For Windows and Linux)

image

VTune Performance Analyzer is Intel Corporation‘s tool for call graph or analyzing a set of tuning events. It works with C/C++/Fortan/.NET/Java and other applications on Linux or Windows, but only when running on selected Intel hardware. he GUI assists in various kinds of code profiling including time based profile, Event based profile etc.

VTune Performance Analyzer evaluates applications on all sizes of systems based on Intel® processors, from embedded systems through supercomputers, to help you improve application performance. VTune Performance Analyzer makes application performance tuning easier and is indispensable for making your software run its fastest on the latest single and multi-core systems.

8. AQTime (For Windows)

image

AQtime is a complete, top-to-bottom performance profiler and memory usage debugger for all major Windows development tools (Delphi, Visual Basic, Visual C++, and C++Builder), .NET compilers (including both Microsoft and non-Microsoft) and for VBScript and JScript routines. It includes several profilers and productivity tools that provide you with an unrivaled level of information on the state of your 32- and 64-bit software projects - from inception to delivery!

9. BullseyeCoverage (For Windows, Linux, Solaris and many more)

image

BullseyeCoverage is a code coverage analyzer for C++ and C that tells you how much of your source code was tested. You can use this information to quickly focus your testing effort and pinpoint areas that need to be reviewed. Code coverage analysis is useful during unit testing, integration testing, and final release. BullseyeCoverage enables you to create more reliable code and save time.

Function coverage gives you a quick overview and condition/decision coverage gives you high precision Works with everything you can write in C++ and C, including system-level and kernel mode Short learning curve, Integrated with Microsoft Visual Studio, Include or exclude any portion of your project code, Run-time source code included, for custom environments, Simple licensing, Merge results from distributed testing etc.

10. C++ Profiler Tool (For Windows)

image

The C++ Profiler tool enables the collection and display of execution profile data on C++ software source code bases of arbitrary size.

11. Shiny Profiler (For Windows, UNIX Like, Source code is portable)

Shiny is a low overhead, well documented and lightning fast C++ profiler. Easy to use on existing projects, no extensive surgery, just have fun with it. Nicely formatted output provides call tree (call graph), flat profile and is simple to custom display

12. Deleaker (For Windows)

image

Deleaker is a run-time error detection and debugging tool for Visual C++ developers. Deleaker is a really useful add-in for Visual Studio that helps you to analyze programming errors, many of which are unique to Visual C++. With Deleaker you can detect and localize resource leaks such as memory, GDI and USER objects, handles.

Resource leaks. Even a robust Windows-based application is not safe from them. Of all bugs they are the most difficult to find especially when they appear in graphics device interface (GDI) objects and menus. And as we all know, the earlier you catch bugs, the less expensive they are. Plenty of tools exist to help track down memory leaks. But there are very few good tools to help track GDI resource leaks that can ruin performance on any Windows operating system. Deleaker is one of them and the fact that it almost doesn’t slow down your application makes it your natural choice.

Related Posts with Thumbnails