About gprof2dot

This is a Python script to convert the output from many profilers into a dot graph.

It can:

• read output from:
  • Linux perf
  • Valgrind's callgrind tool
  • OProfile
  • Sysprof
  • Xperf
  • VTune
  • Very Sleepy
  • Python profilers
  • Java's HPROF
  • prof, gprof
  • DTrace
  • stackcollapse from FlameGraph
• prune nodes and edges below a certain threshold;
• use an heuristic to propagate time inside mutually recursive functions;
• use color efficiently to draw attention to hot-spots;
• work on any platform where Python and Graphviz is available, i.e, virtually anywhere;
• compare two graphs with almost identical structures for the analysis of performance metrics such as time or function calls.

If you want an interactive viewer for the graphs generated by gprof2dot, check xdot.py.

Status

gprof2dot currently fulfills my needs, and I have little or no time for its maintenance. So I'm afraid that any requested features are unlikely to be implemented, and I might be slow processing issue reports or pull requests.

Example

This is the result from the example data in the Linux Gazette article with the default settings:

Requirements

• Python: known to work with version >=3.8; it will most likely not work with earlier releases.
• Graphviz: tested with version 2.26.3, but should work fine with other versions.

Windows users

• Download and install Python for Windows
• Download and install Graphviz for Windows

Linux users

On Debian/Ubuntu run:

apt-get install python3 graphviz

On RedHat/Fedora run

yum install python3 graphviz

Download

• PyPI

  pip install gprof2dot
  

• Standalone script

• Git repository

Documentation

Usage

Usage: 
    gprof2dot.py [options] [file] ...

Options:
  -h, --help            show this help message and exit
  -o FILE, --output=FILE
                        output filename [stdout]
  -n PERCENTAGE, --node-thres=PERCENTAGE
                        eliminate nodes below this threshold [default: 0.5]
  -e PERCENTAGE, --edge-thres=PERCENTAGE
                        eliminate edges below this threshold [default: 0.1]
  -f FORMAT, --format=FORMAT
                        profile format: axe, callgrind, collapse, dtrace,
                        hprof, json, oprofile, perf, prof, pstats, sleepy,
                        sysprof or xperf [default: prof]
  --total=TOTALMETHOD   preferred method of calculating total time: callratios
                        or callstacks (currently affects only perf format)
                        [default: callratios]
  -c THEME, --colormap=THEME
                        color map: bw, color, gray, pink or print [default:
                        color]
  -s, --strip           strip function parameters, template parameters, and
                        const modifiers from demangled C++ function names
  --color-nodes-by-selftime
                        color nodes by self time, rather than by total time
                        (sum of self and descendants)
  -w, --wrap            wrap function names
  --show-samples        show function samples
  --node-label=MEASURE  measurements to on show the node (can be specified
                        multiple times): self-time, self-time-percentage,
                        total-time or total-time-percentage [default: total-
                        time-percentage, self-time-percentage]
  --list-functions=LIST_FUNCTIONS
                        list functions available for selection in -z or -l,
                        requires selector argument ( use '+' to select all).
                        Recall that the selector argument is used with
                        Unix/Bash globbing/pattern matching, and that entries
                        are formatted '<pkg>:<linenum>:<function>'. When
                        argument starts with '%', a dump of all available
                        information is performed for selected entries,  after
                        removal of leading '%'.
  -z ROOT, --root=ROOT  prune call graph to show only descendants of specified
                        root function
  -l LEAF, --leaf=LEAF  prune call graph to show only ancestors of specified
                        leaf function
  --depth=DEPTH         prune call graph to show only descendants or ancestors
                        until specified depth
  --skew=THEME_SKEW     skew the colorization curve.  Values < 1.0 give more
                        variety to lower percentages.  Values > 1.0 give less
                        variety to lower percentages
  -p FILTER_PATHS, --path=FILTER_PATHS
                       Filter all modules not in a specified path
  --compare             Compare two graphs with almost identical structure. With this
                        option two files should be provided.gprof2dot.py
                        [options] --compare [file1] [file2] ...
  --compare-tolerance=TOLERANCE
                        Tolerance threshold for node difference
                        (default=0.001%).If the difference is below this value
                        the nodes are considered identical.
  --compare-only-slower
                        Display comparison only for function which are slower
                        in second graph.
  --compare-only-faster
                        Display comparison only for function which are faster
                        in second graph.
  --compare-color-by-difference
                        Color nodes based on the value of the difference.
                        Nodes with the largest differences represent the hot
                        spots.

Examples

Linux perf

perf record -g -- /path/to/your/executable
perf script | c++filt | gprof2dot.py -f perf | dot -Tpng -o output.png

oprofile

opcontrol --callgraph=16
opcontrol --start
/path/to/your/executable arg1 arg2
opcontrol --stop
opcontrol --dump
opreport -cgf | gprof2dot.py -f oprofile | dot -Tpng -o output.png

xperf

If you're not familiar with xperf then read this excellent article first. Then do:

• Start xperf as

  xperf -on Latency -stackwalk profile
  

• Run your application.

• Save the data. ` xperf -d output.etl

• Start the visualizer:

  xperf output.etl
  

• In Trace menu, select Load Symbols. Configure Symbol Paths if necessary.

• Select an area of interest on the CPU sampling graph, right-click, and select Summary Table.

• In the Columns menu, make sure the Stack column is enabled and visible.

• Right click on a row, choose Export Full Table, and save to output.csv.

• Then invoke gprof2dot as

  gprof2dot.py -f xperf output.csv | dot -Tpng -o output.png
  

VTune Amplifier XE

• Collect profile data as (also can be done from GUI):

  amplxe-cl -collect hotspots -result-dir output -- your-app
  

• Visualize profile data as:

  amplxe-cl -report gprof-cc -result-dir output -format text -report-output output.txt
  gprof2dot.py -f axe output.txt | dot -Tpng -o output.png
  

See also Kirill Rogozhin's blog post.

gprof

/path/to/your/executable arg1 arg2
gprof path/to/your/executable | gprof2dot.py | dot -Tpng -o output.png

python profile

python -m profile -o output.pstats path/to/your/script arg1 arg2
gprof2dot.py -f pstats output.pstats | dot -Tpng -o output.png

python cProfile (formerly known as lsprof)

python -m cProfile -o output.pstats path/to/your/script arg1 arg2
gprof2dot.py -f pstats output.pstats | dot -Tpng -o output.png

Java HPROF

java -agentlib:hprof=cpu=samples ...
gprof2dot.py -f hprof java.hprof.txt | dot -Tpng -o output.png

See Russell Power's blog post for details.

DTrace

dtrace -x ustackframes=100 -n 'profile-97 /pid == 12345/ { @[ustack()] = count(); } tick-60s { exit(0); }' -o out.user_stacks
gprof2dot.py -f dtrace out.user_stacks | dot -Tpng -o output.png

# Notice: sometimes, the dtrace outputs format may be latin-1, and gprof2dot will fail to parse it.
# To solve this problem, you should use iconv to convert to UTF-8 explicitly.
# TODO: add an encoding flag to tell gprof2dot how to decode the profile file.
iconv -f ISO-8859-1 -t UTF-8 out.user_stacks | gprof2dot.py -f dtrace

stackcollapse

Brendan Gregg's FlameGraph tool takes as its input a text file containing one line per sample. This format can be generated from various other inputs using the stackcollapse scripts in the FlameGraph repository. It can also be generated by tools such as py-spy.

Example usage:

• Perf

  perf record -g -- /path/to/your/executable
  perf script | FlameGraph/stackcollapse-perf.pl > out.collapse
  gprof2dot.py -f collapse out.collapse | dot -Tpng -o output.png
  

• Py-spy

  py-spy record -p <pidfile> -f raw -o out.collapse
  gprof2dot.py -f collapse out.collapse | dot -Tpng -o output.png
  

Compare Example

This image illustrates an example usage of the --compare and --compare-color-by-difference options.

Arrow pointing to the right indicate node where the function performed faster in the profile provided as the second one (second profile), while arrow pointing to the left indicate node where the function was faster in the profile provided as the first one (first profile).

Node

+-----------------------------+
|        function name          \
| total time %  -/+ total_diff   \
| ( self time % ) -/+ self_diff  /
| total calls1 / total calls2   /
+-----------------------------+

Where - total time % and self time % come from the first profile - diff is calculated as the absolute value of time in the first profile - time in the second profile.

Note The compare option has been tested for pstats, axe and callgrind profiles.

Output

A node in the output graph represents a function and has the following layout:

+------------------------------+
|        function name         |
| total time % ( self time % ) |
|         total calls          |
+------------------------------+

where:

• total time % is the percentage of the running time spent in this function and all its children;
• self time % is the percentage of the running time spent in this function alone;
• total calls is the total number of times this function was called (including recursive calls).

An edge represents the calls between two functions and has the following layout:

           total time %
              calls
parent --------------------> children

Where:

• total time % is the percentage of the running time transferred from the children to this parent (if available);
• calls is the number of calls the parent function called the children.

Note that in recursive cycles, the total time % in the node is the same for the whole functions in the cycle, and there is no total time % figure in the edges inside the cycle, since such figure would make no sense.

The color of the nodes and edges varies according to the total time % value. In the default temperature-like color-map, functions where most time is spent (hot-spots) are marked as saturated red, and functions where little time is spent are marked as dark blue. Note that functions where negligible or no time is spent do not appear in the graph by default.

Listing functions

The flag --list-functions permits listing the function entries found in the gprof input. This is intended as a tool to prepare for utilisations with the --leaf (-l) or --root (-z) flags.

~~~ prof2dot.py -f pstats /tmp/myLog.profile --list-functions "test_segments::"

test_segments:5:, test_segments:206:TestSegments, test_segments:46:

• The selector argument is used with Unix/Bash globbing/pattern matching, in the same fashion as performed by the -l and -z flags.

• Entries are formatted '\<pkg>:\<linen