In my previous post, I shared how to profile a single function using line_profiler. In this post, I want to share how to profile Python script/module and visualize the profile stat to pinpoint the slow part of the code.
Generate the raw profiling stat#
In Python, we can use cProfile to generate the raw profile result:
python -m cProfile -o {script_name}.prof {your_script}.py [script_argument]
The -o options tells cProfile to generate a profile stat file, which is in binary format and not human readable.
In order to interpret the profiling stat easily, we need to install some tools to visualize the result.
static profile graph generation#
flameprof#
flameprof can generate a flame graph in SVG format. It is not actively maintained anymore.
pip install flameprof
flameprof input.prof > output.svg
gprof2dot#
gprof2dot can generate a directed call graph for your script with their running time.
pip install gprof2dot
gprof2dot -f pstats input.prof | dot -Tsvg -o output.svg
interactive profile graph generation#
snakeviz#
snakeviz can also generate flamegraph from cProfile stat. It also has great interactive UI to explore the profiling result.
pip install snakeviz
snakeviz input.prof
tuna#
Tuna is like snakeviz, but it claims to solve some of the limitation of snakeviz. The UI is less polished compared to snakeviz, though.
Py-spy#
py-spy can work with speedscope to generate flamegraph.
pip install py-spy
# on macOS, needs sudo to run py-spy
sudo py-spy record --format speedscope -r 500 -F -o output.speedscope python some_script.py
chown -R {your-user-name} output.speedscope
# use speedscope to open the file, https://www.speedscope.app/
py-spy can also profile the running program without interrupting it.
pyinstrument#
pyinstrument is another tool you can use to generate profiling graph. You can directly show the result in terminal:
pip install pyinstrument
pyinstrument your_script.py
You can also generate other output format:
pyinstrument -r html your_script.py
# or generate speedscope file, then inspect with speedscope
pyinstrument -r speedscope your_script.py