如何逐行分析python代码?

I've been using cProfile to profile my code, and it's been working great. I also use gprof2dot.py to visualize the results (makes it a little clearer).

我一直在用cProfile来剖析我的代码，而且效果很好。我也使用gprof2dot。py将结果可视化(使其更清晰)。

However, cProfile (and most other python profilers I've seen so far) seem to only profile at the function-call level. This causes confusion when certain functions are called from different places - I have no idea if call #1 or call #2 is taking up the majority of the time. This gets even worse when the function in question is 6 levels deep, called from 7 other places.

然而，cProfile(以及我到目前为止看到的大多数其他python分析器)似乎只在函数调用级别配置文件。当从不同的地方调用某些函数时，这会导致混淆——我不知道调用#1还是调用#2会占用大部分时间。当问题中的函数深度为6层时(从7个其他地方调用)，情况就更糟了。

So my question is: how do I get a line-by-line profiling? Instead of this:

我的问题是:如何逐行分析?而不是:

function #12, total time: 2.0s

I'd like to see something like this:

我想看看这样的东西:

function #12 (called from somefile.py:102) 0.5s
function #12 (called from main.py:12) 1.5s

cProfile does show how much of the total time "transfers" to the parent, but again this connection is lost when you have a bunch of layers and interconnected calls.

cProfile确实显示了对父节点的总时间“传输”的多少，但是当您有一堆层和相互连接的调用时，这种连接就会丢失。

Ideally, I'd love to have a GUI that would parse through the data, then show me my source file with a total time given to each line. Something like this:

理想情况下，我希望有一个可以解析数据的GUI，然后向我显示源文件，并给出每一行的总时间。是这样的:

main.py:

a = 1 # 0.0s
result = func(a) # 0.4s
c = 1000 # 0.0s
result = func(c) # 5.0s

Then I'd be able to click on the second "func(c)" call to see what's taking up time in that call, separate from the "func(a)" call.

然后，我可以点击第二个“func(c)”调用，看看在调用中占用了多少时间，与“func(a)”调用分离。

Does that make sense? Is there any profiling library that collects this type of info? Is there some awesome tool I've missed? Any feedback is appreciated. Thanks!!

这说得通吗?有收集这类信息的分析库吗?有什么工具我错过了吗?任何反馈都是感激。谢谢! !

3 个解决方案

#1

I believe that's what Robert Kern's line_profiler is intended for. From the link:

我相信这就是Robert Kern的line_profiler的目的。从链接:

File: pystone.py
Function: Proc2 at line 149
Total time: 0.606656 s

Line #      Hits         Time  Per Hit   % Time  Line Contents
==============================================================
   149                                           @profile
   150                                           def Proc2(IntParIO):
   151     50000        82003      1.6     13.5      IntLoc = IntParIO + 10
   152     50000        63162      1.3     10.4      while 1:
   153     50000        69065      1.4     11.4          if Char1Glob == 'A':
   154     50000        66354      1.3     10.9              IntLoc = IntLoc - 1
   155     50000        67263      1.3     11.1              IntParIO = IntLoc - IntGlob
   156     50000        65494      1.3     10.8              EnumLoc = Ident1
   157     50000        68001      1.4     11.2          if EnumLoc == Ident1:
   158     50000        63739      1.3     10.5              break
   159     50000        61575      1.2     10.1      return IntParIO

Hope that helps!

希望会有帮助!

#2

You could also use pprofile(pypi). If you want to profile the entire execution, it does not require source code modification. You can also profile a subset of a larger program in two ways:

您还可以使用pprofile(pypi)。如果您希望对整个执行进行概要分析，则不需要修改源代码。你也可以用两种方式描述一个大程序的子集:

toggle profiling when reaching a specific point in the code, such as:

当达到代码中的特定点时，切换分析，例如:

import pprofile
profiler = pprofile.Profile()
with profiler:
    some_code
# Process profile content: generate a cachegrind file and send it to user.

toggle profiling asynchronously from call stack (requires a way to trigger this code in considered application, for example a signal handler or an available worker thread) by using statistical profiling:

通过使用统计分析方法，从调用堆栈异步调用分析(需要一种方法来触发被考虑的应用程序中的代码，例如信号处理器或可用的工作线程):
```
import pprofile
profiler = pprofile.StatisticalProfile()
statistical_profiler_thread = pprofile.StatisticalThread(
    profiler=profiler,
)
with statistical_profiler_thread:
    sleep(n)
# Likewise, process profile content
```

Code annotation output format is much like line profiler:

代码注释输出格式很像行分析器:

$ pprofile --threads 0 demo/threads.py
Command line: ['demo/threads.py']
Total duration: 1.00573s
File: demo/threads.py
File duration: 1.00168s (99.60%)
Line #|      Hits|         Time| Time per hit|      %|Source code
------+----------+-------------+-------------+-------+-----------
     1|         2|  3.21865e-05|  1.60933e-05|  0.00%|import threading
     2|         1|  5.96046e-06|  5.96046e-06|  0.00%|import time
     3|         0|            0|            0|  0.00%|
     4|         2|   1.5974e-05|  7.98702e-06|  0.00%|def func():
     5|         1|      1.00111|      1.00111| 99.54%|  time.sleep(1)
     6|         0|            0|            0|  0.00%|
     7|         2|  2.00272e-05|  1.00136e-05|  0.00%|def func2():
     8|         1|  1.69277e-05|  1.69277e-05|  0.00%|  pass
     9|         0|            0|            0|  0.00%|
    10|         1|  1.81198e-05|  1.81198e-05|  0.00%|t1 = threading.Thread(target=func)
(call)|         1|  0.000610828|  0.000610828|  0.06%|# /usr/lib/python2.7/threading.py:436 __init__
    11|         1|  1.52588e-05|  1.52588e-05|  0.00%|t2 = threading.Thread(target=func)
(call)|         1|  0.000438929|  0.000438929|  0.04%|# /usr/lib/python2.7/threading.py:436 __init__
    12|         1|  4.79221e-05|  4.79221e-05|  0.00%|t1.start()
(call)|         1|  0.000843048|  0.000843048|  0.08%|# /usr/lib/python2.7/threading.py:485 start
    13|         1|  6.48499e-05|  6.48499e-05|  0.01%|t2.start()
(call)|         1|   0.00115609|   0.00115609|  0.11%|# /usr/lib/python2.7/threading.py:485 start
    14|         1|  0.000205994|  0.000205994|  0.02%|(func(), func2())
(call)|         1|      1.00112|      1.00112| 99.54%|# demo/threads.py:4 func
(call)|         1|  3.09944e-05|  3.09944e-05|  0.00%|# demo/threads.py:7 func2
    15|         1|  7.62939e-05|  7.62939e-05|  0.01%|t1.join()
(call)|         1|  0.000423908|  0.000423908|  0.04%|# /usr/lib/python2.7/threading.py:653 join
    16|         1|  5.26905e-05|  5.26905e-05|  0.01%|t2.join()
(call)|         1|  0.000320196|  0.000320196|  0.03%|# /usr/lib/python2.7/threading.py:653 join

Note that because pprofile does not rely on code modification it can profile top-level module statements, allowing to profile program startup time (how long it takes to import modules, initialise globals, ...).

注意，因为pprofile不依赖于代码修改，所以它可以配置*模块语句，允许配置程序启动时间(导入模块需要多长时间，初始化全局，…)。

It can generate cachegrind-formatted output, so you can use kcachegrind to browse large results easily.

它可以生成cachegrind格式的输出，因此可以使用kcachegrind轻松浏览大型结果。

Disclosure: I am pprofile author.

我是pprofile的作者。

#3

PyVmMonitor has a live-view which can help you there (you can connect to a running program and get statistics from it).

PyVmMonitor有一个实时视图，可以在那里帮助您(您可以连接到正在运行的程序并从中获取统计信息)。

See: http://www.pyvmmonitor.com/

参见:http://www.pyvmmonitor.com/

#1