|
|
| 第1行： |
第1行： |
| − | == Article purpose ==
| |
| − | This article provides useful information to start using Linux<sup>®</sup> tracing, monitoring and debugging environments.
| |
| | | | |
| − | Two entry points are proposed in this article:
| |
| − | *[[#Linux tracing, monitoring and debugging tools|Linux tracing, monitoring and debugging tools]], which gives an overview of some Linux<sup>®</sup> tools including usage and application domain. {{highlight|This chapter is useful when you already know the domain or the interface to search for}}.
| |
| − | *[[#Trace and debug overview per Linux software frameworks|Trace and debug overview per Linux software frameworks]], which points to articles explaining how to get trace and debug information about the Linux<sup>®</sup> software frameworks that are relevant for the STM32MPU Embedded Software. {{highlight|This chapter is useful when you know the Linux<sup>®</sup> framework to search for}}.
| |
| − |
| |
| − | == Linux trace architecture overview ==
| |
| − |
| |
| − | The Linux<sup>®</sup> trace architecture can be organized into four levels as shown in the figure below (''inspired by Brendan Gregg presentation''<ref>http://www.brendangregg.com/linuxperf.html</ref>):
| |
| − | [[File:Linux tracing architecture.png|center|link=|Linux kernel tracing architecture]]
| |
| − |
| |
| − | === Back-end instrumentation ===
| |
| − | The back-end instrumentation provides tracing sources built in the Linux<sup>®</sup> kernel. They are split into three categories:
| |
| − | *'''tracepoints''': kernel static tracing, statically placed at logical places in the kernel. It provides key event details as a "format" string.
| |
| − | *'''kprobes''': kernel dynamic tracing. It allows to trace function calls, returns and line numbers.
| |
| − | *'''uprobes''': dynamic user-level tracing.
| |
| − |
| |
| − | === Tracing framework ===
| |
| − | Also named tracers, they use tracing sources.
| |
| − |
| |
| − | Tracing frameworks include kernel in-tree tracers such as ftrace and perf_events, and out-of-tree tracers such as SystemTap and sysdig.
| |
| − |
| |
| − | === Front-end tools ===
| |
| − | Front-end tools come on top of tracers and help to configure them. For example:
| |
| − | * trace-cmd or LTTng for ftrace
| |
| − | * perf or perf-Tools for perf_events
| |
| − |
| |
| − | === Add-on tools and viewer ===
| |
| − | Add-on tools are also on top of tracers. However, they are not embedded inside the Linux<sup>®</sup> kernel.
| |
| − |
| |
| − | Viewer tools propose Visual interpretation of trace data. For example:
| |
| − | * '''[[Trace-cmd and kernelshark trace viewer#Visualizing trace using kernelshark|kernelshark]]''' for [[Ftrace|ftrace]]/[[Trace-cmd and kernelshark trace viewer|trace-cmd]]
| |
| − | * '''[[LTTng#Import LTTng traces in a Trace Compass trace viewer|Trace Compass]]'''<ref>http://www.tracecompass.org</ref> for [[LTTng|LTTng]] (and more)
| |
| − | * '''[[Perf#Visualizing trace using Flame Graphs|Flame Graph]]'''<ref>http://www.brendangregg.com/flamegraphs.html</ref> for [[Perf|perf]]
| |
| − |
| |
| − | == Linux tracing, monitoring and debugging tools ==
| |
| − | Linux<sup>®</sup> provides many tools that are either dedicated to one function or multifunction (generic).
| |
| − |
| |
| − | They cover both Linux<sup>®</sup> kernel and Linux<sup>®</sup> user space.
| |
| − |
| |
| − | === Domain mapping ===
| |
| − | The following mapping, done by Brendan Gregg <ref>http://www.brendangregg.com/linuxperf.html </ref>, shows the different existing tools associated to the different Linux<sup>®</sup> frameworks.
| |
| − | {{ImageMap|
| |
| − | Image:linux_tracing_observability_tools.png {{!}} frame {{!}} center {{!}} Note: The above image has been created by '''Brendan Gregg''' (Netflix) and can be found on his [http://www.brendangregg.com/linuxperf.html official web site].
| |
| − | rect 12 374 100 392 [[Blktrace | blktrace]]
| |
| − | rect 12 334 100 354 [[Sysstat tool suite| iostat]]
| |
| − | rect 36 166 104 184 [[Perf | perf]]
| |
| − | rect 36 184 104 202 [[Ftrace | ftrace]]
| |
| − | rect 36 205 104 222 [[Systemtap | stap]]
| |
| − | rect 36 280 104 300 [[LTTng | LTTng]]
| |
| − | rect 50 120 130 140 [[Sysstat tool suite| pidstat]]
| |
| − | rect 72 80 140 100 [[Strace | strace]]
| |
| − | rect 148 80 220 100 [[Ltrace | ltrace]]
| |
| − | rect 180 332 232 354 [[Perf | perf]]
| |
| − | rect 380 80 465 100 [[Netstat | netstat]]
| |
| − | rect 380 475 460 495 [[Ethtool | ethtool]]
| |
| − | rect 472 80 540 100 [[Sysdig | sysdig]]
| |
| − | rect 528 109 578 129 [[Perf | perf]]
| |
| − | rect 528 256 605 276 [[Sysstat tool suite| pidstat]]
| |
| − | rect 532 234 574 254 [[Top Linux command line | top]]
| |
| − | rect 685 256 740 276 [[Perf | perf]]
| |
| − | rect 545 160 615 180 [[Sysstat tool suite| mpstat]]
| |
| − | rect 500 358 580 380 [[Tcpdump | tcpdump]]
| |
| − | rect 650 418 730 438 [[Netstat | netstat]]
| |
| − | rect 650 438 730 458 [[IP Linux command line | ip]]
| |
| − | rect 660 97 722 117 [[Sysstat tool suite| sar]]
| |
| − | rect 660 137 722 157 [[Pseudo filesystem | /proc]]
| |
| − | rect 150 232 280 252 [[OpenSTLinux filesystem | Filesytem]]
| |
| − | }}
| |
| − |
| |
| − | {{InternalInfo|To check if required to address, vmstat,free, dtrace and ebpf}}
| |
| − |
| |
| − | === Tool overview ===
| |
| − | {{:Trace_and_debug_tools_assignment_table_template}}
| |
| − | {{:Blktrace}}
| |
| − | |-
| |
| − | {{:Core dump}}
| |
| − | |-
| |
| − | {{:Ethtool}}
| |
| − | |-
| |
| − | {{:Ftrace}}
| |
| − | |-
| |
| − | {{:GDB}}
| |
| − | |-
| |
| − | {{:Ifconfig}}
| |
| − | |-
| |
| − | {{:IP Linux command line | IP}}
| |
| − | |-
| |
| − | {{:Kmemleak}}
| |
| − | |-
| |
| − | {{:Trace-cmd and kernelshark trace viewer}}
| |
| − | |-
| |
| − | {{:Ltrace}}
| |
| − | |-
| |
| − | {{:LTTng}}
| |
| − | |-
| |
| − | {{:Netdata}}
| |
| − | |-
| |
| − | {{:Netstat}}
| |
| − | |-
| |
| − | {{:Perf}}
| |
| − | |-
| |
| − | {{:Strace}}
| |
| − | |-
| |
| − | {{:Sysprof}}
| |
| − | |-
| |
| − | {{:Sysstat tool suite}}
| |
| − | |-
| |
| − | {{:Tcpdump}}
| |
| − | |-
| |
| − | {{:Top Linux command line | Top}}
| |
| − | |-
| |
| − | {{:Valgrind}}
| |
| − | |}
| |
| − |
| |
| − | == Trace and debug overview per Linux software frameworks ==
| |
| − | The picture below allows accessing to different Linux software frameworks which provide specific trace and debug information in their {{highlight|"''How to trace and debug the framework''"}} dedicated chapter.
| |
| − |
| |
| − | {{:STM32MP15 Linux kernel overview}}
| |
| − |
| |
| − | == Tips ==
| |
| − | [[How to find Linux kernel driver associated to a device]].
| |
| − |
| |
| − | [[How to use the kernel dynamic debug]].
| |
| − |
| |
| − | == Documentation and web articles ==
| |
| − |
| |
| − | A lot of articles on the web mention Linux<sup>®</sup> kernel tracing and profiling. The following links provide a good introduction to these topics:
| |
| − | * [http://www.brendangregg.com/Slides/LISA2014_LinuxPerfAnalysisNewTools.pdf Linux Performance Analysis - New Tools and Old Secrets]: description of the Linux<sup>®</sup> technology and of the different tools available.
| |
| − | * [https://wiki.yoctoproject.org/wiki/Tracing_and_Profiling Yocto project: Tracing and profiling]: How to enable tracing and profiling tools using Yocto
| |
| − | * [http://www.brendangregg.com/linuxperf.html#Tools Brendan Gregg Linux performance page]
| |
| − |
| |
| − | {{Info| More general Linux performance information are available in [http://fr.slideshare.net/brendangregg/linux-performance-tools-2014 nice slideshare presentation (Brendan Gregg)], on the [http://www.brendangregg.com/linuxperf.html Brendan Gregg official web site] or in LinuxCon2014 [http://www.linux.com/news/enterprise/systems-management/785426-a-template-for-monitoring-linux-performance-tools-like-from-netflix article].}}
| |
| − |
| |
| − | Reference list:
| |
| − | <references />
| |
