“Linux tracing, monitoring and debugging”的版本间的差异

来自百问网嵌入式Linux wiki
(创建页面,内容为“== Article purpose == This article provides useful information to start using Linux<sup>®</sup> tracing, monitoring and debugging environments. Two entry points…”)
 
(清空页面)
标签清空
第1行: 第1行:
== Article purpose ==
 
This article provides useful information to start using Linux<sup>&reg;</sup> tracing, monitoring and debugging environments.
 
  
Two entry points are proposed in this article:
 
*[[#Linux tracing, monitoring and debugging tools|Linux tracing, monitoring and debugging tools]], which gives an overview of some Linux<sup>&reg;</sup> tools including usage and application domain. {{highlight|This chapter is useful when you already know the domain or the interface to search for}}.
 
*[[#Trace and debug overview per Linux software frameworks|Trace and debug overview per Linux software frameworks]], which points to articles explaining how to get trace and debug information about the Linux<sup>&reg;</sup> software frameworks that are relevant for the STM32MPU Embedded Software.  {{highlight|This chapter is useful when you know the Linux<sup>&reg;</sup> framework to search for}}.
 
 
== Linux trace architecture overview ==
 
 
The Linux<sup>&reg;</sup> trace architecture can be organized into four levels as shown in the figure below (''inspired by Brendan Gregg presentation''<ref>http://www.brendangregg.com/linuxperf.html</ref>):
 
[[File:Linux tracing architecture.png|center|link=|Linux kernel tracing architecture]]
 
 
=== Back-end instrumentation ===
 
The back-end instrumentation provides tracing sources built in the Linux<sup>&reg;</sup> kernel. They are split into three categories:
 
*'''tracepoints''': kernel static tracing, statically placed at logical places in the kernel. It provides key event details as  a "format" string.
 
*'''kprobes''': kernel dynamic tracing. It allows to trace function calls, returns and line numbers.
 
*'''uprobes''': dynamic user-level tracing.
 
 
=== Tracing framework ===
 
Also named tracers, they use tracing sources.
 
 
Tracing frameworks include kernel in-tree tracers such as ftrace and perf_events, and out-of-tree tracers such as SystemTap and sysdig.
 
 
=== Front-end tools ===
 
Front-end tools come on top of tracers and help to configure them. For example:
 
* trace-cmd or LTTng for ftrace
 
* perf or perf-Tools for perf_events
 
 
=== Add-on tools and viewer ===
 
Add-on tools are also on top of tracers. However, they are not embedded inside the Linux<sup>&reg;</sup> kernel.
 
 
Viewer tools propose Visual interpretation of trace data. For example:
 
* '''[[Trace-cmd and kernelshark trace viewer#Visualizing trace using kernelshark|kernelshark]]''' for [[Ftrace|ftrace]]/[[Trace-cmd and kernelshark trace viewer|trace-cmd]]
 
* '''[[LTTng#Import LTTng traces in a Trace Compass trace viewer|Trace Compass]]'''<ref>http://www.tracecompass.org</ref> for [[LTTng|LTTng]] (and more)
 
* '''[[Perf#Visualizing trace using Flame Graphs|Flame Graph]]'''<ref>http://www.brendangregg.com/flamegraphs.html</ref> for [[Perf|perf]]
 
 
== Linux tracing, monitoring and debugging tools ==
 
Linux<sup>&reg;</sup> provides many tools that are either dedicated to one function or multifunction (generic).
 
 
They cover both Linux<sup>&reg;</sup> kernel and Linux<sup>&reg;</sup> user space.
 
 
=== Domain mapping ===
 
The following mapping, done by Brendan Gregg <ref>http://www.brendangregg.com/linuxperf.html </ref>, shows the different existing tools associated to the different Linux<sup>&reg;</sup> frameworks.
 
{{ImageMap|
 
Image:linux_tracing_observability_tools.png {{!}} frame {{!}} center {{!}} Note: The above image has been created by '''Brendan Gregg''' (Netflix) and can be found on his [http://www.brendangregg.com/linuxperf.html official web site].
 
rect 12 374 100 392 [[Blktrace | blktrace]]
 
rect 12 334 100 354 [[Sysstat tool suite| iostat]]
 
rect 36 166 104 184 [[Perf | perf]]
 
rect 36 184 104 202 [[Ftrace | ftrace]]
 
rect 36 205 104 222 [[Systemtap | stap]]
 
rect 36 280 104 300 [[LTTng | LTTng]]
 
rect 50 120 130 140 [[Sysstat tool suite| pidstat]]
 
rect 72 80 140 100 [[Strace | strace]]
 
rect 148 80 220 100 [[Ltrace | ltrace]]
 
rect 180 332 232 354 [[Perf | perf]]
 
rect 380 80 465 100 [[Netstat | netstat]]
 
rect 380 475 460 495 [[Ethtool | ethtool]]
 
rect 472 80 540 100 [[Sysdig | sysdig]]
 
rect 528 109 578 129 [[Perf | perf]]
 
rect 528 256 605 276 [[Sysstat tool suite| pidstat]]
 
rect 532 234 574 254 [[Top Linux command line | top]]
 
rect 685 256 740 276 [[Perf | perf]]
 
rect 545 160 615 180 [[Sysstat tool suite| mpstat]]
 
rect 500 358 580 380 [[Tcpdump | tcpdump]]
 
rect 650 418 730 438 [[Netstat | netstat]]
 
rect 650 438 730 458 [[IP Linux command line | ip]]
 
rect 660 97 722 117 [[Sysstat tool suite| sar]]
 
rect 660 137 722 157 [[Pseudo filesystem | /proc]]
 
rect 150 232 280 252 [[OpenSTLinux filesystem | Filesytem]]
 
}}
 
 
{{InternalInfo|To check if required to address, vmstat,free, dtrace and ebpf}}
 
 
=== Tool overview ===
 
{{:Trace_and_debug_tools_assignment_table_template}}
 
{{:Blktrace}}
 
|-
 
{{:Core dump}}
 
|-
 
{{:Ethtool}}
 
|-
 
{{:Ftrace}}
 
|-
 
{{:GDB}}
 
|-
 
{{:Ifconfig}}
 
|-
 
{{:IP Linux command line | IP}}
 
|-
 
{{:Kmemleak}}
 
|-
 
{{:Trace-cmd and kernelshark trace viewer}}
 
|-
 
{{:Ltrace}}
 
|-
 
{{:LTTng}}
 
|-
 
{{:Netdata}}
 
|-
 
{{:Netstat}}
 
|-
 
{{:Perf}}
 
|-
 
{{:Strace}}
 
|-
 
{{:Sysprof}}
 
|-
 
{{:Sysstat tool suite}}
 
|-
 
{{:Tcpdump}}
 
|-
 
{{:Top Linux command line | Top}}
 
|-
 
{{:Valgrind}}
 
|}
 
 
== Trace and debug overview per Linux software frameworks ==
 
The picture below allows accessing to different Linux software frameworks which provide specific trace and debug information in their {{highlight|"''How to trace and debug the framework''"}} dedicated chapter.
 
 
{{:STM32MP15 Linux kernel overview}}
 
 
== Tips ==
 
[[How to find Linux kernel driver associated to a device]].
 
 
[[How to use the kernel dynamic debug]].
 
 
== Documentation and web articles ==
 
 
A lot of articles on the web mention Linux<sup>&reg;</sup> kernel tracing and profiling. The following links provide a good introduction to these topics:
 
* [http://www.brendangregg.com/Slides/LISA2014_LinuxPerfAnalysisNewTools.pdf Linux Performance Analysis - New Tools and Old Secrets]: description of the Linux<sup>&reg;</sup> technology and of the different tools available.
 
* [https://wiki.yoctoproject.org/wiki/Tracing_and_Profiling Yocto project: Tracing and profiling]: How to enable tracing and profiling tools using Yocto
 
* [http://www.brendangregg.com/linuxperf.html#Tools Brendan Gregg Linux performance page]
 
 
{{Info| More general Linux performance information are available in [http://fr.slideshare.net/brendangregg/linux-performance-tools-2014 nice slideshare presentation (Brendan Gregg)], on the [http://www.brendangregg.com/linuxperf.html Brendan Gregg official web site] or in LinuxCon2014 [http://www.linux.com/news/enterprise/systems-management/785426-a-template-for-monitoring-linux-performance-tools-like-from-netflix article].}}
 
 
Reference list:
 
<references />
 

2020年11月4日 (三) 18:04的版本