1 thought on “cheap wholesale piercing jewelry How to choose a Linux Tracer”
Walter
stainless steel wholesale jewelry Tracer is a high -level performance analysis and diagnosis tool, but don't let this term stunher you. If you have used Strace and TCPMP, you have already used Tracer. The system tracer can get more system calls and packets. They usually track any kernel and applications. The many Linux Tracer can choose. Each has its official (or unofficial) cartoon unicorn mascot, which is enough to support a "children's drama".
So which tracer should we use? I will answer this question for two types of readers, most people and performance/kernel engineers. After some time, these may change, and I will continue to follow up and add, and it will be updated once a year. Most people D. Most people (developers, system administrators, development managers, operation and maintenance personnel, evaluation staff, etc.) The details of the system tracking device do not care about the system tracker. The following is what you should know and do for the tracker: 1. Use perf_events to analyze the performance of CPU
In Perf_Events for CPU performance analysis. Performance indicators can use tools such as Flame Graph for visualization. git clone -Depth 1 github // perf record -F 99 -A -G -Sleep 30 perf script | .//perf.pl | .//.pl> perf. SVG
linux perf_events (also known as "perf", the same command name) is the official tracker and performance analyzer of Linux users. Endentized in the kernel code, it is well maintained (recently obtained rapid enhancement), usually adding the Linux command line tool package. Perf has many functions. If you can only recommend one, I choose the CPU performance analysis. Although this is just sampling, not technically tracking the incident. The hardest part is to get the complete stack and information. I have said this problem in a speech I made for Java and Node.js. Said: "You don't need to know how to operate the X -ray machine, but once you swallow a coin, you have to know that you have to do X -ray." You should know what kind of tracer can do so that you can be in you When the tracer is really needed at work, you can choose to learn and use later, or you can hire the corresponding person to complete. The short: Almost everything can be used to analyze and track. For example, file systems, network processors, hardware drivers, all applications. You can take a look at the articles about ftrace on my personal website, and the introduction of Perf_Events documentation I wrote, which can be used as an example of tracking (or performance analysis). 3. Seeking front -end support tool
If you want to buy a performance analysis tool that supports tracking Linux (there are many companies selling such tools). Imagine that you only need to click on the interface to "insight" the entire system kernel, including the hot diagram of hidden stack positions. I introduced a tool with a graphical interface in Talk. I opened some of the front -end tools I developed, although it is only a CLI (command line interface) instead of (graphic interface). These tools also make people use Tracer more quickly and easier. For example, the following example, use my perf_tool to track a new process: # ./ exec () s. Ctrl-C to end. PID PPID ARGS 22898 22004 man ls 22905 22898 Preconv -e UTF -8 22908 22898 Pager -S 22907 22898 NROFF -MANDOC -RLL = 164N -RLT = 164N -TUTF8 [...] On Netflix, we created a Vector, an example of an analysis tool is also the final front end of the tracer on Linux. The performance or kernel engineer
This work becomes more and more difficult. Many people will ask us how to track and which path can be used! In order to correctly understand a path, you often need to spend at least 100 hours to do it. It is a huge project to understand all the Linux paths to make rationality. (I may be the only person who is close to this) The here is my suggestion, one can choose one: a) Select a vertical path It takes a lot of time to figure out its subtle differences and security in the test environment. I now recommend the latest version of SystemTap (IE, from the source code). I know that some companies have chosen LTTNG, and they use well, although it is not very powerful (although it is safer). Sysdig can be another candidate if it can increase the tracking point or KPROBES. b) Follow the flowchart provided by me, which will mean as much as possible using FTRACE or Perf_Event. EBPF will be integrated. After that, other paths like SystemTap/LTTNG will fill this gap. This is what I currently do in Netflix. tracer's comments: 1. FTRACE I I like to use FTRACE. It is the first choice of kernel hacker. It is built into the system kernel. You can use the tracking point (static checkpoint). Debugging tools. And provide several such functions: the event tracking function with optional filters and parameters; event counting and timing functions of statistical statistics in the kernel; and function of functional process traversal. Take a look at the FTRACE.TXT example in the kernel code to find out. Ftrace is controlled by /sys and only supports a single root user (but you can crack multi -user through a buffer instance). At some time, the ftrace operation interface is very cumbersome, but it is indeed very "hack", and it has a front -end interface. Steven Rostedt, the main author of FTACE, created the Trace-CMD command tool, and I created the PERF tool set. My biggest dissatisfaction with this tool is that it is not programmed. For example, you cannot save and get the timestamp, you cannot calculate the delay, and you cannot save these calculations into the form of a histogram. You need to reserve an event to the user level, and spend some time to deal with the results. Ftrace can be transformed into programmable via EBPF. 2.perf_events
perf_events is the main tracking tool of Linux users. It is built into the kernel source code and is usually added via Linux-Tools-Commom. Also known as "PERF", the same front -end tool name is usually used to track and reserve information into a file called Perf.Data. Perf.data files are equivalent to a dynamic buffer area that needs to be processed after saving. Ftrace can do, most of Perf_Events can also do it. Perf-Events cannot do a function process traversing, and less "hack" energy (but better support for security/error check). It can perform CPU analysis and performance statistics, user -level stack analysis, and can also use debugging information generated by tracking each line of local variables. It also supports multi -user concurrent operation. It does not support programming as FTRACE. If I only recommend one tracer, it must be Perf. It can solve many problems and it is relatively safe. 3. ebpf
extended Berkeley Packet Filter (EBPF) is an efficient kernel virtual machine (JIT) that can run the program on the event. It may eventually provide kernel programming of ftrace and perf_events and strengthen other tracer. This is currently developed by Alexei and has not been fully integrated yet, but from 4.1, it has suffered sufficient kernels for some excellent tools, such as delayed hot maps of block equipment I/O. You can refer to the BPF Slides and EBPF SAMPLES of the main author Alexei. 4. Systemtap
systemtap is the most powerful tracer. It can do everything, such as summary analysis, tracking points, probes, Uprobes (from SystemTap), USDT and kernel programming. It compiles the program as a kernel module and then loads, which is a clever approach to obtain safety. It is also developing from Tree, and there are many problems in the past (many terrible). Many are not the fault of SystemTap itself -it is often the first to use the kernel tracking function, but also the first to encounter BUG. SystemTap's latest version is much better (compilation must be compiled by the source code), but many people are still scared by early versions. If you want to use it, you can use it in the test environment first, and communicate with the #SystemTap developers on IRC.Freenode. (Netflix has a fault tolerance mechanism, we have used SystemTap, but maybe there are fewer safety issues we consider than you.) My greatest dissatisfaction is that it seems that you often think that you often have the kernel DEBUG information. In fact, you can do a lot of things without it, but lack of documents and examples (I have to start learning by myself). 5. LTTNG Lttng optimized the event collection, which is better than other traces. It develops from TREE, its core is very simple: the event is written into the tracking buffer through a set of small -scale fixed instruction sets. This method makes it safe and fast. The disadvantage is that it does not have a simple way to encode the kernel encoding. I have always heard that this is not a big problem, because although it is necessary to deal with it, it has also been optimized to fully measure. In addition, it also created a different analytical technology, and more of the black box records that follow the incident will be studied later. What I care about is how to solve the problem of missing events to be recorded in the early stage, but what I really need to do is to spend more time to see how it is used in practice. This is a tracer with the least time I spent (there is no special reason). 6. Ktap KTAP was a Tracer with a good prospect in the past. It uses LUA virtual machine processing in the kernel to run well on the embedded device without debugging information. It is divided into several steps and seems to surpass all the trackers on Linux for a period of time. Then EBPF starts the kernel integration, and the integration of KTAP starts with EBPF instead of its own virtual machine. Because EBPF will continue to integrate for several months, KTAP developers will continue to wait for some time. I hope it can be re -developed later this year. 7.
mainly was done by Paul Fox alone in his spare time. It was the Linux version of Sun DTRACE. It introduces attention, and there are some provider that can run, but to some extent, it is not complete, and it is more an experimental tool (unsafe). In my opinion, people will carefully contribute to the contribution code: because the sun open source DTRACE uses the CDDL protocol, it is unlikely to eventually enter the Linux Kernel. Paul's method is likely to make it an add-on. I am happy to see the completion of the DTRACE and this project on the Linux platform. I think that when I join Netflix, I will spend some time to assist in completing the project. However, I still want to continue using built -in traces, such as ftrace and perf_events. 8.ol dtrace
oncle Linux dtrace has made great efforts to introduce DTRACE into Linux, especially Oracle Linux. Multiple versions released over the years have shown its stable progress. Developers talk about improving the DTRACE test kit with a promising attitude towards this project. Many useful provider has been completed, such as: SYSCALL, Profile, SDT, ProC, SCHED, and USDT. I am looking forward to the completion of FBT (Function Boundary Tracing, for kernel dynamic tracking), which is a very good provider on the Linux kernel. OL DTRACE's ultimate success will depend on how much interest in people's interest in running Oracle Linux (for technical support), on the other hand, it depends on whether it is completely open source: its kernel components are open source, and I have not seen its user level Code. 9. Sysdig
sysdig is a new tracer using TCPMP syntax to operate system events. It uses LUA to submit the process. It is excellent, and it witnesses the change in the field of system tracking. Its limitations are that it is currently called only by the current system, and all events are reserved into a user level when submitting. You can use system calls to do a lot of things, but I still hope it can support tracking points, KPROBE and Uprobe. I also look forward to supporting the EBPF as the kernel. At present, SYSDIG developers are increasing container support. Pay attention to these contents. This extension
. My work on tracer includes: ftrace: my Perf-Tools tool set (reference instance directory); my article on FTRACE on LWN; LISA14 Speaking; there are also posts: function counts, iOSNOP, OpenSnoop, ExecsNoop, TCP forwarding, Uprobes and USDT. perf_evenets: My webpage Perf_Events instance; the linux performance analysis of the linux of Netflix on the scale; there are also posts CPU sampling, static tracking point, hot spot map, counting, kernel tracing, OFF-CPU time diagram. ebpf: Post EBPF: Take a small step, and some BPF tools (I need to post more). systemtap: I wrote a little expired post with SystemTap a long time ago. Recently, I released some tools SystemTap-lwtools to demonstrate how to use SystemTap without kernel diagnostic information. Lttng: I only spent a little time, not enough to publish any content. ktap: My webpage KTAP instance contains some early versions of single -line applets and scripts. : I gave some examples in the book performance book and once developed some small repair procedures, such as. ol dtrace: Since it is transformed directly from DTRACE, many of my early work on dtrace is correlated (if there are too many links here, you can search on my homepage). When it is more complete, I will develop some special tools. sysdig: I contributed code to SubseCond Office. others: I wrote the precautions for strace. Please do not have more tracer! If you want to know why Linux has not only one tracer, or only with your own DTRACE, you can find the answer from DTRACE to Linux, starting from 28 slides. The editor of Deingrdré Straughan, and Tracing pony created with the creator of the General Zoi.
stainless steel wholesale jewelry Tracer is a high -level performance analysis and diagnosis tool, but don't let this term stunher you. If you have used Strace and TCPMP, you have already used Tracer. The system tracer can get more system calls and packets. They usually track any kernel and applications.
The many Linux Tracer can choose. Each has its official (or unofficial) cartoon unicorn mascot, which is enough to support a "children's drama".
So which tracer should we use?
I will answer this question for two types of readers, most people and performance/kernel engineers. After some time, these may change, and I will continue to follow up and add, and it will be updated once a year.
Most people
D. Most people (developers, system administrators, development managers, operation and maintenance personnel, evaluation staff, etc.) The details of the system tracking device do not care about the system tracker. The following is what you should know and do for the tracker:
1. Use perf_events to analyze the performance of CPU
In Perf_Events for CPU performance analysis. Performance indicators can use tools such as Flame Graph for visualization.
git clone -Depth 1 github //
perf record -F 99 -A -G -Sleep 30
perf script | .//perf.pl | .//.pl> perf. SVG
linux perf_events (also known as "perf", the same command name) is the official tracker and performance analyzer of Linux users. Endentized in the kernel code, it is well maintained (recently obtained rapid enhancement), usually adding the Linux command line tool package.
Perf has many functions. If you can only recommend one, I choose the CPU performance analysis. Although this is just sampling, not technically tracking the incident. The hardest part is to get the complete stack and information. I have said this problem in a speech I made for Java and Node.js. Said: "You don't need to know how to operate the X -ray machine, but once you swallow a coin, you have to know that you have to do X -ray." You should know what kind of tracer can do so that you can be in you When the tracer is really needed at work, you can choose to learn and use later, or you can hire the corresponding person to complete.
The short: Almost everything can be used to analyze and track. For example, file systems, network processors, hardware drivers, all applications. You can take a look at the articles about ftrace on my personal website, and the introduction of Perf_Events documentation I wrote, which can be used as an example of tracking (or performance analysis).
3. Seeking front -end support tool
If you want to buy a performance analysis tool that supports tracking Linux (there are many companies selling such tools). Imagine that you only need to click on the interface to "insight" the entire system kernel, including the hot diagram of hidden stack positions. I introduced a tool with a graphical interface in Talk.
I opened some of the front -end tools I developed, although it is only a CLI (command line interface) instead of (graphic interface). These tools also make people use Tracer more quickly and easier. For example, the following example, use my perf_tool to track a new process:
# ./ exec () s. Ctrl-C to end.
PID PPID ARGS
22898 22004 man ls
22905 22898 Preconv -e UTF -8
22908 22898 Pager -S
22907 22898 NROFF -MANDOC -RLL = 164N -RLT = 164N -TUTF8
[...]
On Netflix, we created a Vector, an example of an analysis tool is also the final front end of the tracer on Linux.
The performance or kernel engineer
This work becomes more and more difficult. Many people will ask us how to track and which path can be used! In order to correctly understand a path, you often need to spend at least 100 hours to do it. It is a huge project to understand all the Linux paths to make rationality. (I may be the only person who is close to this)
The here is my suggestion, one can choose one:
a) Select a vertical path It takes a lot of time to figure out its subtle differences and security in the test environment. I now recommend the latest version of SystemTap (IE, from the source code). I know that some companies have chosen LTTNG, and they use well, although it is not very powerful (although it is safer). Sysdig can be another candidate if it can increase the tracking point or KPROBES.
b) Follow the flowchart provided by me, which will mean as much as possible using FTRACE or Perf_Event. EBPF will be integrated. After that, other paths like SystemTap/LTTNG will fill this gap. This is what I currently do in Netflix.
tracer's comments:
1. FTRACE
I I like to use FTRACE. It is the first choice of kernel hacker. It is built into the system kernel. You can use the tracking point (static checkpoint). Debugging tools. And provide several such functions: the event tracking function with optional filters and parameters; event counting and timing functions of statistical statistics in the kernel; and function of functional process traversal. Take a look at the FTRACE.TXT example in the kernel code to find out. Ftrace is controlled by /sys and only supports a single root user (but you can crack multi -user through a buffer instance). At some time, the ftrace operation interface is very cumbersome, but it is indeed very "hack", and it has a front -end interface. Steven Rostedt, the main author of FTACE, created the Trace-CMD command tool, and I created the PERF tool set. My biggest dissatisfaction with this tool is that it is not programmed. For example, you cannot save and get the timestamp, you cannot calculate the delay, and you cannot save these calculations into the form of a histogram. You need to reserve an event to the user level, and spend some time to deal with the results. Ftrace can be transformed into programmable via EBPF.
2.perf_events
perf_events is the main tracking tool of Linux users. It is built into the kernel source code and is usually added via Linux-Tools-Commom. Also known as "PERF", the same front -end tool name is usually used to track and reserve information into a file called Perf.Data. Perf.data files are equivalent to a dynamic buffer area that needs to be processed after saving. Ftrace can do, most of Perf_Events can also do it. Perf-Events cannot do a function process traversing, and less "hack" energy (but better support for security/error check). It can perform CPU analysis and performance statistics, user -level stack analysis, and can also use debugging information generated by tracking each line of local variables. It also supports multi -user concurrent operation. It does not support programming as FTRACE. If I only recommend one tracer, it must be Perf. It can solve many problems and it is relatively safe.
3. ebpf
extended Berkeley Packet Filter (EBPF) is an efficient kernel virtual machine (JIT) that can run the program on the event. It may eventually provide kernel programming of ftrace and perf_events and strengthen other tracer. This is currently developed by Alexei and has not been fully integrated yet, but from 4.1, it has suffered sufficient kernels for some excellent tools, such as delayed hot maps of block equipment I/O. You can refer to the BPF Slides and EBPF SAMPLES of the main author Alexei.
4. Systemtap
systemtap is the most powerful tracer. It can do everything, such as summary analysis, tracking points, probes, Uprobes (from SystemTap), USDT and kernel programming. It compiles the program as a kernel module and then loads, which is a clever approach to obtain safety. It is also developing from Tree, and there are many problems in the past (many terrible). Many are not the fault of SystemTap itself -it is often the first to use the kernel tracking function, but also the first to encounter BUG. SystemTap's latest version is much better (compilation must be compiled by the source code), but many people are still scared by early versions. If you want to use it, you can use it in the test environment first, and communicate with the #SystemTap developers on IRC.Freenode. (Netflix has a fault tolerance mechanism, we have used SystemTap, but maybe there are fewer safety issues we consider than you.) My greatest dissatisfaction is that it seems that you often think that you often have the kernel DEBUG information. In fact, you can do a lot of things without it, but lack of documents and examples (I have to start learning by myself).
5. LTTNG
Lttng optimized the event collection, which is better than other traces. It develops from TREE, its core is very simple: the event is written into the tracking buffer through a set of small -scale fixed instruction sets. This method makes it safe and fast. The disadvantage is that it does not have a simple way to encode the kernel encoding. I have always heard that this is not a big problem, because although it is necessary to deal with it, it has also been optimized to fully measure. In addition, it also created a different analytical technology, and more of the black box records that follow the incident will be studied later. What I care about is how to solve the problem of missing events to be recorded in the early stage, but what I really need to do is to spend more time to see how it is used in practice. This is a tracer with the least time I spent (there is no special reason).
6. Ktap
KTAP was a Tracer with a good prospect in the past. It uses LUA virtual machine processing in the kernel to run well on the embedded device without debugging information. It is divided into several steps and seems to surpass all the trackers on Linux for a period of time. Then EBPF starts the kernel integration, and the integration of KTAP starts with EBPF instead of its own virtual machine. Because EBPF will continue to integrate for several months, KTAP developers will continue to wait for some time. I hope it can be re -developed later this year.
7.
mainly was done by Paul Fox alone in his spare time. It was the Linux version of Sun DTRACE. It introduces attention, and there are some provider that can run, but to some extent, it is not complete, and it is more an experimental tool (unsafe). In my opinion, people will carefully contribute to the contribution code: because the sun open source DTRACE uses the CDDL protocol, it is unlikely to eventually enter the Linux Kernel. Paul's method is likely to make it an add-on. I am happy to see the completion of the DTRACE and this project on the Linux platform. I think that when I join Netflix, I will spend some time to assist in completing the project. However, I still want to continue using built -in traces, such as ftrace and perf_events.
8.ol dtrace
oncle Linux dtrace has made great efforts to introduce DTRACE into Linux, especially Oracle Linux. Multiple versions released over the years have shown its stable progress. Developers talk about improving the DTRACE test kit with a promising attitude towards this project. Many useful provider has been completed, such as: SYSCALL, Profile, SDT, ProC, SCHED, and USDT. I am looking forward to the completion of FBT (Function Boundary Tracing, for kernel dynamic tracking), which is a very good provider on the Linux kernel. OL DTRACE's ultimate success will depend on how much interest in people's interest in running Oracle Linux (for technical support), on the other hand, it depends on whether it is completely open source: its kernel components are open source, and I have not seen its user level Code.
9. Sysdig
sysdig is a new tracer using TCPMP syntax to operate system events. It uses LUA to submit the process. It is excellent, and it witnesses the change in the field of system tracking. Its limitations are that it is currently called only by the current system, and all events are reserved into a user level when submitting. You can use system calls to do a lot of things, but I still hope it can support tracking points, KPROBE and Uprobe. I also look forward to supporting the EBPF as the kernel. At present, SYSDIG developers are increasing container support. Pay attention to these contents.
This extension
. My work on tracer includes:
ftrace: my Perf-Tools tool set (reference instance directory); my article on FTRACE on LWN; LISA14 Speaking; there are also posts: function counts, iOSNOP, OpenSnoop, ExecsNoop, TCP forwarding, Uprobes and USDT.
perf_evenets: My webpage Perf_Events instance; the linux performance analysis of the linux of Netflix on the scale; there are also posts CPU sampling, static tracking point, hot spot map, counting, kernel tracing, OFF-CPU time diagram.
ebpf: Post EBPF: Take a small step, and some BPF tools (I need to post more).
systemtap: I wrote a little expired post with SystemTap a long time ago. Recently, I released some tools SystemTap-lwtools to demonstrate how to use SystemTap without kernel diagnostic information.
Lttng: I only spent a little time, not enough to publish any content.
ktap: My webpage KTAP instance contains some early versions of single -line applets and scripts.
: I gave some examples in the book performance book and once developed some small repair procedures, such as.
ol dtrace: Since it is transformed directly from DTRACE, many of my early work on dtrace is correlated (if there are too many links here, you can search on my homepage). When it is more complete, I will develop some special tools.
sysdig: I contributed code to SubseCond Office.
others: I wrote the precautions for strace.
Please do not have more tracer! If you want to know why Linux has not only one tracer, or only with your own DTRACE, you can find the answer from DTRACE to Linux, starting from 28 slides.
The editor of Deingrdré Straughan, and Tracing pony created with the creator of the General Zoi.