Skip to content

Commit 07ef337

Browse files
ellishgellishg
committed
Add documentation for MIP
Add documentation for MIP instrumentation and the `llvm-mipdata` tool. Differential Revision: https://reviews.llvm.org/D104083
1 parent 65a7ad5 commit 07ef337

File tree

5 files changed

+333
-3
lines changed

5 files changed

+333
-3
lines changed

‎llvm/docs/CommandGuide/index.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,7 @@ Basic Commands
2929
llvm-link
3030
llvm-lipo
3131
llvm-mca
32+
llvm-mipdata
3233
llvm-otool
3334
llvm-profdata
3435
llvm-readobj

‎llvm/docs/CommandGuide/llvm-mipdata.rst

Lines changed: 118 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,118 @@
1+
llvm-mipdata - MIP tool
2+
=======================
3+
4+
.. program:: llvm-mipdata
5+
6+
SYNOPSIS
7+
--------
8+
9+
:program:`llvm-mipdata` *command* [*args...*]
10+
11+
DESCRIPTION
12+
-----------
13+
14+
The :program:`llvm-mipdata` tool creates, manipulates, and reports
15+
information about :doc:`MIP <../MachineProfile>` profiles (``.mip`` files).
16+
17+
General Options
18+
---------------
19+
20+
.. option:: --help
21+
22+
Print a summary of command line options.
23+
24+
.. option:: --profile=<profile>, -p <profile>
25+
26+
Specify the profile to work on.
27+
28+
.. option:: --output=<output>, -o <output>
29+
30+
Specify the output file name.
31+
32+
.. option:: --version
33+
34+
Print the current version of this tool.
35+
36+
37+
COMMANDS
38+
--------
39+
40+
* :ref:`create <create>`
41+
* :ref:`merge <merge>`
42+
* :ref:`show <show>`
43+
44+
.. _create:
45+
46+
CREATE
47+
------
48+
49+
SYNOPSIS
50+
^^^^^^^^
51+
52+
:program:`llvm-mipdata create` [*options*] **-p <profile>** <map_file>
53+
54+
DESCRIPTION
55+
^^^^^^^^^^^
56+
57+
:program:`llvm-mipdata create` creates an empty <profile> from a <map_file>
58+
(``.mipmap`` file).
59+
60+
.. _merge:
61+
62+
MERGE
63+
-----
64+
65+
SYNOPSIS
66+
^^^^^^^^
67+
68+
:program:`llvm-mipdata merge` [*options*] **-p <profile>** <raw_files>
69+
70+
DESCRIPTION
71+
^^^^^^^^^^^
72+
73+
:program:`llvm-mipdata merge` merges raw profiles (``.mipraw`` files) into the
74+
profile.
75+
76+
OPTIONS
77+
^^^^^^^
78+
79+
.. option:: --strict
80+
81+
With strict mode, merging a corrupt raw profile will fail the command.
82+
Disabled by default.
83+
84+
.. _show:
85+
86+
SHOW
87+
----
88+
89+
SYNOPSIS
90+
^^^^^^^^
91+
92+
:program:`llvm-mipdata show` [*options*] **-p <profile>**
93+
94+
DESCRIPTION
95+
^^^^^^^^^^^
96+
97+
:program:`llvm-mipdata show` prints the data from the given profile.
98+
99+
OPTIONS
100+
^^^^^^^
101+
102+
.. option:: --debug=<debug_info>, -d <debug_info>
103+
104+
Use <debug_info> to symbolicate addresses in the profile.
105+
106+
.. option:: --regex=<regex>, -r <regex>
107+
108+
Only process function names that match <regex>.
109+
110+
111+
EXAMPLE
112+
-------
113+
114+
.. code-block:: console
115+
116+
$ llvm-mipdata create -p default.mip default.mipmap
117+
$ llvm-mipdata merge -p default.mip default-0000.mipraw default-0001.mipraw default-0002.mipraw
118+
$ llvm-mipdata show -p default.mip --debug a.out

‎llvm/docs/MIPBinaryFormat.rst

Lines changed: 126 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,126 @@
1+
======================================
2+
Machine IR Profile (MIP) Binary Format
3+
======================================
4+
5+
.. contents::
6+
:local:
7+
8+
Introduction
9+
============
10+
MIP, which is described in :doc:`MachineProfile <MachineProfile>`, uses three file types to hold
11+
profile data: ``.mip``, ``.mipmap``, ``.mipraw``. Each format has a common
12+
header and the data is layed out in little endian format.
13+
14+
Header Format
15+
=============
16+
17+
.. code-block:: c
18+
19+
Header {
20+
i8[4] : Magic = {0xFB, 'M', 'I', 'P'} // 0xFB 0x4D 0x49 0x50
21+
i16 : Version = 0x8
22+
i16 : FileType
23+
i32 : ProfileType
24+
i32 : ModuleHash // MD5Hash(LinkUnitName)
25+
i32 : Reserved
26+
i32 : OffsetToData = 0x18 // 24 bytes
27+
}
28+
29+
``FileType``
30+
------------
31+
=========== ======
32+
Type Value
33+
----------- ------
34+
``.mipraw`` 0x0001
35+
``.mipmap`` 0x0002
36+
``.mip`` 0x0003
37+
=========== ======
38+
39+
``ProfileType``
40+
---------------
41+
============================== ==========
42+
Type Value
43+
------------------------------ ----------
44+
Function Coverage 0x00000001
45+
Block Coverage 0x00000002
46+
Function Timestamp 0x00000004
47+
Function Call Count 0x00000008
48+
Return Address Instrumentation 0x00000010
49+
============================== ==========
50+
51+
``.mipraw``
52+
===========
53+
If block coverage is not enabled then ``BlockCount`` is zero for all profiles
54+
and padding is not necessary.
55+
56+
Function Coverage
57+
-----------------
58+
.. code-block:: c
59+
60+
Header
61+
RawProfile[] {
62+
i8 : IsCovered
63+
i8[BlockCount] : IsCovered
64+
// Pad to 8 bits
65+
}
66+
67+
Function Timestamp + Function Call Count
68+
----------------------------------------
69+
.. code-block:: c
70+
71+
Header
72+
RawProfile[] {
73+
i32 : CallCount
74+
i32 : Timestamp
75+
i8[BlockCount] : IsCovered
76+
// Pad to 32 bits
77+
}
78+
79+
``.mipmap``
80+
===========
81+
.. code-block:: c
82+
83+
Header
84+
FunctionProfile[] {
85+
i32 : RawSectionStartPCOffset
86+
i32 : RawProfilePCOffset
87+
i32 : FunctionPCOffset
88+
i32 : FunctionSize
89+
i32 : ControlFlowGraphSignature
90+
i32 : NonEntryBlockCount
91+
BlockProfile[NonEntryBlockCount] {
92+
i32 : Offset
93+
}
94+
i32 : FunctionNameLength
95+
char[FunctionNameLength] : FunctionName
96+
// Pad to 64 bits
97+
}
98+
99+
``.mip``
100+
========
101+
102+
.. code-block:: c
103+
104+
Header
105+
i64 : ProfileCount
106+
FunctionProfile[ProfileCount] {
107+
i64 : FunctionSignature // MD5Hash(FunctionName)
108+
i32 : RawProfileDataOffset
109+
i32 : FunctionStartOffset
110+
i32 : FunctionSize
111+
i32 : ControlFlowGraphSignature
112+
i32 : NonEntryBlockCount
113+
i32 : MergeCount
114+
i64 : CallCount
115+
i64 : TimestampSum
116+
BlockProfile[NonEntryBlockCount] {
117+
i32 : Offset
118+
i8 : IsCovered
119+
}
120+
i32 : CallEdgeCount
121+
CallEdges[CallEdgeCount] {
122+
// Reserved
123+
}
124+
}
125+
i64 : FunctionNamesLength
126+
char[FunctionNamesLength] : FunctionNames // nullbyte separated

‎llvm/docs/MachineProfile.rst

Lines changed: 80 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,80 @@
1+
========================
2+
Machine IR Profile (MIP)
3+
========================
4+
5+
.. contents::
6+
:local:
7+
8+
Introduction
9+
============
10+
MIP is an instrumentation framework for collecting runtime statistics and using
11+
the profiles for analysis or as input to the compiler for optimization. The
12+
metrics we collect include function call count, function call order, and a
13+
sample of the dynamic call graph. There is also a lightweight mode where
14+
function coverage and basic block coverage are collected with minimal impact
15+
on the performance and size of the instrumented binaries. MIP is currently
16+
implemented for both ELF and MachO with the following architectures:
17+
18+
* x86_64
19+
* aarch64
20+
* armv7 (thumb1 and thumb2)
21+
22+
MIP uses three file types (``.mip``, ``.mipmap``, ``.mipraw``) which are
23+
described in :doc:`MIPBinaryFormat <MIPBinaryFormat>`.
24+
25+
Motivation
26+
----------
27+
Existing instrumentation frameworks such as :doc:`XRay <XRay>` and
28+
`Clang Coverage <https://clang.llvm.org/docs/SourceBasedCodeCoverage.html>`
29+
produce instrumented binaries that are too large or too slow to be used in
30+
many scenarios without sacrificing runtime behavior. Our goals for MIP are to
31+
32+
Usage
33+
=====
34+
Building
35+
--------
36+
To build an instrumented binary, use the ``-fmachine-profile-generate`` clang
37+
flag.
38+
39+
.. code-block:: console
40+
41+
$ clang -fmachine-profile-generate main.cpp
42+
43+
44+
Running
45+
-------
46+
Run the instrumented binary like normal to cover as many use cases as
47+
possible. When the program exits the default behavior is to write a raw
48+
profile to ``default.mipraw``. This path can be overwritten with the
49+
``LLVM_MIP_PROFILE_FILENAME`` environment variable.
50+
51+
.. code-block:: console
52+
53+
$ LLVM_MIP_PROFILE_FILENAME=/path/to/profile.mipraw ./a.out
54+
55+
56+
Post Processing
57+
---------------
58+
After collecting one or more raw profiles (``.mipraw`` files) we can merge
59+
them into one profile (``.mip`` file) for analysis or optimization. First, we
60+
need to extract and create an empty profile from our instrumented binary.
61+
62+
.. code-block:: console
63+
64+
$ llvm-objcopy --dump-section=__llvm_mipmap=default.mipmap <binary> /dev/null
65+
$ llvm-mipdata create -p default.mip default.mipmap
66+
67+
Then we can use the :doc:`llvm-mipdata <CommandGuide/llvm-mipdata>` tool to
68+
merge our raw profiles.
69+
70+
.. code-block:: console
71+
72+
$ llvm-mipdata merge -p default.mip default.mipraw [default2.mipraw ...]
73+
74+
The resulting ``default.mip`` file can be used as input to the
75+
``-fmachine-profile-use=default.mip`` frontent flag, or by the
76+
:doc:`llvm-mipdata <CommandGuide/llvm-mipdata>` tool for analysis.
77+
78+
.. code-block:: console
79+
80+
$ llvm-mipdata show -p default.mip

‎llvm/docs/Reference.rst

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,9 @@ LLVM and API reference documentation.
3131
LangRef
3232
LibFuzzer
3333
MarkedUpDisassembly
34+
MachineProfile
3435
MIRLangRef
36+
MIPBinaryFormat
3537
OptBisect
3638
PDB/index
3739
ScudoHardenedAllocator
@@ -139,9 +141,12 @@ Testing and Debugging
139141
:doc:`GwpAsan`
140142
A sampled heap memory error detection toolkit designed for production use.
141143

142-
====
143-
XRay
144-
====
144+
===============
145+
Instrumentation
146+
===============
147+
148+
:doc:`MachineProfile`
149+
Describes how to use instrument binaries with MIP.
145150

146151
:doc:`XRay`
147152
High-level documentation of how to use XRay in LLVM.

0 commit comments

Comments
 (0)