.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "auto_examples/plot_profiling.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        :ref:`Go to the end <sphx_glr_download_auto_examples_plot_profiling.py>`
        to download the full example code.

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_auto_examples_plot_profiling.py:


.. _l-onnx-array-onnxruntime-profiling:

Profiling with onnxruntime
==========================

*onnxruntime* optimizes the onnx graph by default before running
the inference. It modifies, fuses or add new operators.
Some of them are standard onnx operators, some of them
are implemented in onnxruntime (see `Supported Operators
<https://github.com/microsoft/onnxruntime/blob/main/docs/OperatorKernels.md>`_).
This example profiles the two models.

Optimize a model with onnxruntime
+++++++++++++++++++++++++++++++++

.. GENERATED FROM PYTHON SOURCE LINES 18-38

.. code-block:: Python


    import os
    import numpy
    import matplotlib.pyplot as plt
    from onnxruntime import get_available_providers
    from onnx_array_api.ext_test_case import example_path
    from onnx_array_api.ort.ort_optimizers import ort_optimized_model
    from onnx_array_api.ort.ort_profile import ort_profile, merge_ort_profile
    from onnx_array_api.plotting.stat_plot import plot_ort_profile


    suffix = ""
    filename = example_path(f"data/small{suffix}.onnx")
    optimized = filename + ".optimized.onnx"
    print(f"model={filename!r}")

    if not os.path.exists(optimized):
        ort_optimized_model(filename, output=optimized)
    print(f"optimized={optimized!r}")





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    model='data/small.onnx'
    optimized='data/small.onnx.optimized.onnx'




.. GENERATED FROM PYTHON SOURCE LINES 39-43

.. _l-example-ort-profiling:

Profiling
+++++++++

.. GENERATED FROM PYTHON SOURCE LINES 43-55

.. code-block:: Python


    feeds = {"input": numpy.random.random((1, 3, 112, 112)).astype(numpy.float32)}
    prof_base = ort_profile(
        filename,
        feeds,
        repeat=6,
        disable_optimization=True,
        providers=["CPUExecutionProvider"],
    )
    prof_base.to_excel(f"prof_base{suffix}.xlsx", index=False)
    prof_base






.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>cat</th>
          <th>pid</th>
          <th>tid</th>
          <th>dur</th>
          <th>ts</th>
          <th>ph</th>
          <th>name</th>
          <th>args_op_name</th>
          <th>op_name</th>
          <th>args_thread_scheduling_stats</th>
          <th>args_output_type_shape</th>
          <th>args_output_size</th>
          <th>args_parameter_size</th>
          <th>args_activation_size</th>
          <th>args_node_index</th>
          <th>args_input_type_shape</th>
          <th>args_provider</th>
          <th>event_name</th>
          <th>iteration</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>0</th>
          <td>Session</td>
          <td>85022</td>
          <td>85022</td>
          <td>483</td>
          <td>3</td>
          <td>X</td>
          <td>model_loading_uri</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>model_loading_uri</td>
          <td>-1</td>
        </tr>
        <tr>
          <th>1</th>
          <td>Session</td>
          <td>85022</td>
          <td>85022</td>
          <td>466</td>
          <td>518</td>
          <td>X</td>
          <td>session_initialization</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>session_initialization</td>
          <td>-1</td>
        </tr>
        <tr>
          <th>2</th>
          <td>Node</td>
          <td>85022</td>
          <td>85022</td>
          <td>0</td>
          <td>1139</td>
          <td>X</td>
          <td>n0_fence_before</td>
          <td>Conv</td>
          <td>n0</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>fence_before</td>
          <td>-1</td>
        </tr>
        <tr>
          <th>3</th>
          <td>Node</td>
          <td>85022</td>
          <td>85022</td>
          <td>965</td>
          <td>1141</td>
          <td>X</td>
          <td>n0_kernel_time</td>
          <td>Conv</td>
          <td>n0</td>
          <td>{'main_thread': {'thread_pool_name': 'session-...</td>
          <td>[{'float': [1, 64, 112, 112]}]</td>
          <td>3211264</td>
          <td>7168</td>
          <td>150528</td>
          <td>0</td>
          <td>[{'float': [1, 3, 112, 112]}, {'float': [64, 3...</td>
          <td>CPUExecutionProvider</td>
          <td>kernel_time</td>
          <td>-1</td>
        </tr>
        <tr>
          <th>4</th>
          <td>Node</td>
          <td>85022</td>
          <td>85022</td>
          <td>0</td>
          <td>2114</td>
          <td>X</td>
          <td>n0_fence_after</td>
          <td>Conv</td>
          <td>n0</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>fence_after</td>
          <td>-1</td>
        </tr>
        <tr>
          <th>...</th>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
        </tr>
        <tr>
          <th>261</th>
          <td>Node</td>
          <td>85022</td>
          <td>85022</td>
          <td>0</td>
          <td>238872</td>
          <td>X</td>
          <td>n13_fence_before</td>
          <td>Add</td>
          <td>n13</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>fence_before</td>
          <td>4</td>
        </tr>
        <tr>
          <th>262</th>
          <td>Node</td>
          <td>85022</td>
          <td>85022</td>
          <td>70</td>
          <td>238873</td>
          <td>X</td>
          <td>n13_kernel_time</td>
          <td>Add</td>
          <td>n13</td>
          <td>{'main_thread': {'thread_pool_name': 'session-...</td>
          <td>[{'float': [1, 64, 56, 56]}]</td>
          <td>802816</td>
          <td>0</td>
          <td>1605632</td>
          <td>13</td>
          <td>[{'float': [1, 64, 56, 56]}, {'float': [1, 64,...</td>
          <td>CPUExecutionProvider</td>
          <td>kernel_time</td>
          <td>4</td>
        </tr>
        <tr>
          <th>263</th>
          <td>Node</td>
          <td>85022</td>
          <td>85022</td>
          <td>0</td>
          <td>238949</td>
          <td>X</td>
          <td>n13_fence_after</td>
          <td>Add</td>
          <td>n13</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>fence_after</td>
          <td>4</td>
        </tr>
        <tr>
          <th>264</th>
          <td>Session</td>
          <td>85022</td>
          <td>85022</td>
          <td>12851</td>
          <td>226101</td>
          <td>X</td>
          <td>SequentialExecutor::Execute</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>SequentialExecutor::Execute</td>
          <td>5</td>
        </tr>
        <tr>
          <th>265</th>
          <td>Session</td>
          <td>85022</td>
          <td>85022</td>
          <td>12876</td>
          <td>226084</td>
          <td>X</td>
          <td>model_run</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>model_run</td>
          <td>5</td>
        </tr>
      </tbody>
    </table>
    <p>266 rows × 19 columns</p>
    </div>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 56-57

And the optimized model.

.. GENERATED FROM PYTHON SOURCE LINES 57-68

.. code-block:: Python


    prof_opti = ort_profile(
        optimized,
        feeds,
        repeat=6,
        disable_optimization=True,
        providers=["CPUExecutionProvider"],
    )
    prof_opti.to_excel(f"prof_opti{suffix}.xlsx", index=False)
    prof_opti






.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>cat</th>
          <th>pid</th>
          <th>tid</th>
          <th>dur</th>
          <th>ts</th>
          <th>ph</th>
          <th>name</th>
          <th>args_op_name</th>
          <th>op_name</th>
          <th>args_thread_scheduling_stats</th>
          <th>args_output_type_shape</th>
          <th>args_output_size</th>
          <th>args_parameter_size</th>
          <th>args_activation_size</th>
          <th>args_node_index</th>
          <th>args_input_type_shape</th>
          <th>args_provider</th>
          <th>event_name</th>
          <th>iteration</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>0</th>
          <td>Session</td>
          <td>85022</td>
          <td>85022</td>
          <td>497</td>
          <td>2</td>
          <td>X</td>
          <td>model_loading_uri</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>model_loading_uri</td>
          <td>-1</td>
        </tr>
        <tr>
          <th>1</th>
          <td>Session</td>
          <td>85022</td>
          <td>85022</td>
          <td>378</td>
          <td>525</td>
          <td>X</td>
          <td>session_initialization</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>session_initialization</td>
          <td>-1</td>
        </tr>
        <tr>
          <th>2</th>
          <td>Node</td>
          <td>85022</td>
          <td>85022</td>
          <td>0</td>
          <td>1023</td>
          <td>X</td>
          <td>r0_nchwc_fence_before</td>
          <td>Conv</td>
          <td>r0_nchwc</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>fence_before</td>
          <td>-1</td>
        </tr>
        <tr>
          <th>3</th>
          <td>Node</td>
          <td>85022</td>
          <td>85022</td>
          <td>672</td>
          <td>1026</td>
          <td>X</td>
          <td>r0_nchwc_kernel_time</td>
          <td>Conv</td>
          <td>r0_nchwc</td>
          <td>{'main_thread': {'thread_pool_name': 'session-...</td>
          <td>[{'float': [1, 64, 112, 112]}]</td>
          <td>3211264</td>
          <td>7168</td>
          <td>150528</td>
          <td>0</td>
          <td>[{'float': [1, 3, 112, 112]}, {'float': [64, 3...</td>
          <td>CPUExecutionProvider</td>
          <td>kernel_time</td>
          <td>-1</td>
        </tr>
        <tr>
          <th>4</th>
          <td>Node</td>
          <td>85022</td>
          <td>85022</td>
          <td>0</td>
          <td>1704</td>
          <td>X</td>
          <td>r0_nchwc_fence_after</td>
          <td>Conv</td>
          <td>r0_nchwc</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>fence_after</td>
          <td>-1</td>
        </tr>
        <tr>
          <th>...</th>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
        </tr>
        <tr>
          <th>351</th>
          <td>Node</td>
          <td>85022</td>
          <td>85022</td>
          <td>0</td>
          <td>249744</td>
          <td>X</td>
          <td>ReorderOutput_token_16_fence_before</td>
          <td>ReorderOutput</td>
          <td>ReorderOutput_token_16</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>fence_before</td>
          <td>4</td>
        </tr>
        <tr>
          <th>352</th>
          <td>Node</td>
          <td>85022</td>
          <td>85022</td>
          <td>48</td>
          <td>249745</td>
          <td>X</td>
          <td>ReorderOutput_token_16_kernel_time</td>
          <td>ReorderOutput</td>
          <td>ReorderOutput_token_16</td>
          <td>{'main_thread': {'thread_pool_name': 'session-...</td>
          <td>[{'float': [1, 64, 56, 56]}]</td>
          <td>802816</td>
          <td>0</td>
          <td>802816</td>
          <td>18</td>
          <td>[{'float': [1, 64, 56, 56]}]</td>
          <td>CPUExecutionProvider</td>
          <td>kernel_time</td>
          <td>4</td>
        </tr>
        <tr>
          <th>353</th>
          <td>Node</td>
          <td>85022</td>
          <td>85022</td>
          <td>0</td>
          <td>249796</td>
          <td>X</td>
          <td>ReorderOutput_token_16_fence_after</td>
          <td>ReorderOutput</td>
          <td>ReorderOutput_token_16</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>fence_after</td>
          <td>4</td>
        </tr>
        <tr>
          <th>354</th>
          <td>Session</td>
          <td>85022</td>
          <td>85022</td>
          <td>23834</td>
          <td>225966</td>
          <td>X</td>
          <td>SequentialExecutor::Execute</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>SequentialExecutor::Execute</td>
          <td>5</td>
        </tr>
        <tr>
          <th>355</th>
          <td>Session</td>
          <td>85022</td>
          <td>85022</td>
          <td>23860</td>
          <td>225951</td>
          <td>X</td>
          <td>model_run</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>model_run</td>
          <td>5</td>
        </tr>
      </tbody>
    </table>
    <p>356 rows × 19 columns</p>
    </div>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 69-70

And the graph is:

.. GENERATED FROM PYTHON SOURCE LINES 70-78

.. code-block:: Python


    unique_op = set(prof_base["args_op_name"])
    fig, ax = plt.subplots(2, 2, figsize=(10, len(unique_op)), sharex="col")
    plot_ort_profile(prof_base, ax[0, 0], ax[0, 1], title="baseline")
    plot_ort_profile(prof_opti, ax[1, 0], ax[1, 1], title="optimized")
    fig.tight_layout()
    fig.savefig(f"plot_profiling{suffix}.png")




.. image-sg:: /auto_examples/images/sphx_glr_plot_profiling_001.png
   :alt: baseline, n occurences, optimized, n occurences
   :srcset: /auto_examples/images/sphx_glr_plot_profiling_001.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 79-85

Merging profiles
++++++++++++++++

Let's try to compare both profiles assuming every iteration
process the same image and the input and output size are the
same at every iteration.

.. GENERATED FROM PYTHON SOURCE LINES 85-90

.. code-block:: Python


    merge, gr = merge_ort_profile(prof_base, prof_opti)
    merge.to_excel(f"plot_profiling_merged{suffix}.xlsx", index=False)
    merge





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    /home/xadupre/github/onnx-array-api/onnx_array_api/ort/ort_profile.py:260: FutureWarning: The provided callable <function sum at 0x7fc0d2399d80> is currently using SeriesGroupBy.sum. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string "sum" instead.
      .agg(
    /home/xadupre/github/onnx-array-api/onnx_array_api/ort/ort_profile.py:260: FutureWarning: The provided callable <function sum at 0x7fc0d2399d80> is currently using SeriesGroupBy.sum. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string "sum" instead.
      .agg(


.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>args_op_name</th>
          <th>args_output_type_shape</th>
          <th>args_input_type_shape</th>
          <th>args_provider</th>
          <th>idx</th>
          <th>durbase</th>
          <th>countbase</th>
          <th>duropti</th>
          <th>countopti</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>0</th>
          <td>Add</td>
          <td>[{'float': [1, 64, 56, 56]}]</td>
          <td>[{'float': [1, 64, 56, 56]}, {'float': [1, 64,...</td>
          <td>CPUExecutionProvider</td>
          <td>0</td>
          <td>2420.0</td>
          <td>6.0</td>
          <td>NaN</td>
          <td>NaN</td>
        </tr>
        <tr>
          <th>1</th>
          <td>BatchNormalization</td>
          <td>[{'float': [1, 64, 112, 112]}]</td>
          <td>[{'float': [1, 64, 112, 112]}, {'float': [64]}...</td>
          <td>CPUExecutionProvider</td>
          <td>0</td>
          <td>2548.0</td>
          <td>6.0</td>
          <td>8064.0</td>
          <td>6.0</td>
        </tr>
        <tr>
          <th>2</th>
          <td>Concat</td>
          <td>[{'float': [1, 2, 112, 112]}]</td>
          <td>[{'float': [1, 1, 112, 112]}, {'float': [1, 1,...</td>
          <td>CPUExecutionProvider</td>
          <td>0</td>
          <td>166.0</td>
          <td>6.0</td>
          <td>127.0</td>
          <td>6.0</td>
        </tr>
        <tr>
          <th>3</th>
          <td>Conv</td>
          <td>[{'float': [1, 1, 112, 112]}]</td>
          <td>[{'float': [1, 2, 112, 112]}, {'float': [1, 2,...</td>
          <td>CPUExecutionProvider</td>
          <td>0</td>
          <td>2497.0</td>
          <td>6.0</td>
          <td>NaN</td>
          <td>NaN</td>
        </tr>
        <tr>
          <th>4</th>
          <td>Conv</td>
          <td>[{'float': [1, 64, 112, 112]}]</td>
          <td>[{'float': [1, 3, 112, 112]}, {'float': [64, 3...</td>
          <td>CPUExecutionProvider</td>
          <td>0</td>
          <td>5364.0</td>
          <td>6.0</td>
          <td>4311.0</td>
          <td>6.0</td>
        </tr>
        <tr>
          <th>5</th>
          <td>Conv</td>
          <td>[{'float': [1, 64, 112, 112]}]</td>
          <td>[{'float': [1, 64, 112, 112]}, {'float': [64, ...</td>
          <td>CPUExecutionProvider</td>
          <td>0</td>
          <td>101656.0</td>
          <td>6.0</td>
          <td>102240.0</td>
          <td>6.0</td>
        </tr>
        <tr>
          <th>6</th>
          <td>Conv</td>
          <td>[{'float': [1, 64, 56, 56]}]</td>
          <td>[{'float': [1, 64, 112, 112]}, {'float': [64, ...</td>
          <td>CPUExecutionProvider</td>
          <td>0</td>
          <td>9524.0</td>
          <td>6.0</td>
          <td>1767.0</td>
          <td>6.0</td>
        </tr>
        <tr>
          <th>7</th>
          <td>Conv</td>
          <td>[{'float': [1, 64, 56, 56]}]</td>
          <td>[{'float': [1, 64, 112, 112]}, {'float': [64, ...</td>
          <td>CPUExecutionProvider</td>
          <td>0</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>53992.0</td>
          <td>6.0</td>
        </tr>
        <tr>
          <th>8</th>
          <td>Conv</td>
          <td>[{'float': [1, 64, 56, 56]}]</td>
          <td>[{'float': [1, 64, 112, 112]}, {'float': [64, ...</td>
          <td>CPUExecutionProvider</td>
          <td>0</td>
          <td>59088.0</td>
          <td>6.0</td>
          <td>NaN</td>
          <td>NaN</td>
        </tr>
        <tr>
          <th>9</th>
          <td>Conv</td>
          <td>[{'float': [1, 8, 112, 112]}]</td>
          <td>[{'float': [1, 2, 112, 112]}, {'float': [8, 2,...</td>
          <td>CPUExecutionProvider</td>
          <td>0</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>19992.0</td>
          <td>6.0</td>
        </tr>
        <tr>
          <th>10</th>
          <td>Mul</td>
          <td>[{'float': [1, 64, 112, 112]}]</td>
          <td>[{'float': [1, 1, 112, 112]}, {'float': [1, 64...</td>
          <td>CPUExecutionProvider</td>
          <td>0</td>
          <td>2140.0</td>
          <td>6.0</td>
          <td>5791.0</td>
          <td>6.0</td>
        </tr>
        <tr>
          <th>11</th>
          <td>PRelu</td>
          <td>[{'float': [1, 64, 112, 112]}]</td>
          <td>[{'float': [1, 64, 112, 112]}, {'float': [64, ...</td>
          <td>CPUExecutionProvider</td>
          <td>0</td>
          <td>12364.0</td>
          <td>6.0</td>
          <td>1828.0</td>
          <td>6.0</td>
        </tr>
        <tr>
          <th>12</th>
          <td>PRelu</td>
          <td>[{'float': [1, 64, 112, 112]}]</td>
          <td>[{'float': [1, 64, 112, 112]}, {'float': [64, ...</td>
          <td>CPUExecutionProvider</td>
          <td>1</td>
          <td>2347.0</td>
          <td>6.0</td>
          <td>1783.0</td>
          <td>6.0</td>
        </tr>
        <tr>
          <th>13</th>
          <td>ReduceMax</td>
          <td>[{'float': [1, 1, 112, 112]}]</td>
          <td>[{'float': [1, 64, 112, 112]}]</td>
          <td>CPUExecutionProvider</td>
          <td>0</td>
          <td>13900.0</td>
          <td>6.0</td>
          <td>23211.0</td>
          <td>6.0</td>
        </tr>
        <tr>
          <th>14</th>
          <td>ReduceMean</td>
          <td>[{'float': [1, 1, 112, 112]}]</td>
          <td>[{'float': [1, 64, 112, 112]}]</td>
          <td>CPUExecutionProvider</td>
          <td>0</td>
          <td>20925.0</td>
          <td>6.0</td>
          <td>5553.0</td>
          <td>6.0</td>
        </tr>
        <tr>
          <th>15</th>
          <td>ReorderInput</td>
          <td>[{'float': [1, 64, 112, 112]}]</td>
          <td>[{'float': [1, 64, 112, 112]}]</td>
          <td>CPUExecutionProvider</td>
          <td>0</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>6279.0</td>
          <td>6.0</td>
        </tr>
        <tr>
          <th>16</th>
          <td>ReorderInput</td>
          <td>[{'float': [1, 64, 112, 112]}]</td>
          <td>[{'float': [1, 64, 112, 112]}]</td>
          <td>CPUExecutionProvider</td>
          <td>1</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>1533.0</td>
          <td>6.0</td>
        </tr>
        <tr>
          <th>17</th>
          <td>ReorderInput</td>
          <td>[{'float': [1, 64, 112, 112]}]</td>
          <td>[{'float': [1, 64, 112, 112]}]</td>
          <td>CPUExecutionProvider</td>
          <td>2</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>1280.0</td>
          <td>6.0</td>
        </tr>
        <tr>
          <th>18</th>
          <td>ReorderOutput</td>
          <td>[{'float': [1, 1, 112, 112]}]</td>
          <td>[{'float': [1, 8, 112, 112]}]</td>
          <td>CPUExecutionProvider</td>
          <td>0</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>146.0</td>
          <td>6.0</td>
        </tr>
        <tr>
          <th>19</th>
          <td>ReorderOutput</td>
          <td>[{'float': [1, 64, 112, 112]}]</td>
          <td>[{'float': [1, 64, 112, 112]}]</td>
          <td>CPUExecutionProvider</td>
          <td>0</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>6803.0</td>
          <td>6.0</td>
        </tr>
        <tr>
          <th>20</th>
          <td>ReorderOutput</td>
          <td>[{'float': [1, 64, 112, 112]}]</td>
          <td>[{'float': [1, 64, 112, 112]}]</td>
          <td>CPUExecutionProvider</td>
          <td>1</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>1323.0</td>
          <td>6.0</td>
        </tr>
        <tr>
          <th>21</th>
          <td>ReorderOutput</td>
          <td>[{'float': [1, 64, 56, 56]}]</td>
          <td>[{'float': [1, 64, 56, 56]}]</td>
          <td>CPUExecutionProvider</td>
          <td>0</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>931.0</td>
          <td>6.0</td>
        </tr>
        <tr>
          <th>22</th>
          <td>Sigmoid</td>
          <td>[{'float': [1, 1, 112, 112]}]</td>
          <td>[{'float': [1, 1, 112, 112]}]</td>
          <td>CPUExecutionProvider</td>
          <td>0</td>
          <td>310.0</td>
          <td>6.0</td>
          <td>NaN</td>
          <td>NaN</td>
        </tr>
      </tbody>
    </table>
    </div>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 91-92

More detailed

.. GENERATED FROM PYTHON SOURCE LINES 92-96

.. code-block:: Python


    gr.to_excel(f"plot_profiling_merged_details{suffix}.xlsx", index=False)
    gr






.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>durbase</th>
          <th>duropti</th>
          <th>countbase</th>
          <th>countopti</th>
        </tr>
        <tr>
          <th>label</th>
          <th></th>
          <th></th>
          <th></th>
          <th></th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>[+CPU]Conv(f-1x2x112x112,f-8x2x7x7)-&gt;f-1x8x112x112</th>
          <td>0.0</td>
          <td>19992.0</td>
          <td>0.0</td>
          <td>6.0</td>
        </tr>
        <tr>
          <th>[+CPU]Conv(f-1x64x112x112,f-64x64x3x3,f-64,f-1x64x56x56)-&gt;f-1x64x56x56</th>
          <td>0.0</td>
          <td>53992.0</td>
          <td>0.0</td>
          <td>6.0</td>
        </tr>
        <tr>
          <th>[+CPU]ReorderInput(f-1x64x112x112)-&gt;f-1x64x112x112</th>
          <td>0.0</td>
          <td>9092.0</td>
          <td>0.0</td>
          <td>18.0</td>
        </tr>
        <tr>
          <th>[+CPU]ReorderOutput(f-1x64x112x112)-&gt;f-1x64x112x112</th>
          <td>0.0</td>
          <td>8126.0</td>
          <td>0.0</td>
          <td>12.0</td>
        </tr>
        <tr>
          <th>[+CPU]ReorderOutput(f-1x64x56x56)-&gt;f-1x64x56x56</th>
          <td>0.0</td>
          <td>931.0</td>
          <td>0.0</td>
          <td>6.0</td>
        </tr>
        <tr>
          <th>[+CPU]ReorderOutput(f-1x8x112x112)-&gt;f-1x1x112x112</th>
          <td>0.0</td>
          <td>146.0</td>
          <td>0.0</td>
          <td>6.0</td>
        </tr>
        <tr>
          <th>[-CPU]Add(f-1x64x56x56,f-1x64x56x56)-&gt;f-1x64x56x56</th>
          <td>2420.0</td>
          <td>0.0</td>
          <td>6.0</td>
          <td>0.0</td>
        </tr>
        <tr>
          <th>[-CPU]Conv(f-1x2x112x112,f-1x2x7x7)-&gt;f-1x1x112x112</th>
          <td>2497.0</td>
          <td>0.0</td>
          <td>6.0</td>
          <td>0.0</td>
        </tr>
        <tr>
          <th>[-CPU]Conv(f-1x64x112x112,f-64x64x3x3,f-64)-&gt;f-1x64x56x56</th>
          <td>59088.0</td>
          <td>0.0</td>
          <td>6.0</td>
          <td>0.0</td>
        </tr>
        <tr>
          <th>[-CPU]Sigmoid(f-1x1x112x112)-&gt;f-1x1x112x112</th>
          <td>310.0</td>
          <td>0.0</td>
          <td>6.0</td>
          <td>0.0</td>
        </tr>
        <tr>
          <th>[=CPU]BatchNormalization(f-1x64x112x112,f-64,f-64,f-64,f-64)-&gt;f-1x64x112x112</th>
          <td>2548.0</td>
          <td>8064.0</td>
          <td>6.0</td>
          <td>6.0</td>
        </tr>
        <tr>
          <th>[=CPU]Concat(f-1x1x112x112,f-1x1x112x112)-&gt;f-1x2x112x112</th>
          <td>166.0</td>
          <td>127.0</td>
          <td>6.0</td>
          <td>6.0</td>
        </tr>
        <tr>
          <th>[=CPU]Conv(f-1x3x112x112,f-64x3x3x3,f-64)-&gt;f-1x64x112x112</th>
          <td>5364.0</td>
          <td>4311.0</td>
          <td>6.0</td>
          <td>6.0</td>
        </tr>
        <tr>
          <th>[=CPU]Conv(f-1x64x112x112,f-64x64x1x1,f-64)-&gt;f-1x64x56x56</th>
          <td>9524.0</td>
          <td>1767.0</td>
          <td>6.0</td>
          <td>6.0</td>
        </tr>
        <tr>
          <th>[=CPU]Conv(f-1x64x112x112,f-64x64x3x3,f-64)-&gt;f-1x64x112x112</th>
          <td>101656.0</td>
          <td>102240.0</td>
          <td>6.0</td>
          <td>6.0</td>
        </tr>
        <tr>
          <th>[=CPU]Mul(f-1x1x112x112,f-1x64x112x112)-&gt;f-1x64x112x112</th>
          <td>2140.0</td>
          <td>5791.0</td>
          <td>6.0</td>
          <td>6.0</td>
        </tr>
        <tr>
          <th>[=CPU]PRelu(f-1x64x112x112,f-64x1x1)-&gt;f-1x64x112x112</th>
          <td>14711.0</td>
          <td>3611.0</td>
          <td>12.0</td>
          <td>12.0</td>
        </tr>
        <tr>
          <th>[=CPU]ReduceMax(f-1x64x112x112)-&gt;f-1x1x112x112</th>
          <td>13900.0</td>
          <td>23211.0</td>
          <td>6.0</td>
          <td>6.0</td>
        </tr>
        <tr>
          <th>[=CPU]ReduceMean(f-1x64x112x112)-&gt;f-1x1x112x112</th>
          <td>20925.0</td>
          <td>5553.0</td>
          <td>6.0</td>
          <td>6.0</td>
        </tr>
      </tbody>
    </table>
    </div>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 97-99

Final plot
++++++++++

.. GENERATED FROM PYTHON SOURCE LINES 99-117

.. code-block:: Python


    # let's filter out unsignificant operator.
    grmax = gr["durbase"] + gr["duropti"]
    total = grmax.sum()
    grmax /= total
    gr = gr[grmax >= 0.01]


    fig, ax = plt.subplots(1, 2, figsize=(14, min(gr.shape[0], 500)), sharey=True)
    gr[["durbase", "duropti"]].plot.barh(ax=ax[0])
    ax[0].set_title("Side by side duration")
    gr = gr.copy()
    gr[["countbase", "countopti"]].plot.barh(ax=ax[1])
    ax[1].set_title("Side by side count")
    fig.tight_layout()
    fig.savefig(f"plot_profiling_side_by_side{suffix}.png")





.. image-sg:: /auto_examples/images/sphx_glr_plot_profiling_002.png
   :alt: Side by side duration, Side by side count
   :srcset: /auto_examples/images/sphx_glr_plot_profiling_002.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 118-120

On CUDA
+++++++

.. GENERATED FROM PYTHON SOURCE LINES 120-172

.. code-block:: Python



    if "CUDAExecutionProvider" in get_available_providers():
        print("Profiling on CUDA")
        prof_base = ort_profile(
            filename,
            feeds,
            repeat=6,
            disable_optimization=True,
            providers=["CUDAExecutionProvider"],
        )
        prof_base.to_excel(f"prof_cuda_base{suffix}.xlsx", index=False)

        prof_opti = ort_profile(
            optimized,
            feeds,
            repeat=6,
            disable_optimization=True,
            providers=["CUDAExecutionProvider", "CPUExecutionProvider"],
        )
        prof_opti.to_excel(f"prof_cuda_opti{suffix}.xlsx", index=False)

        unique_op = set(prof_base["args_op_name"])
        fig, ax = plt.subplots(2, 2, figsize=(10, len(unique_op)), sharex="col")
        plot_ort_profile(prof_base, ax[0, 0], ax[0, 1], title="baseline")
        plot_ort_profile(prof_opti, ax[1, 0], ax[1, 1], title="optimized")
        fig.tight_layout()
        fig.savefig(f"plot_profiling_cuda{suffix}.png")

        merge, gr = merge_ort_profile(prof_base, prof_opti)
        merge.to_excel(f"plot_profiling_merged{suffix}.xlsx", index=False)
        gr.to_excel(f"plot_profiling_merged_details{suffix}.xlsx", index=False)

        grmax = gr["durbase"] + gr["duropti"]
        total = grmax.sum()
        grmax /= total
        gr = gr[grmax >= 0.01]

        fig, ax = plt.subplots(1, 2, figsize=(14, min(gr.shape[0], 500)), sharey=True)
        gr[["durbase", "duropti"]].plot.barh(ax=ax[0])
        ax[0].set_title("Side by side duration")
        gr = gr.copy()
        gr[["countbase", "countopti"]].plot.barh(ax=ax[1])
        ax[1].set_title("Side by side count")
        fig.tight_layout()
        fig.savefig(f"plot_profiling_side_by_side_cuda{suffix}.png")

    else:
        print(f"CUDA not available in {get_available_providers()}.")
        fig, ax = None, None

    ax



.. rst-class:: sphx-glr-horizontal


    *

      .. image-sg:: /auto_examples/images/sphx_glr_plot_profiling_003.png
         :alt: baseline, n occurences, optimized, n occurences
         :srcset: /auto_examples/images/sphx_glr_plot_profiling_003.png
         :class: sphx-glr-multi-img

    *

      .. image-sg:: /auto_examples/images/sphx_glr_plot_profiling_004.png
         :alt: Side by side duration, Side by side count
         :srcset: /auto_examples/images/sphx_glr_plot_profiling_004.png
         :class: sphx-glr-multi-img


.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    Profiling on CUDA
    /home/xadupre/github/onnx-array-api/onnx_array_api/ort/ort_profile.py:260: FutureWarning: The provided callable <function sum at 0x7fc0d2399d80> is currently using SeriesGroupBy.sum. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string "sum" instead.
      .agg(
    /home/xadupre/github/onnx-array-api/onnx_array_api/ort/ort_profile.py:260: FutureWarning: The provided callable <function sum at 0x7fc0d2399d80> is currently using SeriesGroupBy.sum. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string "sum" instead.
      .agg(

    array([<Axes: title={'center': 'Side by side duration'}, ylabel='label'>,
           <Axes: title={'center': 'Side by side count'}, ylabel='label'>],
          dtype=object)




.. rst-class:: sphx-glr-timing

   **Total running time of the script:** (0 minutes 6.359 seconds)


.. _sphx_glr_download_auto_examples_plot_profiling.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: plot_profiling.ipynb <plot_profiling.ipynb>`

    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: plot_profiling.py <plot_profiling.py>`

    .. container:: sphx-glr-download sphx-glr-download-zip

      :download:`Download zipped: plot_profiling.zip <plot_profiling.zip>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_