Skip to content

Importing plotly takes 1.6s and 2046924 function calls #419

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
max-sixty opened this issue Mar 9, 2016 · 5 comments
Closed

Importing plotly takes 1.6s and 2046924 function calls #419

max-sixty opened this issue Mar 9, 2016 · 5 comments

Comments

@max-sixty
Copy link

Is this consistent with expectations? It looks like it's doing a lot of string parsing in Python (JSON?) and SSL connections, even though we're only using it for offline use?

Thanks for the wonderful library!

#%prun import plotly

         2046924 function calls (1057455 primitive calls) in 1.615 seconds

   Ordered by: internal time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
1142268/196040    0.476    0.000    0.566    0.000 encoder.py:341(_iterencode_dict)
       10    0.136    0.014    0.136    0.014 {method 'read' of '_ssl._SSLSocket' objects}
   196040    0.076    0.000    0.642    0.000 encoder.py:417(_iterencode)
214729/214727    0.068    0.000    0.070    0.000 {isinstance}
       12    0.041    0.003    0.689    0.057 encoder.py:186(encode)
44754/6522    0.037    0.000    0.050    0.000 utils.py:355(node_generator)
      260    0.030    0.000    0.088    0.000 doccer.py:12(docformat)
    74312    0.025    0.000    0.025    0.000 {_json.encode_basestring_ascii}
   2622/1    0.025    0.000    0.039    0.039 utils.py:301(decode_unicode)
       15    0.023    0.002    0.023    0.002 decoder.py:370(raw_decode)
       62    0.021    0.000    0.028    0.000 collections.py:293(namedtuple)
        1    0.017    0.017    0.017    0.017 index.py:3590(_add_accessors)
    16488    0.016    0.000    0.024    0.000 encoder.py:288(_iterencode_list)
     6823    0.014    0.000    0.014    0.000 {method 'expandtabs' of 'str' objects}
     3769    0.014    0.000    0.014    0.000 {method 'reduce' of 'numpy.ufunc' objects}
      260    0.013    0.000    0.022    0.000 doccer.py:128(indentcount_lines)
        1    0.012    0.012    0.031    0.031 graph_reference.py:379(_get_objects)
        1    0.012    0.012    0.030    0.030 graph_reference.py:458(_get_arrays)
       27    0.012    0.000    0.113    0.004 __init__.py:1(<module>)
    62830    0.012    0.000    0.012    0.000 {method 'append' of 'list' objects}
        1    0.011    0.011    0.028    0.028 graph_reference.py:430(_patch_objects)
        3    0.011    0.004    0.045    0.015 machar.py:116(_do_init)
7729/7712    0.010    0.000    0.014    0.000 {method 'join' of 'str' objects}
@chriddyp
Copy link
Member

chriddyp commented Mar 9, 2016

The main reason this takes so long is because we test imports on pandas, numpy, and a few others on import, and pandas itself takes about ~1 second to load!

This makes the some downstream code paths a bit easier, but still we should really only do these imports when we need to. The 1.7 second import lag is rough!!

@max-sixty
Copy link
Author

Thanks for the speedy response @chriddyp

The main reason this takes so long is because we test imports on pandas, numpy, and a few others on import, and pandas itself takes about ~1 second to load!

That's not what I'm seeing - if I import pandas first (which also imports numpy) I still get a 1.5s load time, and that still looks to be made up of string parsing + some external connections (curious why there are external connections on import?)

import pandas

%prun import plotly

         2052914 function calls (1061125 primitive calls) in 1.452 seconds

   Ordered by: internal time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
1142268/196040    0.491    0.000    0.573    0.000 encoder.py:332(_iterencode_dict)
   196040    0.079    0.000    0.651    0.000 encoder.py:411(_iterencode)
      239    0.067    0.000    0.067    0.000 {built-in method marshal.loads}
        6    0.062    0.010    0.062    0.010 {method 'read' of '_ssl._SSLSocket' objects}
   212756    0.050    0.000    0.052    0.000 {built-in method builtins.isinstance}
       12    0.043    0.004    0.697    0.058 encoder.py:182(encode)
44754/6522    0.040    0.000    0.050    0.000 utils.py:355(node_generator)
    450/1    0.040    0.000    1.453    1.453 {built-in method builtins.exec}
      260    0.034    0.000    0.090    0.000 doccer.py:12(docformat)
    74312    0.028    0.000    0.028    0.000 {built-in method _json.encode_basestring_ascii}
   2622/1    0.025    0.000    0.035    0.035 utils.py:301(decode_unicode)
  666/627    0.016    0.000    0.215    0.000 {built-in method builtins.__build_class__}
    16488    0.016    0.000    0.023    0.000 encoder.py:276(_iterencode_list)
    47/38    0.015    0.000    0.030    0.001 {built-in method _imp.create_dynamic}
      260    0.013    0.000    0.024    0.000 doccer.py:128(indentcount_lines)
    58526    0.012    0.000    0.012    0.000 {method 'append' of 'list' objects}
       14    0.011    0.001    0.011    0.001 decoder.py:345(raw_decode)
     6753    0.011    0.000    0.011    0.000 {method 'splitlines' of 'str' objects}
    10312    0.009    0.000    0.010    0.000 {method 'join' of 'str' objects}
      573    0.009    0.000    0.020    0.000 inspect.py:2065(_signature_from_function)
        1    0.009    0.009    0.009    0.009 {built-in method _socket.getaddrinfo}

@max-sixty
Copy link
Author

max-sixty commented Nov 23, 2016 •
edited
Loading

FYI this is still an issue in the latest version of plot.ly. The latest run spent eight seconds (cumulatively, including functions called within) in get_config_file. Let me know if you have any suggestions how to fix. Thank you

@theengineear
Copy link
Contributor

I think this will be fixed when we stop:

  1. updating the plot schema on import
  2. auto-bundle graph objects instead of creating them dynamically on import.

@theengineear theengineear mentioned this issue Dec 30, 2016
Closed
@theengineear
Copy link
Contributor

Resolved in 1.13.0, at least partially so. There is still some work that gets done on load. I'm going to close this since the bulk of the problem was the network connection (which no longer happens).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@chriddyp @max-sixty @theengineear