This isn't broken! Our import completion correctly traverses directories which do not contain an __init__.py, since #792, because empty __init__.py files are no longer required for namespace packages.

• Maybe we shouldn't crawl these at all. Apparently unittest doesn't crawl them, a sort of similar problem. At some point in the past, IPython skipped these too.

• Maybe we should use different crawling logic for the current directory that bpython is run from than we do for other locations on sys.path: namespace packages in other locations will still be picked up, but running bpython from a filesystem path which contains a massive node_modules directory or similar won't chew up CPU. Users who manually add e.g. their home directory to sys.path would still see this issue.

We could implement a (user-configurable) skip list which is checked before traversing into directories. We could make it default to node_modules, .git, and maybe other well known directories which won't contain Python modules.

I still see a lot of directories visited

$ python -m bpython.importcompletion | wc -l
   63247

(this is a script from #943)

from all the junk I have in my home directory so I'm thinking about this again. Same ideas as before: since running bpython from my home directory is very common for me and it's the '' entry in sys.path that causes all the trouble for me, maybe this entry could be treated differently; and maybe traversing non-init.py directories could be an option to turn off, or even off by default.

Another IPython reference bug ipython/ipython#8875, they suggest Jedi for this.