SVG savefig + LaTeX extremely slow on macOS

When I saving a SVG figure is slowed down significantly (10-100x) when I use LaTeX, enabled by the following rc-setting:

```python
text.usetex : true
```

Note that rendering is somewhat slower, but not much. It is really the saving. I am on macOS 10.12.6 with matplotlib (2.1.0).