Mathtext \left and \right with other delimiters (\Vert)

I'm getting an issue similar to https://github.com/matplotlib/matplotlib/pull/715, except that delimiters such as `(`, `[` and `\{` work, but not others, particularly `\Vert`. Something like 

``` python
ax.set_xlabel(r'$\left\Vert A \right\Vert$')
```

will crash with the same exception: `Expected a delimiter (at char 5), (line:1, col:6)`

Matplotlib 1.4.2 from debian sid repos
