Bug report

Bug summary

When there are multiple subplots, the colorbar takes space even after successfully removed. New colorbar will have to be positioned to the left of old ones.

It can be seen in the following example that the contour plot is being shrinked in width because space is being occupied by colorbar.

Code for reproduction

import numpy as np
import matplotlib.pyplot as plt

x = np.arange(0., 5., 1.)
y = np.arange(0., 10., 1.)
z = np.tensordot(y, x, axes=0)

fig = plt.figure()
fig.add_subplot(1, 3, 1)
fig.add_subplot(1, 3, 2)
ax = fig.add_subplot(1, 3, 2)

ax.clear()
contour = ax.contourf(x, y, z)

cb = fig.colorbar(contour)
# The following should not have no effect on the final outcome if uncommented
# ax.clear()
# cb.remove()
# contour = ax.contourf(x, y, z)
# cb_new = fig.colorbar(contour)

Actual outcome
This is the result I get if the last 4 lines are NOT commented; i.e. colorbar cb is removed and a new, identical one is added.

If I do this process (remove -> readd) again, things get even worse,

Expected outcome
This is the result I get if the last 4 lines ARE commented; i.e., only 1 colorbar is ever created

Matplotlib version

• Operating system: Darwin MacBook-Pro-15.local 19.2.0 Darwin Kernel Version 19.2.0: Sat Nov 9 03:47:04 PST 2019; root:xnu-6153.61.1~20/RELEASE_X86_64 x86_64
• Matplotlib version: 3.1.1
• Matplotlib backend: nbAgg
• Python version: 3.7.3
• jupyter core : 4.6.0
• jupyter-notebook : 6.0.1
• conda: 4.7.12