It appears that for the TeXLive 2014 distribution, the text color is not respected in output from the pgf backend and is only rendered in black. TeXLive 2013 does not exhibit this behavior for any versions of Matplotlib (1.3.1 through 1.4.2). The question is whether this is a bug in the PGF code in the TexLive distribution, or whether there was a legitimate change that requires an update to the pgf backend.

Following is a MWE that exhibits the behavior. All that is required is to switch your TeXLive distribution.

import matplotlib as mpl
import matplotlib.pyplot as plt
from matplotlib.backends.backend_pgf import FigureCanvasPgf
mpl.backend_bases.register_backend('pdf', FigureCanvasPgf)

plt.rcParams['font.family'] = 'Arial'
plt.rcParams['font.size'] = 18
plt.rcParams['pgf.rcfonts'] = False

plt.plot(range(10))
plt.text(3, 6, 'Text', color='red')
plt.xlabel('x axis label', color='blue')
plt.ylabel('y axis label', color='green')

plt.savefig('untitled.pdf')