Hey, I'm playing around with some interesting but unpredictable models (this one, specifically) and it displays very strong creativity and good prose, but it's also prone to breaking. So, I had the idea to do a "poor man's ensemble", and add the raw token activations of a less creative, but more consistent model with the creative one, and then apply softmax. Hopefully, this will maintain some degree of creativity while being more consistent.

I've looked through the code, but the codebase is already large enough to be impenetrable for newcomers. Does this feature already exist? If not, is there some way I can get around it? The best thing I've found are the layer by layer callbacks, but I'm not sure exactly how to use those.