* Add tests for eager task retry

* Fixes #6135

If autoretry_for is set too broad on Exception, then autoretry may get a Retry
if that's the case, rethrow directly instead of wrapping it in another Retry
to avoid loosing new args

* Add state to KeyValueStoreBackend.set method

This way, a backend implementation is able to take decisions based on
current state to store meta in case of failures.

* Add retry on recoverable exception for the backend

acks.late makes celery acknowledge messages only after processing and
storing result on the backend.

However, in case of backend unreachable, it will shadow a Retry
exception and put the task as failed in the backend not retrying the
task and acknoledging it on the broker.

With this new result_backend_always_retry setting, if the backend
exception is recoverable (to be defined per backend implementation),
it will retry the backend operation with an exponential backoff.

* Make elasticsearch backward compatible with 6.x

* Make ES retry storing updates in a better way

if existing value in the backend is success, then do nothing.
if it is a ready status, then update it only if new value is a ready status as well.
else update it.

This way, a SUCCESS cannot be overriden so that we do not loose
results but any ready state other than success (FAILURE, REVOKED) can
be overriden by another ready status (i.e. a SUCCESS)

* Add test for value not found in ES backend

random.randrange should be called with the actual so that all numbers
have equivalent probability, otherwise maximum value does have a way
higher probability of occuring.

#6142)

* fix #6136. celery 4.4.3 always trying create /var/run/celery directory, even if it's not needed.

* fix #6136. cleanup

* Add internal_error signal

There is no special signal for an out of body error which can be the
result of a bad result backend.

* Fix syntax error.

* Document the task_internal_error signal.

Co-authored-by: Laurentiu Dragan <ldragan@bloomberg.net>