tf.linalg.lu_solve returns inconsistent output from CPU vs GPU #98345
Description
Issue type
Bug
Have you reproduced the bug with TensorFlow Nightly?
Yes
Source
source
TensorFlow version
2.21.0-dev20250801
Custom code
Yes
OS platform and distribution
Linux Ubuntu 24.04
Mobile device
No response
Python version
3.12
Bazel version
No response
GCC/compiler version
No response
CUDA/cuDNN version
No response
GPU model and memory
No response
Current behavior?
When running tf.linalg.lu_solve with a float64 tesnor of large (but not overflowing) values, CPU and GPU returns wildly different values.
Also reproducible with Tensorflow 2.19.0, please take a look at the gist.
Standalone code to reproduce the issue
import tensorflow as tf
import numpy as np
print("TensorFlow version:", tf.__version__) # TensorFlow version: 2.21.0-dev20250801
rng = np.random.default_rng(957)
lower_upper = tf.constant(rng.uniform(-9223372036854772000., -9223372036854771000., size=(3, 3)), dtype=tf.float64)
perm = tf.constant(rng.uniform(0., 0., size=(3,)), dtype=tf.int64)
rhs = tf.constant(rng.uniform(-100., 100., size=(3, 3)), dtype=tf.float64)
validate_args = True
name = "constant"
with tf.device("/CPU:0"):
result = tf.linalg.lu_solve(
lower_upper=lower_upper,
perm=perm,
rhs=rhs,
validate_args=validate_args,
name=name
)
print("Result (CPU):\n", result.numpy()[0, 2])
with tf.device("/GPU:0"):
result = tf.linalg.lu_solve(
lower_upper=lower_upper,
perm=perm,
rhs=rhs,
validate_args=validate_args,
name=name
)
print("Result (GPU):\n", result.numpy()[0, 2])Relevant log output
...
TensorFlow version: 2.21.0-dev20250801
...
Result (CPU):
-0.0
Result (GPU):
-47201.82360354644