Fix NaN gradient of tf.math.igamma at a=1, x=0 - #123580
copybara-service[bot] merged 1 commit into
Conversation
There was a problem hiding this comment.
Code Review
This pull request replaces (a - 1) * log(x) with xlogy(a - 1, x) in the gradient computation of igamma (_IgammaGrad) to prevent NaN values when a == 1 and x == 0, and adds corresponding unit tests. The reviewer feedback suggests applying the same xlogy fix to the complementary incomplete gamma function gradient (_IgammacGrad) and updating the new unit tests to run in both graph and eager modes using tf.GradientTape instead of being restricted to V1 graph mode.
| partial_x = math_ops.exp(-x + math_ops.xlogy(a - 1, x) - | ||
| math_ops.lgamma(a)) |
There was a problem hiding this comment.
The complementary incomplete gamma function gradient _IgammacGrad (defined in this file) shares the same formula and NaN gradient issue at a=1, x=0. Applying the same xlogy fix to _IgammacGrad and adding corresponding test coverage for igammac will prevent similar NaN gradient issues for tf.math.igammac.
There was a problem hiding this comment.
_IgammacGrad doesn't have its own formula it calls _IgammaGrad and negates the result (return (-igamma_grad_a, -igamma_grad_x)), so the xlogy fix already propagates to tf.math.igammac. I confirmed igammac's x-gradient at a=1, x=0 goes from NaN to the correct -1.0, and added a regression test (testIgammacGradXAtZero).
| def _igamma_x_grad(self, a, x): | ||
| return self.evaluate(gradients.gradients(math_ops.igamma(a, x), x)[0]) |
There was a problem hiding this comment.
The tests currently only run in V1 graph mode due to @test_util.run_deprecated_v1 and the use of gradients.gradients. Using @test_util.run_in_graph_and_eager_modes along with tf.GradientTape allows testing the gradient in both graph and eager modes, which provides better coverage for TensorFlow 2.x execution paths.
| def _igamma_x_grad(self, a, x): | |
| return self.evaluate(gradients.gradients(math_ops.igamma(a, x), x)[0]) | |
| def _igamma_x_grad(self, a, x): | |
| from tensorflow.python.eager import backprop | |
| with backprop.GradientTape() as tape: | |
| tape.watch(x) | |
| y = math_ops.igamma(a, x) | |
| return self.evaluate(tape.gradient(y, x)) |
References
- Verify that unit tests are included for any new logic, feature, or bug fix and cover edge cases across supported environments. (link)
There was a problem hiding this comment.
Good call rewrote the tests to use tf.GradientTape under @test_util.run_in_graph_and_eager_modes so both execution paths are exercised.
c02d2e5 to
de6e1f6
Compare
dmiltr3
left a comment
There was a problem hiding this comment.
Thank you for this contribution. The fix for the NaN gradient in tf.math.igamma at
Analysis
The original formulation in _IgammaGrad computed partial_x = exp(-x + (a - 1) * log(x) - lgamma(a)).
At (a - 1) * log(x) evaluates to 0 * -inf = NaN, despite the true mathematical derivative at that point being
Replacing (a - 1) * log(x) with xlogy(a - 1, x) robustly handles this edge case because xlogy natively maps
Regimes Verification
-
$a > 1, x = 0$ :xlogy(a-1, 0)evaluates to$-\infty$ (since$a-1 > 0$ ). The exponent becomes$-\infty$ , and$e^{-\infty} = 0$ . This is correct as$x^{a-1} = 0$ for$a > 1$ . -
$a < 1, x = 0$ :xlogy(a-1, 0)evaluates to$+\infty$ (since$a-1 < 0$ ). The exponent becomes$+\infty$ , and$e^{+\infty} = +\infty$ . This is correct as$x^{a-1} = \infty$ for$a < 1$ . -
$x > 0$ :xlogy(a-1, x)is identical to$(a-1)\log(x)$ .
Unit Tests
We verified this behavior with the following unit test:
import tensorflow as tf
from absl.testing import parameterized
class IgammaGradTest(tf.test.TestCase, parameterized.TestCase):
@parameterized.parameters(tf.float32, tf.float64)
def testNaNEdgeCase(self, dtype):
a = tf.constant(1.0, dtype=dtype)
x = tf.constant(0.0, dtype=dtype)
with tf.GradientTape() as tape:
tape.watch(x)
y = tf.math.igamma(a, x)
grad = tape.gradient(y, x)
self.assertAllClose(grad, 1.0)This ensures both single and double precision backends behave consistently.
Fixes #123578
_IgammaGrad computed partial_x = exp(-x + (a-1)*log(x) - lgamma(a)).
At a==1, x==0 the term (a-1)*log(x) is 0 * -inf = NaN, so the gradient of tf.math.igamma w.r.t. x is NaN there. The true derivative is d/dx igamma(1, x) = e^-x = 1. Use xlogy(a-1, x) instead, matching _BetaincGrad which already does this. Other regimes unchanged (a>1 -> 0, a<1 -> +inf, nonzero x identical). Adds IgammaGradTest.
Reproduced on tf-nightly 2.22.0-dev20260718.