bigquery: allow callers to configure query job polling retries #20121

Is your feature request related to a problem? Please describe.

Job.waitForQuery polls jobs.getQueryResults while waiting for query jobs to complete. Polling errors currently retry internally using jobRetryReasons, which includes jobRateLimitExceeded and internalError.

Under high query fanout, BigQuery can return jobRateLimitExceeded during polling. Because the client retries that internally, callers cannot observe the backpressure, log it, slow down, cancel cleanly, or fail fast. Higher-level systems can appear to hang silently until their context expires.

Describe the solution you'd like

Add an API-compatible option that lets callers configure query job polling retry behavior without changing the default behavior.

For example:

client, err := bigquery.NewClient(ctx, projectID,
    bigquery.WithJobPollingRetryPolicy(bigquery.JobPollingRetryPolicy{
        DisableRetries: true,
    }),
)

The option should apply narrowly to query job polling via jobs.getQueryResults, not to job creation retries for jobs.insert or jobs.query.

Ideally it should allow callers to:

• keep the current default behavior
• disable retries for polling errors
• optionally customize polling retry reasons/backoff

Describe alternatives you've considered

One alternative is to remove jobRateLimitExceeded from the default polling retry reasons. That would surface backpressure immediately, but it would also change behavior for existing callers.

Another alternative is to set shorter context deadlines. That does not solve the problem because the SDK still hides the real polling error until the deadline expires.

A third option is for callers to avoid Job.Read / query polling and manually poll job status. That is more work for callers and does not help code paths that rely on the existing BigQuery client abstractions.

Additional context

Related history:

• bigquery: add jobRateLimitExceeded as retryable error #3795 requested adding jobRateLimitExceeded as a retryable job error.
• fix(bigquery): add jobRateLimitExceeded in jobRetryReasons #9726 added jobRateLimitExceeded to jobRetryReasons.
• fix(bigquery): reduce default backoffs #10558 reduced default job polling backoff.
• bigquery: improve retrying job experience #5518 is a broader BigQuery job retry improvement issue.
• fix(bigquery): don't retry jobRateLimitExceeded/internalError while polling query completion [AI] #20070 proposes changing the default polling retry reason set.