Skip to content

bigquery: add jobRateLimitExceeded as retryable error #3795

Closed

Description

@benjbaron

Is your feature request related to a problem? Please describe.

With the limit of 100 concurrent jobs, we have the jobRateLimitExceeded error when we have 100s of concurrent calls to client.Query(...).Read(ctx):

googleapi: Error 400: Job exceeded rate limits: Your project_and_region exceeded quota for concurrent queries. For more information, see https://cloud.google.com/bigquery/troubleshooting-errors, jobRateLimitExceeded

Describe the solution you'd like

Since the error rateLimitExceeded is already a retryable error, we could add the error jobRateLimitExceeded to the set of retryable errors since it us very similar to rateLimitExceeded and could benefit from the backoff of gax.Backoff.

Describe alternatives you've considered

Alternatively, we could implement the retry mechanism client side and wrap the BigQuery client, however having the client automatically retry on jobRateLimitExceeded would greatly simplify the approach and avoid logic duplication.

Activity

  1. changed the title [-]BigQuery Go Client: add jobRateLimitExceeded as retryable error[/-] [+]bigquery: add jobRateLimitExceeded as retryable error[/+] on Mar 11, 2021
  2. added
    type: feature request‘Nice-to-have’ improvement, new feature or different behavior or design.
    and removed
    🚨This issue needs some love.
    triage meI really want to be triaged.
    on Mar 17, 2021
  3. alvarowolfx commented on Apr 18, 2024

    @alvarowolfx
    Contributor

    fixed by #9726

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

api: bigqueryIssues related to the BigQuery API.type: feature request‘Nice-to-have’ improvement, new feature or different behavior or design.

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions