Skip to content

fix(bigquery): Add retry predicates for BigQuery dataset IAM operations#26429

Draft
ElliotSwart wants to merge 1 commit intohashicorp:mainfrom
ElliotSwart:fix/bigquery-dataset-iam-retry-predicates
Draft

fix(bigquery): Add retry predicates for BigQuery dataset IAM operations#26429
ElliotSwart wants to merge 1 commit intohashicorp:mainfrom
ElliotSwart:fix/bigquery-dataset-iam-retry-predicates

Conversation

@ElliotSwart
Copy link
Copy Markdown

Summary

  • Adds IamServiceAccountNotFound and IsBigqueryIAMQuotaError retry predicates to BigqueryDatasetIamUpdater.SetResourceIamPolicy
  • Fixes transient 400 errors when service accounts referenced in BigQuery dataset IAM policies haven't fully propagated
  • Fixes transient 403 quota errors during bulk BigQuery IAM operations

Problem

SetResourceIamPolicy in iam_bigquery_dataset.go calls SendRequest without any ErrorRetryPredicates. This means:

  1. When a google_service_account and a google_bigquery_dataset_iam_policy referencing that SA are created in the same terraform apply, the IAM operation can fail with Error 400: Service account <SA> does not exist if the SA hasn't fully propagated yet.

  2. During bulk BigQuery IAM operations, Error 403: exceeded rate limits errors are not retried.

Both predicates already exist in error_retry_predicates.go and are used by other IAM resource implementations, but were not connected to BigQuery dataset IAM operations.

Fix

Pass the two existing predicates to SendRequest in SetResourceIamPolicy:

ErrorRetryPredicates: []transport_tpg.RetryErrorPredicateFunc{
    transport_tpg.IamServiceAccountNotFound,
    transport_tpg.IsBigqueryIAMQuotaError,
},

This follows the same pattern used by other IAM implementations in the provider (e.g., Cloud Run IAM).

Test plan

  • Run provider acceptance tests for google_bigquery_dataset_iam_policy
  • Run provider acceptance tests for google_bigquery_dataset_iam_binding
  • Run provider acceptance tests for google_bigquery_dataset_iam_member
  • Verify retry behavior with debug logging when SA propagation delay occurs

References

  • IamServiceAccountNotFound predicate: google/transport/error_retry_predicates.go:470
  • IsBigqueryIAMQuotaError predicate: google/transport/error_retry_predicates.go:304
  • Source file: mmv1/third_party/terraform/services/bigquery/iam_bigquery_dataset.go

🤖 Generated with Claude Code

BigQuery dataset IAM operations (SetResourceIamPolicy) currently do not
pass any ErrorRetryPredicates to SendRequest, causing transient errors
from service account propagation delays to fail immediately instead of
being retried.

This adds two existing retry predicates:
- IamServiceAccountNotFound: retries 400 errors when a service account
  referenced in an IAM policy does not yet exist (propagation delay)
- IsBigqueryIAMQuotaError: retries 403 errors from BigQuery IAM quota
  limits

This follows the same pattern used by other IAM resource implementations
in the provider and addresses errors observed when service accounts and
BigQuery dataset IAM policies are created in the same Terraform apply.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@github-actions
Copy link
Copy Markdown

This repository is generated by https://github.com/GoogleCloudPlatform/magic-modules. Any changes made directly to this repository will likely be overwritten. If you have further questions, please feel free to ping your reviewer or, internal employees, reach out to one of the engineers. Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant