Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to content
This repository was archived by the owner on Mar 6, 2026. It is now read-only.

feat: resource tags in dataset#2090

Merged
Linchin merged 24 commits intogoogleapis:mainfrom
keunsoopark:feat/resource_tags_in_dataset
Jan 14, 2025
Merged

feat: resource tags in dataset#2090
Linchin merged 24 commits intogoogleapis:mainfrom
keunsoopark:feat/resource_tags_in_dataset

Conversation

@keunsoopark
Copy link
Contributor

@keunsoopark keunsoopark commented Dec 19, 2024

  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)

Fixes #2091 🦕

@keunsoopark keunsoopark requested a review from a team December 19, 2024 13:26
@keunsoopark keunsoopark requested a review from a team as a code owner December 19, 2024 13:26
@product-auto-label product-auto-label bot added the size: m Pull request size is medium. label Dec 19, 2024
@product-auto-label product-auto-label bot added the api: bigquery Issues related to the googleapis/python-bigquery API. label Dec 19, 2024
@keunsoopark keunsoopark marked this pull request as draft December 19, 2024 13:50
@keunsoopark keunsoopark marked this pull request as ready for review December 19, 2024 13:50
@keunsoopark keunsoopark marked this pull request as draft December 19, 2024 15:38
@keunsoopark keunsoopark marked this pull request as ready for review December 19, 2024 15:38
Copy link
Contributor

@Linchin Linchin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you so much for adding resource tags in dataset! The PR looks great overall, I just made some minor modifications. I wonder if you could add a system test, as well as a unit test for setter with None? Please let us know if you need any help. Happy holidays!

@Linchin Linchin added kokoro:run Add this label to force Kokoro to re-run the tests. kokoro:force-run Add this label to force Kokoro to re-run the tests. labels Dec 20, 2024
@Linchin Linchin assigned Linchin and unassigned yirutang Dec 20, 2024
@yoshi-kokoro yoshi-kokoro removed kokoro:run Add this label to force Kokoro to re-run the tests. kokoro:force-run Add this label to force Kokoro to re-run the tests. labels Dec 20, 2024
keunsoopark and others added 5 commits December 23, 2024 09:54
@keunsoopark
Copy link
Contributor Author

system test

Hi @Linchin, thanks for the review.

I added system tests in system/test_clients.py

I fixed unittests for None, and added more tests for invalid values. See test_resource_tags_setter_bad_value.

Merry Christmas and happy new year!

@Linchin Linchin added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Jan 13, 2025
@yoshi-kokoro yoshi-kokoro removed kokoro:run Add this label to force Kokoro to re-run the tests. kokoro:force-run Add this label to force Kokoro to re-run the tests. labels Jan 13, 2025
@Linchin
Copy link
Contributor

Linchin commented Jan 13, 2025

The 3.12 system test is still failing - after trying to create the resource tag and finding that it already exists, the code tries to access it and gets PermissionDenied error, which indicates that the tag either doesn't exist, or the client doesn't have the permission to access it. I suspect there is a race condition where the tag was created by sys test 3.8 when 3.12 was trying to create it, and deleted when 3.12 tries to access it 😂 Let me rerun the tests to see if it's flaky, so we can verify the assumption.

If this is indeed the case, we may need to append the python version to the tags to avoid conflict.

@Linchin Linchin added kokoro:run Add this label to force Kokoro to re-run the tests. kokoro:force-run Add this label to force Kokoro to re-run the tests. labels Jan 13, 2025
@yoshi-kokoro yoshi-kokoro removed kokoro:run Add this label to force Kokoro to re-run the tests. kokoro:force-run Add this label to force Kokoro to re-run the tests. labels Jan 13, 2025
@Linchin Linchin added kokoro:run Add this label to force Kokoro to re-run the tests. kokoro:force-run Add this label to force Kokoro to re-run the tests. labels Jan 14, 2025
@yoshi-kokoro yoshi-kokoro removed kokoro:run Add this label to force Kokoro to re-run the tests. kokoro:force-run Add this label to force Kokoro to re-run the tests. labels Jan 14, 2025
@keunsoopark
Copy link
Contributor Author

The 3.12 system test is still failing - after trying to create the resource tag and finding that it already exists, the code tries to access it and gets PermissionDenied error, which indicates that the tag either doesn't exist, or the client doesn't have the permission to access it. I suspect there is a race condition where the tag was created by sys test 3.8 when 3.12 was trying to create it, and deleted when 3.12 tries to access it 😂 Let me rerun the tests to see if it's flaky, so we can verify the assumption.

If this is indeed the case, we may need to append the python version to the tags to avoid conflict.

Thanks for a good inspiration! I agree that we should make each of test running independent. Instead of postfixing python version, I postfixed random characters to tag keys. I see it works well from my local testing.

In previous run, all tests failed all of sudden, and I could no see logs any longer. Could you check if it happens again in next CI tests?

@Linchin
Copy link
Contributor

Linchin commented Jan 14, 2025

Thank you for the prompt reply! Adding a prefix lgtm. I think there was something wrong with the test infra, I hope it's just flaky and doesn't happen again.

@Linchin Linchin added kokoro:run Add this label to force Kokoro to re-run the tests. kokoro:force-run Add this label to force Kokoro to re-run the tests. labels Jan 14, 2025
@yoshi-kokoro yoshi-kokoro removed kokoro:run Add this label to force Kokoro to re-run the tests. kokoro:force-run Add this label to force Kokoro to re-run the tests. labels Jan 14, 2025
@Linchin Linchin added the kokoro:run Add this label to force Kokoro to re-run the tests. label Jan 14, 2025
Copy link
Contributor

@Linchin Linchin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for your contribution, now all tests have passed! 🎉

LGTM.

@keunsoopark
Copy link
Contributor Author

keunsoopark commented Jan 14, 2025

Thank you for your contribution, now all tests have passed! 🎉

LGTM.

Thank you so much! It was a pleasant review process with you, @Linchin :)
My motivation to this contribution is for making dbt able to attach resource tags - dbt-labs/dbt-adapters#577. As you see in this issue, it requires this functionality on tables as well.
I am looking forward to other PR - #2093 done soon, but if it takes too long time to get response, I can contribute it as well. Now I can fix this feature pretty quickly :)
Just ping me if you want me to be involved.

@Linchin
Copy link
Contributor

Linchin commented Jan 14, 2025

It was a pleasure working with you! @keunsoopark

Let's first see if the original author of #2093 responses, then we can decide what to do next.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

api: bigquery Issues related to the googleapis/python-bigquery API. size: m Pull request size is medium.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Attach resource tags on datasets

4 participants