Adding script to enable clearing datastore.#369
Conversation
CONTRIBUTING.rst
Outdated
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
6a14935 to
454f82b
Compare
|
Changes Unknown when pulling 454f82b on dhermes:clear-datastore-func into * on GoogleCloudPlatform:master*. |
Also stumbled upon other datastore gotchas (non-ancestor deletes in a transaction) and added printing to indicate what is being deleted.
454f82b to
5ad0798
Compare
|
@tseaver I rebased and 5ad0798 is the while loop fix. However, while fixing I found the other 90% of the iceberg. I tested by running ISTM at this point some of these things like
should go in our codebase. This relates to #336. WDYT? /cc @silvolu as an FYI for |
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
|
I don't know if "delete all entities returned by a query" is a frequent usecase. I do think having a way for a query to return just keys would be good, but projection queries don't seem to support a "key-only" return type. |
|
LGTM. I'd say merge now and consider how to fold useful bits into #336. |
Adding script to enable clearing datastore.
|
@tseaver What do you mean about projecting onto a key that isn't covered by using the |
|
I misspelled the |
|
Added remark to #336 with the idea. Thanks. |
…369) Source-Link: googleapis/synthtool@6b4d5a6 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:f792ee1320e03eda2d13a5281a2989f7ed8a9e50b73ef6da97fac7e1e850b149 Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
…369) Source-Link: googleapis/synthtool@6b4d5a6 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:f792ee1320e03eda2d13a5281a2989f7ed8a9e50b73ef6da97fac7e1e850b149 Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
…369) Source-Link: googleapis/synthtool@6b4d5a6 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:f792ee1320e03eda2d13a5281a2989f7ed8a9e50b73ef6da97fac7e1e850b149 Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
…369) Source-Link: googleapis/synthtool@6b4d5a6 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:f792ee1320e03eda2d13a5281a2989f7ed8a9e50b73ef6da97fac7e1e850b149 Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Source-Link: googleapis/synthtool@fdba3ed Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:1f0dbd02745fb7cf255563dab5968345989308544e52b7f460deadd5e78e63b0
…#369) Co-authored-by: Anthonios Partheniou <partheniou@google.com>
…pprove] (#369) Source-Link: https://togithub.com/googleapis/synthtool/commit/e3a1277ac35fc88c09db1930533e24292b132ced Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:452901c74a22f9b9a3bd02bce780b8e8805c97270d424684bff809ce5be8c2a2
- [ ] Regenerate this pull request now. PiperOrigin-RevId: 472772457 Source-Link: googleapis/googleapis@855b74d Source-Link: googleapis/googleapis-gen@b64b1e7 Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiYjY0YjFlN2RhM2UxMzhmMTVjYTM2MTU1MmVmMDU0NWU1NDg5MWI0ZiJ9
Source-Link: googleapis/synthtool@1b9ad76 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:9db98b055a7f8bd82351238ccaacfd3cda58cdf73012ab58b8da146368330021
Source-Link: https://togithub.com/googleapis/synthtool/commit/eaef28efd179e6eeb9f4e9bf697530d074a6f3b9 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:f8ca7655fa8a449cadcabcbce4054f593dcbae7aeeab34aa3fcc8b5cf7a93c9e
* docs: Operation.Type is now documented in detail docs: Operation.self_link and Operation.target_link given examples feat: Cluster resizes will now have their own operation type (RESIZE_CLUSTER) instead of reusing REPAIR_CLUSTER; they will start using this in the near future PiperOrigin-RevId: 526970355 Source-Link: googleapis/googleapis@83fd1f9 Source-Link: googleapis/googleapis-gen@5a191c3 Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiNWExOTFjM2FhMmM1MjZhNTllZWE1OTdjN2RkZGE3OTQ4MWI4ZTg1OCJ9 docs: Operation.Type is now documented in detail docs: Operation.self_link and Operation.target_link given examples feat: Cluster resizes will now have their own operation type (RESIZE_CLUSTER) instead of reusing REPAIR_CLUSTER; they will start using this in the near future PiperOrigin-RevId: 526025502 Source-Link: googleapis/googleapis@4273180 Source-Link: googleapis/googleapis-gen@772e2cd Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiNzcyZTJjZDQ4ZjU2MWUxZjBhNGQxNzJlOWY5Mjg1YWQ5ZTFjMTIxMCJ9 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * feat: support fleet registration via cluster update PiperOrigin-RevId: 529131876 Source-Link: googleapis/googleapis@d175293 Source-Link: googleapis/googleapis-gen@b680d48 Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiYjY4MGQ0OGEwZjY3NjJkOTRlMGUxNGViZmZiNzg2OGU2YzU5MzM1YiJ9 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * feat: add a new feature for fleet registration via cluster update in v1alpha1, v1beta1, v1 PiperOrigin-RevId: 529516002 Source-Link: googleapis/googleapis@5ce0665 Source-Link: googleapis/googleapis-gen@5122c36 Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiNTEyMmMzNjMwMTQ4MzJkZjIxMTNmNDc0ZjU2MTM3YjRiYzBiM2NhMyJ9 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * docs: clarified release channel defaulting behavior for create cluster requests when release channel is unspecified If release channel is left unspecified on cluster creation and a version is specified, the cluster is now enrolled in the most mature release channel where the version is available (first checking STABLE, then REGULAR, and finally RAPID) instead of being enrolled in STATIC. This only affects clusters with node auto upgrades enabled and no explicit node version specified. To ensure new clusters are enrolled in static, explicitly specify that the cluster should not be enrolled in a channel. PiperOrigin-RevId: 532773421 Source-Link: googleapis/googleapis@633d3c1 Source-Link: googleapis/googleapis-gen@f779f56 Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiZjc3OWY1NjI0NzYxOTlkMWYwMjE1NmFjMjhmOTdlYzllMzcwNjdlYiJ9 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * docs: clarified release channel defaulting behavior for create cluster requests when release channel is unspecified If release channel is left unspecified on cluster creation and a version is specified, the cluster is now enrolled in the most mature release channel where the version is available (first checking STABLE, then REGULAR, and finally RAPID) instead of being enrolled in STATIC. This only affects clusters with node auto upgrades enabled and no explicit node version specified. To ensure new clusters are enrolled in static, explicitly specify that the cluster should not be enrolled in a channel. PiperOrigin-RevId: 532773462 Source-Link: googleapis/googleapis@aeae5ea Source-Link: googleapis/googleapis-gen@d4d433d Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiZDRkNDMzZDRlYWE4YjRkNzgwZjgzMDYyYmYxYTgxZDA0NjQ2ZWE0MSJ9 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * feat: add SoleTenantConfig API PiperOrigin-RevId: 536516473 Source-Link: googleapis/googleapis@0fb1181 Source-Link: googleapis/googleapis-gen@3d69310 Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiM2Q2OTMxMDk5NDI0MmQ1YmRiOWQzNDVmNGQ4OTRkNDViNzI2MmY5ZCJ9 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * feat: add SoleTenantConfig API PiperOrigin-RevId: 536569254 Source-Link: googleapis/googleapis@3cc45f0 Source-Link: googleapis/googleapis-gen@ed8d754 Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiZWQ4ZDc1NDZjZDE3NzA1M2Q1MzEyZDBlNDBmN2Y1MDYzNTJiYTgwNSJ9 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md --------- Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> Co-authored-by: Anthonios Partheniou <partheniou@google.com>
* docs: Minor formatting chore: Update gapic-generator-python to v1.11.5 build: Update rules_python to 0.24.0 PiperOrigin-RevId: 563436317 Source-Link: googleapis/googleapis@42fd37b Source-Link: googleapis/googleapis-gen@280264c Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiMjgwMjY0Y2EwMmZiOTMxNmI0MjM3YTk2ZDBhZjFhMjM0M2E4MWE1NiJ9 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md --------- Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Source-Link: googleapis/synthtool@50db768 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:e09366bdf0fd9c8976592988390b24d53583dd9f002d476934da43725adbb978 Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
* chore: use gapic-generator-python 0.63.4 chore: fix snippet region tag format chore: fix docstring code block formatting PiperOrigin-RevId: 430730865 Source-Link: googleapis/googleapis@ea58002 Source-Link: googleapis/googleapis-gen@ca893ff Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiY2E4OTNmZjhhZjI1ZmM3ZmUwMDFkZTE0MDVhNTE3ZDgwNDQ2ZWNjYSJ9 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * remove redundant samples Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> Co-authored-by: Anthonios Partheniou <partheniou@google.com>
Source-Link: googleapis/synthtool@6ed3a83 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:3abfa0f1886adaf0b83f07cb117b24a639ea1cb9cffe56d43280b977033563eb Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
Source-Link: https://togithub.com/googleapis/synthtool/commit/0ddbff8012e47cde4462fe3f9feab01fbc4cdfd6 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:bced5ca77c4dda0fd2f5d845d4035fc3c5d3d6b81f245246a36aee114970082b
Co-authored-by: Dan Lee <71398022+dandhlee@users.noreply.github.com> Co-authored-by: Dan Lee <71398022+dandhlee@users.noreply.github.com>
…369) Source-Link: googleapis/synthtool@6b4d5a6 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:f792ee1320e03eda2d13a5281a2989f7ed8a9e50b73ef6da97fac7e1e850b149 Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Co-authored-by: Tim Swast <swast@google.com>
Source-Link: googleapis/synthtool@bef813d Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:94bb690db96e6242b2567a4860a94d48fa48696d092e51b0884a1a2c0a79a407 Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
* chore: ignore docs.yml and lint.yml in renovate This change updates renovate.json to prevent renovate-bot from updating the specified workflow files. * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * chore: exclude renovate.json from owlbot updates This ensures that owlbot does not overwrite the changes made to `renovate.json`. * Update owlbot.py * Update owlbot.py --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Bumps [jinja2](https://github.com/pallets/jinja) from 3.1.4 to 3.1.5. - [Release notes](https://github.com/pallets/jinja/releases) - [Changelog](https://github.com/pallets/jinja/blob/main/CHANGES.rst) - [Commits](pallets/jinja@3.1.4...3.1.5) --- updated-dependencies: - dependency-name: jinja2 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Holt Skinner <13262395+holtskinner@users.noreply.github.com>
* updated env tests commit * updated commit
* fix: empty Entities for optional LocalStructuredProperty fields Fixes #369
* updated env tests commit * updated commit
…369) * Removed duplicated tests * Pass mock credentials. To allow unit tests to run without connecting to the google APIs. * Used a fixture to be DRY wrt test credentials. * Used a fixture to be DRY wrt test credentials. * Used a fixture to be DRY wrt test credentials. * Document the fixture.
* chore(python): add support for python 3.11 Source-Link: googleapis/synthtool@7197a00 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:c43f1d918bcf817d337aa29ff833439494a158a0831508fda4ec75dc4c0d0320 * add python 3.11 to noxfile Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> Co-authored-by: Anthonios Partheniou <partheniou@google.com>
Bumps [idna](https://togithub.com/kjd/idna) from 3.3 to 3.7. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://togithub.com/kjd/idna/releases">idna's releases</a>.</em></p> <blockquote> <h2>v3.7</h2> <h2>What's Changed</h2> <ul> <li>Fix issue where specially crafted inputs to encode() could take exceptionally long amount of time to process. [CVE-2024-3651]</li> </ul> <p>Thanks to Guido Vranken for reporting the issue.</p> <p><strong>Full Changelog</strong>: <a href="https://togithub.com/kjd/idna/compare/v3.6...v3.7">https://togithub.com/kjd/idna/compare/v3.6...v3.7</a></p> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://togithub.com/kjd/idna/blob/master/HISTORY.rst">idna's changelog</a>.</em></p> <blockquote> <p>3.7 (2024-04-11) ++++++++++++++++</p> <ul> <li>Fix issue where specially crafted inputs to encode() could take exceptionally long amount of time to process. [CVE-2024-3651]</li> </ul> <p>Thanks to Guido Vranken for reporting the issue.</p> <p>3.6 (2023-11-25) ++++++++++++++++</p> <ul> <li>Fix regression to include tests in source distribution.</li> </ul> <p>3.5 (2023-11-24) ++++++++++++++++</p> <ul> <li>Update to Unicode 15.1.0</li> <li>String codec name is now "idna2008" as overriding the system codec "idna" was not working.</li> <li>Fix typing error for codec encoding</li> <li>"setup.cfg" has been added for this release due to some downstream lack of adherence to PEP 517. Should be removed in a future release so please prepare accordingly.</li> <li>Removed reliance on a symlink for the "idna-data" tool to comport with PEP 517 and the Python Packaging User Guide for sdist archives.</li> <li>Added security reporting protocol for project</li> </ul> <p>Thanks Jon Ribbens, Diogo Teles Sant'Anna, Wu Tingfeng for contributions to this release.</p> <p>3.4 (2022-09-14) ++++++++++++++++</p> <ul> <li>Update to Unicode 15.0.0</li> <li>Migrate to pyproject.toml for build information (PEP 621)</li> <li>Correct another instance where generic exception was raised instead of IDNAError for malformed input</li> <li>Source distribution uses zeroized file ownership for improved reproducibility</li> </ul> <p>Thanks to Seth Michael Larson for contributions to this release.</p> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://togithub.com/kjd/idna/commit/1d365e17e10d72d0b7876316fc7b9ca0eebdd38d"><code>1d365e1</code></a> Release v3.7</li> <li><a href="https://togithub.com/kjd/idna/commit/c1b3154939907fab67c5754346afaebe165ce8e6"><code>c1b3154</code></a> Merge pull request <a href="https://redirect.github.com/kjd/idna/issues/172">#172</a> from kjd/optimize-contextj</li> <li><a href="https://togithub.com/kjd/idna/commit/0394ec76ff022813e770ba1fd89658790ea35623"><code>0394ec7</code></a> Merge branch 'master' into optimize-contextj</li> <li><a href="https://togithub.com/kjd/idna/commit/cd58a23173d2b0a40b95ee680baf3e59e8d33966"><code>cd58a23</code></a> Merge pull request <a href="https://redirect.github.com/kjd/idna/issues/152">#152</a> from elliotwutingfeng/dev</li> <li><a href="https://togithub.com/kjd/idna/commit/5beb28b9dd77912c0dd656d8b0fdba3eb80222e7"><code>5beb28b</code></a> More efficient resolution of joiner contexts</li> <li><a href="https://togithub.com/kjd/idna/commit/1b121483ed04d9576a1291758f537e1318cddc8b"><code>1b12148</code></a> Update ossf/scorecard-action to v2.3.1</li> <li><a href="https://togithub.com/kjd/idna/commit/d516b874c3388047934938a500c7488d52c4e067"><code>d516b87</code></a> Update Github actions/checkout to v4</li> <li><a href="https://togithub.com/kjd/idna/commit/c095c75943413c75ebf8ac74179757031b7f80b7"><code>c095c75</code></a> Merge branch 'master' into dev</li> <li><a href="https://togithub.com/kjd/idna/commit/60a0a4cb61ec6834d74306bd8a1fa46daac94c98"><code>60a0a4c</code></a> Fix typo in GitHub Actions workflow key</li> <li><a href="https://togithub.com/kjd/idna/commit/5918a0ef8034379c2e409ae93ee11d24295bb201"><code>5918a0e</code></a> Merge branch 'master' into dev</li> <li>Additional commits viewable in <a href="https://togithub.com/kjd/idna/compare/v3.3...v3.7">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://togithub.com/googleapis/sphinx-docfx-yaml/network/alerts). </details>
I peeled this off of an ideas branch of been working with:
https://github.com/dhermes/gcloud-python/tree/better-auth-story