Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to content

Migration possibly reporting the wrong database size change on DB testing job

What is this about

Tl;DR; The database testing job reported a DB size change of "+8.00 Kb" for a migration that adds an index. But we added this index async during the weekend, and the index was already part of the test database. So the migration was only supposed to add the index to the schema.

The question is: why did it report adding 8Kb to the database size?

When checking the artifacts produced by the database testing job, I see that indeed no index was created by this migration. Although, there was another migration which also reported "+8.00 Kb" change, which indeed created an index. Could we have a bug in the Database size change: <%= total_size_change(migration) %>, which overrides the result of the migration with another one?

The migration which should not report the 8Kb was: AddIndexIssuesOnProjectHealthStatusDescWorkItemType.

The migration which correctly reports the 8Kb was: AddUniqueIndexForNpmPackagesOnProjectIdNameVersion.

The following discussion from gitlab-org/gitlab!124235 (merged) should be addressed:

  • @project_278964_bot_c29560c0b368f96e598f3882911366f5 started a discussion: (+5 comments)

    Database migrations (on the main database)

    Migrations included in this change have been executed on gitlab.com data for testing purposes. For details, please see the migration testing pipeline (limited access).

    Migration Type Total runtime Result DB size change
    20230620201738 - AddIndexIssuesOnProjectHealthStatusAscWorkItemType Post deploy 3.2 s ✅ +0.00 B
    20230620201817 - AddIndexIssuesOnProjectHealthStatusDescWorkItemType Post deploy 3.0 s ✅ +8.00 KiB
    Runtime Histogram for all migrations
    Query Runtime Count
    0 seconds - 0.01 seconds 0
    0.01 seconds - 0.1 seconds 4
    0.1 seconds - 1 second 0
    1 second - 5 seconds 0
    5 seconds - 15 seconds 0
    15 seconds - 5 minutes 0
    5 minutes + 0

    Migration: 20230620201738 - AddIndexIssuesOnProjectHealthStatusAscWorkItemType

    • Type: Post deploy
    • Duration: 3.2 s
    • Database size change: +0.00 B
    Calls Total Time Max Time Mean Time Rows Query
    2 0.0 ms 0.0 ms 0.0 ms 2 
    SELECT pg_backend_pid()
    Histogram for AddIndexIssuesOnProjectHealthStatusAscWorkItemType
    Query Runtime Count
    0 seconds - 0.01 seconds 0
    0.01 seconds - 0.1 seconds 2
    0.1 seconds - 1 second 0
    1 second - 5 seconds 0
    5 seconds - 15 seconds 0
    15 seconds - 5 minutes 0
    5 minutes + 0

    Migration: 20230620201817 - AddIndexIssuesOnProjectHealthStatusDescWorkItemType

    • Type: Post deploy
    • Duration: 3.0 s
    • Database size change: +8.00 KiB
    Calls Total Time Max Time Mean Time Rows Query
    2 0.0 ms 0.0 ms 0.0 ms 2 
    SELECT pg_backend_pid()
    Histogram for AddIndexIssuesOnProjectHealthStatusDescWorkItemType
    Query Runtime Count
    0 seconds - 0.01 seconds 0
    0.01 seconds - 0.1 seconds 2
    0.1 seconds - 1 second 0
    1 second - 5 seconds 0
    5 seconds - 15 seconds 0
    15 seconds - 5 minutes 0
    5 minutes + 0

    Other information

    Other migrations pending on GitLab.com
    Migration Type Total runtime Result DB size change
    20230616164309 - CreateSubscriptionUserAddOnAssignments Regular 2.4 s ⚠ +32.00 KiB
    20230616164705 - AddForeignKeyAddOnPurchaseIdOnSubscriptionUserAddOnAssignments Regular 2.1 s ✅ +0.00 B
    20230616164731 - AddForeignKeyUserIdOnSubscriptionUserAddOnAssignments Regular 1.9 s ✅ +0.00 B
    20230621065943 - AddElasticsearchRequeueWorkersToApplicationSettings Regular 1.5 s ✅ +0.00 B
    20230621070810 - UpdateRequeueWorkersInApplicationSettingsForGitlabCom Regular 1.5 s ✅ +0.00 B
    20230608145500 - AddPreparedAtIndexToMergeRequests Post deploy 2.9 s ✅ +0.00 B
    20230616082958 - AddUniqueIndexForNpmPackagesOnProjectIdNameVersion Post deploy 45.0 s ⚠ +131.03 MiB
    20230619123701 - ScheduleRemovalIndexJobArtifactsIdAndExpireAt Post deploy 2.2 s ✅ +0.00 B
    20230620134708 - ValidateUserTypeConstraint Post deploy 56.2 s ⚠ +0.00 B
    Clone details
    Clone ID Clone Created At Clone Data Timestamp Expected Removal Time
    database-testing-2057478-10440087-main 2023-06-26T17:26:16Z 2023-06-26T16:22:49Z 2023-06-27 05:32:37 +0000
    database-testing-2057478-10440087-ci 2023-06-26T17:26:16Z 2023-06-26T16:46:57Z 2023-06-27 05:32:37 +0000

    Job artifacts

    Database migrations (on the ci database)

    Migrations included in this change have been executed on gitlab.com data for testing purposes. For details, please see the migration testing pipeline (limited access).

    Migration Type Total runtime Result DB size change
    20230620201738 - AddIndexIssuesOnProjectHealthStatusAscWorkItemType Post deploy 3.7 s ✅ +0.00 B
    20230620201817 - AddIndexIssuesOnProjectHealthStatusDescWorkItemType Post deploy 3.7 s ✅ +8.00 KiB
    Runtime Histogram for all migrations
    Query Runtime Count
    0 seconds - 0.01 seconds 0
    0.01 seconds - 0.1 seconds 4
    0.1 seconds - 1 second 0
    1 second - 5 seconds 0
    5 seconds - 15 seconds 0
    15 seconds - 5 minutes 0
    5 minutes + 0

    Migration: 20230620201738 - AddIndexIssuesOnProjectHealthStatusAscWorkItemType

    • Type: Post deploy
    • Duration: 3.7 s
    • Database size change: +0.00 B
    Calls Total Time Max Time Mean Time Rows Query
    2 0.0 ms 0.0 ms 0.0 ms 2 
    SELECT pg_backend_pid()
    Histogram for AddIndexIssuesOnProjectHealthStatusAscWorkItemType
    Query Runtime Count
    0 seconds - 0.01 seconds 0
    0.01 seconds - 0.1 seconds 2
    0.1 seconds - 1 second 0
    1 second - 5 seconds 0
    5 seconds - 15 seconds 0
    15 seconds - 5 minutes 0
    5 minutes + 0

    Migration: 20230620201817 - AddIndexIssuesOnProjectHealthStatusDescWorkItemType

    • Type: Post deploy
    • Duration: 3.7 s
    • Database size change: +8.00 KiB
    Calls Total Time Max Time Mean Time Rows Query
    2 0.0 ms 0.0 ms 0.0 ms 2 
    SELECT pg_backend_pid()
    Histogram for AddIndexIssuesOnProjectHealthStatusDescWorkItemType
    Query Runtime Count
    0 seconds - 0.01 seconds 0
    0.01 seconds - 0.1 seconds 2
    0.1 seconds - 1 second 0
    1 second - 5 seconds 0
    5 seconds - 15 seconds 0
    15 seconds - 5 minutes 0
    5 minutes + 0

    Other information

    Other migrations pending on GitLab.com
    Migration Type Total runtime Result DB size change
    20230616164309 - CreateSubscriptionUserAddOnAssignments Regular 2.7 s ✅ +40.00 KiB
    20230616164705 - AddForeignKeyAddOnPurchaseIdOnSubscriptionUserAddOnAssignments Regular 2.7 s ✅ +0.00 B
    20230616164731 - AddForeignKeyUserIdOnSubscriptionUserAddOnAssignments Regular 2.8 s ⚠ +0.00 B
    20230621065943 - AddElasticsearchRequeueWorkersToApplicationSettings Regular 2.3 s ✅ +0.00 B
    20230621070810 - UpdateRequeueWorkersInApplicationSettingsForGitlabCom Regular 2.2 s ✅ +0.00 B
    20230608145500 - AddPreparedAtIndexToMergeRequests Post deploy 4.3 s ✅ +0.00 B
    20230616082958 - AddUniqueIndexForNpmPackagesOnProjectIdNameVersion Post deploy 3.4 s ✅ +8.00 KiB
    20230619123701 - ScheduleRemovalIndexJobArtifactsIdAndExpireAt Post deploy 2.8 s ✅ +0.00 B
    20230620134708 - ValidateUserTypeConstraint Post deploy 2.3 s ✅ +0.00 B
    Clone details
    Clone ID Clone Created At Clone Data Timestamp Expected Removal Time
    database-testing-2057478-10440087-main 2023-06-26T17:26:16Z 2023-06-26T16:22:49Z 2023-06-27 05:32:37 +0000
    database-testing-2057478-10440087-ci 2023-06-26T17:26:16Z 2023-06-26T16:46:57Z 2023-06-27 05:32:37 +0000

    Job artifacts

    Brought to you by gitlab-org/database-team/gitlab-com-database-testing. Epic

Edited by João Alexandre Cunha
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information