Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sentry: error.go:25: unexpected error: forecasted histogram had first bucket with non-zero NumRange or DistinctRange: (1) × Wraps: (2) issue #93892 Wraps: (3) attached stack trace -- stack trace: ... #127945

Closed
cockroach-sentry opened this issue Jul 31, 2024 · 1 comment
Labels
branch-release-24.1 Used to mark GA and release blockers, technical advisories, and bugs for 24.1 C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. O-sentry Originated from an in-the-wild panic report. X-duplicate Closed as a duplicate of another issue.

Comments

@cockroach-sentry
Copy link
Collaborator

cockroach-sentry commented Jul 31, 2024

This issue was auto filed by Sentry. It represents a crash or reported error on a live cluster with telemetry enabled.

Sentry Link: https://cockroach-labs.sentry.io/issues/5660116204/?referrer=webhooks_plugin

Panic Message:

error.go:25: unexpected error: forecasted histogram had first bucket with non-zero NumRange or DistinctRange: 
(1) ×
Wraps: (2) issue #93892
Wraps: (3) attached stack trace
  -- stack trace:
  | github.com/cockroachdb/cockroach/pkg/util/errorutil.UnexpectedWithIssueErrorf
  | 	github.com/cockroachdb/cockroach/pkg/util/errorutil/error.go:26
  | [...repeated from below...]
Wraps: (4) unexpected error
Wraps: (5) attached stack trace
  -- stack trace:
  | github.com/cockroachdb/cockroach/pkg/util/errorutil.UnexpectedWithIssueErrorf
  | 	github.com/cockroachdb/cockroach/pkg/util/errorutil/error.go:25
  | github.com/cockroachdb/cockroach/pkg/sql/stats.forecastColumnStatistics
  | 	github.com/cockroachdb/cockroach/pkg/sql/stats/pkg/sql/stats/forecast.go:386
  | github.com/cockroachdb/cockroach/pkg/sql/stats.ForecastTableStatistics
  | 	github.com/cockroachdb/cockroach/pkg/sql/stats/pkg/sql/stats/forecast.go:153
  | github.com/cockroachdb/cockroach/pkg/sql/stats.(*TableStatisticsCache).getTableStatsFromDB
  | 	github.com/cockroachdb/cockroach/pkg/sql/stats/pkg/sql/stats/stats_cache.go:854
  | github.com/cockroachdb/cockroach/pkg/sql/stats.(*TableStatisticsCache).addCacheEntryLocked.func1
  | 	github.com/cockroachdb/cockroach/pkg/sql/stats/pkg/sql/stats/stats_cache.go:432
  | github.com/cockroachdb/cockroach/pkg/sql/stats.(*TableStatisticsCache).addCacheEntryLocked
  | 	github.com/cockroachdb/cockroach/pkg/sql/stats/pkg/sql/stats/stats_cache.go:434
  | github.com/cockroachdb/cockroach/pkg/sql/stats.(*TableStatisticsCache).getTableStatsFromCache
  | 	github.com/cockroachdb/cockroach/pkg/sql/stats/pkg/sql/stats/stats_cache.go:339
  | github.com/cockroachdb/cockroach/pkg/sql/stats.(*TableStatisticsCache).GetTableStats
  | 	github.com/cockroachdb/cockroach/pkg/sql/stats/pkg/sql/stats/stats_cache.go:238
  | github.com/cockroachdb/cockroach/pkg/sql.(*optCatalog).dataSourceForTable
  | 	github.com/cockroachdb/cockroach/pkg/sql/opt_catalog.go:562
  | github.com/cockroachdb/cockroach/pkg/sql.(*optCatalog).dataSourceForDesc
  | 	github.com/cockroachdb/cockroach/pkg/sql/opt_catalog.go:522
  | github.com/cockroachdb/cockroach/pkg/sql.(*optCatalog).ResolveDataSource
  | 	github.com/cockroachdb/cockroach/pkg/sql/opt_catalog.go:270
  | github.com/cockroachdb/cockroach/pkg/sql/opt.(*Metadata).CheckDependencies
  | 	github.com/cockroachdb/cockroach/pkg/sql/opt/metadata.go:393
  | github.com/cockroachdb/cockroach/pkg/sql/opt/memo.(*Memo).IsStale
  | 	github.com/cockroachdb/cockroach/pkg/sql/opt/memo/memo.go:457
  | github.com/cockroachdb/cockroach/pkg/sql.(*optPlanningCtx).buildExecMemo
  | 	github.com/cockroachdb/cockroach/pkg/sql/plan_opt.go:569
  | github.com/cockroachdb/cockroach/pkg/sql.(*planner).makeOptimizerPlan
  | 	github.com/cockroachdb/cockroach/pkg/sql/plan_opt.go:250
  | github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).makeExecPlan
  | 	github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:2325
  | github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).dispatchToExecutionEngine
  | 	github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:1817
  | github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execStmtInOpenState
  | 	github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:1150
  | github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execStmt.func1
  | 	github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:145
  | github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execWithProfiling
  | 	github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:3457
  | github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execStmt
  | 	github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:144
  | github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execCmd.func1
  | 	github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:2291
  | github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execCmd
  | 	github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:2296
  | github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).run
  | 	github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:2213
  | github.com/cockroachdb/cockroach/pkg/sql.(*InternalExecutor).runWithEx.func1
  | 	github.com/cockroachdb/cockroach/pkg/sql/internal.go:228
  | runtime.goexit
  | 	src/runtime/asm_amd64.s:1695
Wraps: (6) forecasted histogram had first bucket with non-zero NumRange or DistinctRange: 
Error types: (1) *hintdetail.withHint (2) *safedetails.withSafeDetails (3) *withstack.withStack (4) *errutil.withPrefix (5) *withstack.withStack (6) *errutil.leafError
-- report composition:
*errutil.leafError: forecasted histogram had first bucket with non-zero NumRange or DistinctRange: 
error.go:25: *withstack.withStack (top exception)
*errutil.withPrefix: unexpected error
error.go:26: *withstack.withStack (1)
*safedetails.withSafeDetails: issue #93892
*hintdetail.withHint
(check the extra data payloads)
Stacktrace (expand for inline code snippets):

src/runtime/asm_amd64.s#L1694-L1696

go func() {
if err := ex.run(
ctx,

var err error
if err = ex.execCmd(); err != nil {
// Both of these errors are normal ways for the connExecutor to exit.

return err
}()
// Note: we write to ex.statsCollector.PhaseTimes, instead of ex.phaseTimes,

tcmd.LastInBatch || !implicitTxnForBatch)
ev, payload, err = ex.execStmt(
ctx, tcmd.Statement, nil /* portal */, nil /* pinfo */, stmtRes, canAutoCommit,

}
err = ex.execWithProfiling(ctx, ast, preparedStmt, func(ctx context.Context) error {
ev, payload, err = ex.execStmtInOpenState(ctx, parserStmt, portal, pinfo, res, canAutoCommit)

} else {
err = op(ctx)
}

err = ex.execWithProfiling(ctx, ast, preparedStmt, func(ctx context.Context) error {
ev, payload, err = ex.execStmtInOpenState(ctx, parserStmt, portal, pinfo, res, canAutoCommit)
return err

} else {
if err := ex.dispatchToExecutionEngine(stmtCtx, p, res); err != nil {
stmtThresholdSpan.Finish()

// between here and there needs to happen even if there's an error.
err = ex.makeExecPlan(ctx, planner)
defer planner.curPlan.close(ctx)

if err := planner.makeOptimizerPlan(ctx); err != nil {
log.VEventf(ctx, 1, "optimizer plan failed: %v", err)

execMemo, err := opc.buildExecMemo(ctx)
if err != nil {

if ok {
if isStale, err := cachedData.Memo.IsStale(ctx, p.EvalContext(), opc.catalog); err != nil {
return nil, err

// access the object.
if depsUpToDate, err := m.Metadata().CheckDependencies(ctx, evalCtx, catalog); err != nil {
return true, err

tableName := name.ToTableName()
toCheck, _, err = optCatalog.ResolveDataSource(ctx, cat.Flags{}, &tableName)
if err != nil || !dataSource.Equals(toCheck) {

ds, err := oc.dataSourceForDesc(ctx, flags, desc, &oc.tn)
if err != nil {

// Tables require invalidation logic for cached wrappers.
return oc.dataSourceForTable(ctx, flags, desc, name)
}

var err error
tableStats, err = oc.planner.execCfg.TableStatsCache.GetTableStats(ctx, desc)
if err != nil {

https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/sql/stats/pkg/sql/stats/stats_cache.go#L237-L239
https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/sql/stats/pkg/sql/stats/stats_cache.go#L338-L340
https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/sql/stats/pkg/sql/stats/stats_cache.go#L433-L435
https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/sql/stats/pkg/sql/stats/stats_cache.go#L431-L433
https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/sql/stats/pkg/sql/stats/stats_cache.go#L853-L855
https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/sql/stats/pkg/sql/stats/forecast.go#L152-L154
https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/sql/stats/pkg/sql/stats/forecast.go#L385-L387
err := errors.Newf(format, args...)
err = errors.Wrap(err, "unexpected error")
err = errors.WithSafeDetails(err, "issue #%d", errors.Safe(issue))

src/runtime/asm_amd64.s#L1694-L1696
go func() {
if err := ex.run(
ctx,

var err error
if err = ex.execCmd(); err != nil {
// Both of these errors are normal ways for the connExecutor to exit.

return err
}()
// Note: we write to ex.statsCollector.PhaseTimes, instead of ex.phaseTimes,

tcmd.LastInBatch || !implicitTxnForBatch)
ev, payload, err = ex.execStmt(
ctx, tcmd.Statement, nil /* portal */, nil /* pinfo */, stmtRes, canAutoCommit,

}
err = ex.execWithProfiling(ctx, ast, preparedStmt, func(ctx context.Context) error {
ev, payload, err = ex.execStmtInOpenState(ctx, parserStmt, portal, pinfo, res, canAutoCommit)

} else {
err = op(ctx)
}

err = ex.execWithProfiling(ctx, ast, preparedStmt, func(ctx context.Context) error {
ev, payload, err = ex.execStmtInOpenState(ctx, parserStmt, portal, pinfo, res, canAutoCommit)
return err

} else {
if err := ex.dispatchToExecutionEngine(stmtCtx, p, res); err != nil {
stmtThresholdSpan.Finish()

// between here and there needs to happen even if there's an error.
err = ex.makeExecPlan(ctx, planner)
defer planner.curPlan.close(ctx)

if err := planner.makeOptimizerPlan(ctx); err != nil {
log.VEventf(ctx, 1, "optimizer plan failed: %v", err)

execMemo, err := opc.buildExecMemo(ctx)
if err != nil {

if ok {
if isStale, err := cachedData.Memo.IsStale(ctx, p.EvalContext(), opc.catalog); err != nil {
return nil, err

// access the object.
if depsUpToDate, err := m.Metadata().CheckDependencies(ctx, evalCtx, catalog); err != nil {
return true, err

tableName := name.ToTableName()
toCheck, _, err = optCatalog.ResolveDataSource(ctx, cat.Flags{}, &tableName)
if err != nil || !dataSource.Equals(toCheck) {

ds, err := oc.dataSourceForDesc(ctx, flags, desc, &oc.tn)
if err != nil {

// Tables require invalidation logic for cached wrappers.
return oc.dataSourceForTable(ctx, flags, desc, name)
}

var err error
tableStats, err = oc.planner.execCfg.TableStatsCache.GetTableStats(ctx, desc)
if err != nil {

https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/sql/stats/pkg/sql/stats/stats_cache.go#L237-L239
https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/sql/stats/pkg/sql/stats/stats_cache.go#L338-L340
https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/sql/stats/pkg/sql/stats/stats_cache.go#L433-L435
https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/sql/stats/pkg/sql/stats/stats_cache.go#L431-L433
https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/sql/stats/pkg/sql/stats/stats_cache.go#L853-L855
https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/sql/stats/pkg/sql/stats/forecast.go#L152-L154
https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/sql/stats/pkg/sql/stats/forecast.go#L385-L387
func UnexpectedWithIssueErrorf(issue int, format string, args ...interface{}) error {
err := errors.Newf(format, args...)
err = errors.Wrap(err, "unexpected error")

src/runtime/asm_amd64.s in runtime.goexit at line 1695
pkg/sql/internal.go in pkg/sql.(*InternalExecutor).runWithEx.func1 at line 228
pkg/sql/conn_executor.go in pkg/sql.(*connExecutor).run at line 2213
pkg/sql/conn_executor.go in pkg/sql.(*connExecutor).execCmd at line 2296
pkg/sql/conn_executor.go in pkg/sql.(*connExecutor).execCmd.func1 at line 2291
pkg/sql/conn_executor_exec.go in pkg/sql.(*connExecutor).execStmt at line 144
pkg/sql/conn_executor_exec.go in pkg/sql.(*connExecutor).execWithProfiling at line 3457
pkg/sql/conn_executor_exec.go in pkg/sql.(*connExecutor).execStmt.func1 at line 145
pkg/sql/conn_executor_exec.go in pkg/sql.(*connExecutor).execStmtInOpenState at line 1150
pkg/sql/conn_executor_exec.go in pkg/sql.(*connExecutor).dispatchToExecutionEngine at line 1817
pkg/sql/conn_executor_exec.go in pkg/sql.(*connExecutor).makeExecPlan at line 2325
pkg/sql/plan_opt.go in pkg/sql.(*planner).makeOptimizerPlan at line 250
pkg/sql/plan_opt.go in pkg/sql.(*optPlanningCtx).buildExecMemo at line 569
pkg/sql/opt/memo/memo.go in pkg/sql/opt/memo.(*Memo).IsStale at line 457
pkg/sql/opt/metadata.go in pkg/sql/opt.(*Metadata).CheckDependencies at line 393
pkg/sql/opt_catalog.go in pkg/sql.(*optCatalog).ResolveDataSource at line 270
pkg/sql/opt_catalog.go in pkg/sql.(*optCatalog).dataSourceForDesc at line 522
pkg/sql/opt_catalog.go in pkg/sql.(*optCatalog).dataSourceForTable at line 562
pkg/sql/stats/pkg/sql/stats/stats_cache.go in pkg/sql/stats.(*TableStatisticsCache).GetTableStats at line 238
pkg/sql/stats/pkg/sql/stats/stats_cache.go in pkg/sql/stats.(*TableStatisticsCache).getTableStatsFromCache at line 339
pkg/sql/stats/pkg/sql/stats/stats_cache.go in pkg/sql/stats.(*TableStatisticsCache).addCacheEntryLocked at line 434
pkg/sql/stats/pkg/sql/stats/stats_cache.go in pkg/sql/stats.(*TableStatisticsCache).addCacheEntryLocked.func1 at line 432
pkg/sql/stats/pkg/sql/stats/stats_cache.go in pkg/sql/stats.(*TableStatisticsCache).getTableStatsFromDB at line 854
pkg/sql/stats/pkg/sql/stats/forecast.go in pkg/sql/stats.ForecastTableStatistics at line 153
pkg/sql/stats/pkg/sql/stats/forecast.go in pkg/sql/stats.forecastColumnStatistics at line 386
pkg/util/errorutil/error.go in pkg/util/errorutil.UnexpectedWithIssueErrorf at line 26
src/runtime/asm_amd64.s in runtime.goexit at line 1695
pkg/sql/internal.go in pkg/sql.(*InternalExecutor).runWithEx.func1 at line 228
pkg/sql/conn_executor.go in pkg/sql.(*connExecutor).run at line 2213
pkg/sql/conn_executor.go in pkg/sql.(*connExecutor).execCmd at line 2296
pkg/sql/conn_executor.go in pkg/sql.(*connExecutor).execCmd.func1 at line 2291
pkg/sql/conn_executor_exec.go in pkg/sql.(*connExecutor).execStmt at line 144
pkg/sql/conn_executor_exec.go in pkg/sql.(*connExecutor).execWithProfiling at line 3457
pkg/sql/conn_executor_exec.go in pkg/sql.(*connExecutor).execStmt.func1 at line 145
pkg/sql/conn_executor_exec.go in pkg/sql.(*connExecutor).execStmtInOpenState at line 1150
pkg/sql/conn_executor_exec.go in pkg/sql.(*connExecutor).dispatchToExecutionEngine at line 1817
pkg/sql/conn_executor_exec.go in pkg/sql.(*connExecutor).makeExecPlan at line 2325
pkg/sql/plan_opt.go in pkg/sql.(*planner).makeOptimizerPlan at line 250
pkg/sql/plan_opt.go in pkg/sql.(*optPlanningCtx).buildExecMemo at line 569
pkg/sql/opt/memo/memo.go in pkg/sql/opt/memo.(*Memo).IsStale at line 457
pkg/sql/opt/metadata.go in pkg/sql/opt.(*Metadata).CheckDependencies at line 393
pkg/sql/opt_catalog.go in pkg/sql.(*optCatalog).ResolveDataSource at line 270
pkg/sql/opt_catalog.go in pkg/sql.(*optCatalog).dataSourceForDesc at line 522
pkg/sql/opt_catalog.go in pkg/sql.(*optCatalog).dataSourceForTable at line 562
pkg/sql/stats/pkg/sql/stats/stats_cache.go in pkg/sql/stats.(*TableStatisticsCache).GetTableStats at line 238
pkg/sql/stats/pkg/sql/stats/stats_cache.go in pkg/sql/stats.(*TableStatisticsCache).getTableStatsFromCache at line 339
pkg/sql/stats/pkg/sql/stats/stats_cache.go in pkg/sql/stats.(*TableStatisticsCache).addCacheEntryLocked at line 434
pkg/sql/stats/pkg/sql/stats/stats_cache.go in pkg/sql/stats.(*TableStatisticsCache).addCacheEntryLocked.func1 at line 432
pkg/sql/stats/pkg/sql/stats/stats_cache.go in pkg/sql/stats.(*TableStatisticsCache).getTableStatsFromDB at line 854
pkg/sql/stats/pkg/sql/stats/forecast.go in pkg/sql/stats.ForecastTableStatistics at line 153
pkg/sql/stats/pkg/sql/stats/forecast.go in pkg/sql/stats.forecastColumnStatistics at line 386
pkg/util/errorutil/error.go in pkg/util/errorutil.UnexpectedWithIssueErrorf at line 25

Tags

Tag Value
Command server
Environment v24.1.1
Go Version go1.22.3 X:nocoverageredesign
Platform linux amd64
Distribution CCL
Cockroach Release v24.1.1
Cockroach SHA 484f8e2
# of CPUs 4
# of Goroutines 570

Jira issue: CRDB-40684

@cockroach-sentry cockroach-sentry added branch-release-24.1 Used to mark GA and release blockers, technical advisories, and bugs for 24.1 C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. O-sentry Originated from an in-the-wild panic report. labels Jul 31, 2024
@michae2 michae2 added the X-duplicate Closed as a duplicate of another issue. label Aug 8, 2024
@michae2
Copy link
Collaborator

michae2 commented Aug 8, 2024

dup of #93892

@michae2 michae2 closed this as not planned Won't fix, can't repro, duplicate, stale Aug 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
branch-release-24.1 Used to mark GA and release blockers, technical advisories, and bugs for 24.1 C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. O-sentry Originated from an in-the-wild panic report. X-duplicate Closed as a duplicate of another issue.
Projects
None yet
Development

No branches or pull requests

2 participants