"Prometheus TSDB compaction failed"의 두 판 사이의 차이

2024년 3월 28일 (목) 17:29 기준 최신판

1 개요[ | ]

Prometheus TSDB compaction failed

2 persist head block: write compaction: add serires[ | ]

# https://github.com/prometheus/prometheus/issues/3487
# https://github.com/prometheus/prometheus/issues/4108
# https://github.com/prometheus/prometheus/issues/5617
level=error ts=2019-05-31T08:36:52.501Z caller=db.go:363 component=tsdb msg="compaction failed" err="persist head block: write compaction: add series: out-of-order series added with label set ..."

3 reload blocks: head truncate failed[ | ]

# https://www.mail-archive.com/prometheus-users@googlegroups.com/msg05767.html
level=error ts=0000-00-00T00:00:00.000Z caller=db.go:685 component=tsdb msg="compaction failed" err="reload blocks: head truncate failed: create checkpoint: read segments: corruption in segment /var/lib/prometheus2/data/wal/00004615 at 28021780: unexpected full record"

# https://github.com/prometheus/prometheus/issues/5867#issuecomment-630558412
level=error ts=2020-05-18T23:00:02.386Z caller=db.go:659 component=tsdb msg="compaction failed" err="reload blocks: head truncate failed: create checkpoint: read segments: corruption in segment data/wal/00000340 at 62972957: unexpected checksum c23eec43, expected f865211c"

# https://github.com/prometheus/prometheus/issues/7506#issuecomment-653058543
level=error ts=0000-00-00T00:00:00.000Z caller=db.go:675 component=tsdb msg="compaction failed" err="reload blocks: head truncate failed: create checkpoint: read segments: corruption in segment /prometheus/wal/checkpoint.024462/00000000 at 127664128: unexpected non-zero byte in padded page"

# https://groups.google.com/g/prometheus-users/c/fqRjxSBwXQ4
level=error ts=2020-03-30T23:00:37.520Z caller=db.go:617 component=tsdb msg="compaction failed" err="reload blocks: head truncate failed: create checkpoint: unexpected gap to last checkpoint. expected:3, requested:966" 

# https://github.com/prometheus/prometheus/issues/6920
level=error ts=2020-03-02T15:00:02.276Z caller=db.go:561 component=tsdb msg="compaction failed" err="reload blocks: head truncate failed: get segment range: segments are not sequential"