A Deeper Dive into EXPLAIN

A Deeper Dive into EXPLAIN

• 1.
  A deeper dive intoEXPLAIN Michael Christofides
• 2.
  Hi, I’m Michael Halfof the team behind pgMustard Spent a lot of time looking into EXPLAIN Background: product management, database tools pgmustard.com/docs/explain [email protected] michristofides
• 3.
  Picking up fromother EXPLAIN talks Not the basics* 1) Some of the less intuitive arithmetic 2) Some less well covered issues * postgresql.org/docs/current/performance-tips thoughtbot: reading EXPLAIN ANALYZE YouTube: Josh Berkus Explaining EXPLAIN
• 4.
  Picking up fromother EXPLAIN talks Not the basics* 1) Arithmetic: why is this query slow? 2) Issues: what can we do about it? * postgresql.org/docs/current/performance-tips thoughtbot: reading EXPLAIN ANALYZE YouTube: Josh Berkus Explaining EXPLAIN
• 5.
  Arithmetic: loops Many ofthe stats are a per-loop average This includes costs, rows, timings Watch out for rounding, especially to 0 rows
• 6.
  Disclaimer: heavily doctored plansahead, inaccuracies likely.
• 7.
  Nested Loop (cost=0.84..209.82 rows=16width=11) (actual time=0.076..0.368 rows=86 loops=1) -> Index Only Scan using a on b (cost=0.42..4.58 rows=9 width=4) (actual time=0.013..0.019 rows=9 loops=1) -> Index Scan using x on y (cost=0.42..22.73 rows=7 width=15) (actual time=0.012..0.030 rows=10 loops=9) Nested Loop: 86 rows Index Scan: 9 * 10 = 90 rows (Rounding not too bad here)
• 8.
  Arithmetic: threads Costs, rows,and timings are also per-thread Shown as loops Threads = workers + 1 Tip: use VERBOSE <- the leader
• 9.
  Parallel Seq Scanon table (cost=0.00..6772.21 rows=79521 width=22) (actual time=0.090..71.866 rows=63617 loops=3) Output: column1, column2, column3 Worker 0: actual time=0.111..66.325 rows=56225 loops=1 Worker 1: actual time=0.138..66.027 rows=58792 loops=1 Seq Scan: 63617 * 3 = 190851 rows Leader: 190851 - 58792 - 56225 = 75834 rows
• 10.
  Arithmetic: buffers Buffer statsare a total, not per-loop They are inclusive of children
• 11.
  Nested Loop (...loops=1) Buffers: shared hit=105 -> Index Only Scan using a on b (... loops=1) Buffers: shared hit=4 -> Index Scan using x on y (... loops=9) Buffers: shared hit=101 Nested Loop buffers: 105 - (101 + 4) = 0 blocks
• 12.
  Arithmetic: timings Per-loop, per-thread Inclusiveof children Calculating per-node (exclusive) times can get tricky, even for tools
• 13.
  Nested Loop (cost=0.84..209.82 rows=16width=11) (actual time=0.076..0.368 rows=86 loops=1) -> Index Only Scan using a on b (cost=0.42..4.58 rows=9 width=4) (actual time=0.013..0.019 rows=9 loops=1) -> Index Scan using x on y (cost=0.42..22.73 rows=7 width=15) (actual time=0.012..0.030 rows=10 loops=9) Index Scan: 0.030 * 9 = 0.27 ms Nested Loop: 0.368 - 0.27 - 0.019 = 0.079 ms
• 14.
  WITH init AS( SELECT * FROM pg_sleep_for('100ms') UNION ALL SELECT * FROM pg_sleep_for('200ms') ) (SELECT * FROM init LIMIT 1) UNION ALL (SELECT * FROM init); Credit @felixge
• 15.
  Append (actual time=100.359.. 300.688… ) CTE init -> Append (actual time=100.334.. 300.652 … ) -> Function Scan (actual time=100.333.. 100.335 … ) -> Function Scan (actual time=200.310.. 200.312 … ) -> Limit (actual time=100.358.. 100.359 … ) -> CTE Scan a (actual time=100.355..100.356 … ) -> CTE Scan b (actual time=0.001.. 200.322 … ) Execution Time: 300.789 ms Further reading: flame-explain.com/docs/general/quirk-correction Some double-counting in this case.
• 16.
  Arithmetic: tools canhelp eg explain.depesz.com explain.dalibo.com flame-explain.com pgmustard.com <- fellow calculations nerd <- 👋
• 17.
  Issues: let’s skipthe basics Seq Scans with large filters Bad row estimates Sorts and Hashes on disk
• 18.
  Issues: inefficient indexscans Looks out for lots of rows being filtered Filters are per-loop So again, watch out for rounding
• 19.
  -> Index Scanusing x on y (cost=0.42..302502.05 rows=1708602 width=125) (actual time=172810.219..173876.540 rows=1000 loops=1) Index Cond: (id = another_id) Filter: (status = 1) Rows Removed by Filter: 3125626 Index efficiency: 1000/(1000+3125626) = 0.03% Watch out for high loops
• 20.
  Issues: lossy bitmapscans When bitmap would otherwise exceed work_mem Point to a block rather than a row (Tuple Id) Lossy blocks are a total (ie not per-loop)
• 21.
  -> Bitmap HeapScan on table (cost=49153.29..4069724.27 rows=3105598 width=1106) (actual time=591.928..56472.895 rows=3853272 loops=1) Recheck Cond: (something > something_else) Rows Removed by Index Recheck: 5905323 Heap Blocks: exact=14280 lossy=1951048 Lossy blocks: 1951048/(1951048+14280) = 99% Extra rows read: 5.9 million
• 22.
  Issues: lots ofdata read Requires BUFFERS Lots of data being read for the amount returned Can be a sign of bloat Default block size: 8kB
• 23.
  -> Index Scanusing x on y (cost=0.57..2.57 rows=1 width=8) (actual time=0.064..0.064 rows=1 loops=256753) Index Cond: (id = another_id) Filter: (status = 1) Buffers: shared hit=1146405 read=110636 Caveats: width estimated, rows rounded Data read: (1146405 + 110636) * 8kB = 10GB Data returned: 1 * 256753 * 8 bytes = 2MB
• 24.
  Issues: planning time Atthe end of the query plan Can be planning related: eg joins, partitions But other things too: eg extensions, locks Warning: not included in auto_explain
• 25.
  (...) Planning Time: 27.844ms Execution Time: 11.162 ms Planning proportion: 27.844/(27.844 + 11.162) = 71%
• 26.
  Issues: Just InTime compilation At the end of the query plan Included in execution time On by default in PostgreSQL 12 and 13 Start-up time can be a tell-tale
• 27.
  Planning Time: 9.138ms JIT: Functions: 277 Options: Inlining true, Optimization true, Expressions true, Deforming true Timing: Generation 31.602 ms, Inlining 253.114 ms, Optimization 1498.268 ms, Emission 913.945 ms, Total 2696.929 ms Execution Time: 5194.851 ms JIT proportion: 2696.929/(9.138 + 5194.851) = 52%
• 28.
  Very suspicious actualstart-up time from a JIT dominated plan. -> Seq Scan on table (cost=0.00..3.57 rows=72 width=8) (actual time=2262.312..2262.343 rows=54 loops=1) Buffers: shared hit=3
• 29.
  Issues: triggers At theend of the query plan Total time across calls Check foreign keys indexed Before triggers vs after triggers
• 30.
  Planning Time: 0.227ms Trigger: RI_ConstraintTrigger_a_12345 on table time=83129.491 calls=2222623 Execution Time: 87645.739 ms Trigger proportion: 83129.491/(0.227 + 87645.739) = 95%
• 31.
  Summary: check thearithmetic Watch out for loops and threads Watch out for CTEs Tools can help, if in doubt check two
• 32.
  Summary: keep rarerissues in mind Check the end section first Also look out for filters, rechecks, lossy blocks, amount of data Tools, mailing lists, and communities can help
• 33.
  Thank you! Anyquestions? [email protected] michristofides Further reading: * flame-explain.com/docs/general/quirk-correction * pgmustard.com/docs/explain * wiki.postgresql.org/wiki/Slow_Query_Questions