Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
SlideShare a Scribd company logo

Illustrated buffer cache

•
•
Selena Deckelmann
Selena Deckelmann

The document is an illustrated guide to PostgreSQL's buffer cache. It shows how the buffer cache, called shared_buffers, stores table data in memory using pointers to 8K blocks. The buffer cache tracks block usage and status, evicting least recently used blocks to make space for new blocks as needed. It can optimize memory usage by allocating a ring buffer for vacuuming tables larger than one quarter of shared_buffers.

1 of 26
An illustrated guide to the buffer cache Selena Deckelmann Based on Greg Smith’s “Inside the Buffer Cache” http://www.westnet.com/∼gsmith/content/postgresql
Table Databases Table Table Databases are made of tables...
$PGDATA/base/ Table A table is just a directory...
$PGDATA/base/ Table Table Table You can have more than one table.
$PGDATA/base/ Table Table Table Made of Made o f 8K File 8K 8K File 8K 8K File 8K File 8K ... ... Up to 1 GB Files are made up of 8K blocks.* * you can change that at compile time.
shared_buffers
shared_buffers ptr ptr ptr ptr ptr ... An array of pointers to 8K blocks (and some other stuff) shared_buffers is configured at database start. You have to restart Postgres to change it.
shared_buffers: the other stuff FREE FREE FREE FREE FREE FREE In the beginning...
shared_buffers: the other stuff FREE FREE FREE FREE FREE FREE 456 12 100 612 32 409
shared_buffers: the other stuff X X X X X X 456 12 100 612 32 409
shared_buffers: the other stuff X X X X X X Tag base/file1 base/file2 base/file1 base/file3 base/file1 base/file1 456 12 100 612 32 409
shared_buffers: the other stuff X X X X X X Tag base/file1 base/file2 base/file1 base/file3 base/file1 base/file1 456 12 100 612 32 409 Status FLAGS FLAGS FLAGS FLAGS FLAGS FLAGS
shared_buffers: the other stuff X X X X X X Tag base/file1 base/file2 base/file1 base/file3 base/file1 base/file1 456 12 100 612 32 409 Status PINNED flags flags flags flags flags { } UPDATE mytable (status) VALUES(‘my cat is the best ever’) where user = ‘selenamarie’;
shared_buffers: the other stuff X X X X X X Tag base/file1 base/file2 base/file1 base/file3 base/file1 base/file1 456 12 100 612 32 409 Status PINNED DIRTY flags flags flags flags { COMMIT; }
shared_buffers: the other stuff X X X X X X Tag base/file1 base/file2 base/file1 base/file3 base/file1 base/file1 456 12 100 612 32 409 Status PINNED DIRTY OTHER OTHER OTHER OTHER { BLAH BLAH BLAH; }
shared_buffers: the other stuff X X X X X X Tag base/file1 base/file2 base/file1 base/file3 base/file1 base/file1 456 12 100 612 32 409 Status PINNED DIRTY OTHER OTHER OTHER OTHER Usage 1 5 5 4 3 0
shared_buffers: the lifecycle X X X X X X base/file1 base/file2 base/file1 base/file3 base/file1 base/file1 456 12 100 612 32 409 PINNED DIRTY OTHER OTHER OTHER OTHER 1 5 5 4 3 0 { myPgProcess calls: BufferAlloc(‘base/file1’, 100); }
shared_buffers: the lifecycle X X X X X X base/file1 base/file2 base/file1 base/file3 base/file1 base/file1 456 12 100 612 32 409 PINNED DIRTY PINNED OTHER OTHER OTHER 1 5 5 4 3 0 { myPgProcess calls: BufferAlloc(‘base/file1’, 100); }
shared_buffers: the lifecycle X X X X X X base/file1 base/file2 base/file1 base/file3 base/file1 base/file1 456 12 100 612 32 409 PINNED DIRTY OTHER OTHER OTHER OTHER 1 5 5 4 3 0 { myPgProcess calls: BufferAlloc(‘base/file1’, 101); }
shared_buffers: the lifecycle Who’s it gonna be? X X X X X X base/file1 base/file2 base/file1 base/file3 base/file1 base/file1 456 12 100 612 32 409 PINNED DIRTY OTHER OTHER OTHER OTHER 1 5 5 4 3 0 Eviction! { myPgProcess calls: BufferAlloc(‘base/file1’, 101); }
shared_buffers: the lifecycle Who’s it gonna be? X X X X X X base/file1 base/file2 base/file1 base/file3 base/file1 base/file1 456 12 100 612 32 409 PINNED DIRTY OTHER OTHER OTHER OTHER 0 5 5 4 3 0 Scan the cache! Usage != 0, so decrement. { myPgProcess calls: BufferAlloc(‘base/file1’, 101); }
shared_buffers: the lifecycle Who’s it gonna be? X X X X X X base/file1 base/file2 base/file1 base/file3 base/file1 base/file1 Usage == 0, 456 12 100 612 32 409 Goodbye! PINNED DIRTY OTHER OTHER OTHER OTHER 0 4 4 3 2 0 { myPgProcess calls: BufferAlloc(‘base/file1’, 101); }
shared_buffers: the lifecycle Who’s it gonna be? X X X X X X base/file1 base/file2 base/file1 base/file3 base/file1 base/file1 456 12 100 612 32 101 PINNED DIRTY OTHER OTHER OTHER PINNED 0 4 4 3 2 1 { myPgProcess calls: BufferAlloc(‘base/file1’, 101); }
shared_buffers: optimizations* X X X X X X X ... base/file1 base/file2 base/file1 base/file3 base/file1 base/file1 base/file2 ... 456 12 100 612 32 101 891 ... PINNED DIRTY OTHER OTHER OTHER PINNED OTHER ... 0 4 4 3 2 0 1 ... { } Result is greater than: SELECT pg_relation_size(‘mytable’); shared_buffers/4 VACUUM mytable; * Version 8.3 and later
shared_buffers: optimizations X X X X X X X ... base/file1 base/file2 base/file1 base/file3 base/file1 base/file1 base/file2 ... 456 12 100 612 32 101 891 ... PINNED DIRTY OTHER OTHER OTHER PINNED OTHER ... 0 4 4 3 2 0 1 ... 256K ring buffer allocated! { SELECT pg_relation_size(‘mytable’); VACUUM mytable; } * Version 8.3 and later
shared_buffers: optimizations VACUUM Safe during OVERSIMPLIFICATION WARNING! X X X X X X X ... base/file1 base/file2 base/file1 base/file3 base/file1 base/file1 base/file2 ... 456 12 100 612 32 101 891 ... PINNED DIRTY OTHER OTHER OTHER PINNED OTHER ... 0 4 4 3 2 0 1 ... 256K ring buffer allocated! { SELECT pg_relation_size(‘mytable’); VACUUM mytable; } * Version 8.3 and later

More Related Content

Illustrated buffer cache

  • 1. An illustrated guide to the buffer cache Selena Deckelmann Based on Greg Smith’s “Inside the Buffer Cache” http://www.westnet.com/∼gsmith/content/postgresql
  • 2. Table Databases Table Table Databases are made of tables...
  • 3. $PGDATA/base/ Table A table is just a directory...
  • 4. $PGDATA/base/ Table Table Table You can have more than one table.
  • 5. $PGDATA/base/ Table Table Table Made of Made o f 8K File 8K 8K File 8K 8K File 8K File 8K ... ... Up to 1 GB Files are made up of 8K blocks.* * you can change that at compile time.
  • 7. shared_buffers ptr ptr ptr ptr ptr ... An array of pointers to 8K blocks (and some other stuff) shared_buffers is configured at database start. You have to restart Postgres to change it.
  • 8. shared_buffers: the other stuff FREE FREE FREE FREE FREE FREE In the beginning...
  • 9. shared_buffers: the other stuff FREE FREE FREE FREE FREE FREE 456 12 100 612 32 409
  • 10. shared_buffers: the other stuff X X X X X X 456 12 100 612 32 409
  • 11. shared_buffers: the other stuff X X X X X X Tag base/file1 base/file2 base/file1 base/file3 base/file1 base/file1 456 12 100 612 32 409
  • 12. shared_buffers: the other stuff X X X X X X Tag base/file1 base/file2 base/file1 base/file3 base/file1 base/file1 456 12 100 612 32 409 Status FLAGS FLAGS FLAGS FLAGS FLAGS FLAGS
  • 13. shared_buffers: the other stuff X X X X X X Tag base/file1 base/file2 base/file1 base/file3 base/file1 base/file1 456 12 100 612 32 409 Status PINNED flags flags flags flags flags { } UPDATE mytable (status) VALUES(‘my cat is the best ever’) where user = ‘selenamarie’;
  • 14. shared_buffers: the other stuff X X X X X X Tag base/file1 base/file2 base/file1 base/file3 base/file1 base/file1 456 12 100 612 32 409 Status PINNED DIRTY flags flags flags flags { COMMIT; }
  • 15. shared_buffers: the other stuff X X X X X X Tag base/file1 base/file2 base/file1 base/file3 base/file1 base/file1 456 12 100 612 32 409 Status PINNED DIRTY OTHER OTHER OTHER OTHER { BLAH BLAH BLAH; }
  • 16. shared_buffers: the other stuff X X X X X X Tag base/file1 base/file2 base/file1 base/file3 base/file1 base/file1 456 12 100 612 32 409 Status PINNED DIRTY OTHER OTHER OTHER OTHER Usage 1 5 5 4 3 0
  • 17. shared_buffers: the lifecycle X X X X X X base/file1 base/file2 base/file1 base/file3 base/file1 base/file1 456 12 100 612 32 409 PINNED DIRTY OTHER OTHER OTHER OTHER 1 5 5 4 3 0 { myPgProcess calls: BufferAlloc(‘base/file1’, 100); }
  • 18. shared_buffers: the lifecycle X X X X X X base/file1 base/file2 base/file1 base/file3 base/file1 base/file1 456 12 100 612 32 409 PINNED DIRTY PINNED OTHER OTHER OTHER 1 5 5 4 3 0 { myPgProcess calls: BufferAlloc(‘base/file1’, 100); }
  • 19. shared_buffers: the lifecycle X X X X X X base/file1 base/file2 base/file1 base/file3 base/file1 base/file1 456 12 100 612 32 409 PINNED DIRTY OTHER OTHER OTHER OTHER 1 5 5 4 3 0 { myPgProcess calls: BufferAlloc(‘base/file1’, 101); }
  • 20. shared_buffers: the lifecycle Who’s it gonna be? X X X X X X base/file1 base/file2 base/file1 base/file3 base/file1 base/file1 456 12 100 612 32 409 PINNED DIRTY OTHER OTHER OTHER OTHER 1 5 5 4 3 0 Eviction! { myPgProcess calls: BufferAlloc(‘base/file1’, 101); }
  • 21. shared_buffers: the lifecycle Who’s it gonna be? X X X X X X base/file1 base/file2 base/file1 base/file3 base/file1 base/file1 456 12 100 612 32 409 PINNED DIRTY OTHER OTHER OTHER OTHER 0 5 5 4 3 0 Scan the cache! Usage != 0, so decrement. { myPgProcess calls: BufferAlloc(‘base/file1’, 101); }
  • 22. shared_buffers: the lifecycle Who’s it gonna be? X X X X X X base/file1 base/file2 base/file1 base/file3 base/file1 base/file1 Usage == 0, 456 12 100 612 32 409 Goodbye! PINNED DIRTY OTHER OTHER OTHER OTHER 0 4 4 3 2 0 { myPgProcess calls: BufferAlloc(‘base/file1’, 101); }
  • 23. shared_buffers: the lifecycle Who’s it gonna be? X X X X X X base/file1 base/file2 base/file1 base/file3 base/file1 base/file1 456 12 100 612 32 101 PINNED DIRTY OTHER OTHER OTHER PINNED 0 4 4 3 2 1 { myPgProcess calls: BufferAlloc(‘base/file1’, 101); }
  • 24. shared_buffers: optimizations* X X X X X X X ... base/file1 base/file2 base/file1 base/file3 base/file1 base/file1 base/file2 ... 456 12 100 612 32 101 891 ... PINNED DIRTY OTHER OTHER OTHER PINNED OTHER ... 0 4 4 3 2 0 1 ... { } Result is greater than: SELECT pg_relation_size(‘mytable’); shared_buffers/4 VACUUM mytable; * Version 8.3 and later
  • 25. shared_buffers: optimizations X X X X X X X ... base/file1 base/file2 base/file1 base/file3 base/file1 base/file1 base/file2 ... 456 12 100 612 32 101 891 ... PINNED DIRTY OTHER OTHER OTHER PINNED OTHER ... 0 4 4 3 2 0 1 ... 256K ring buffer allocated! { SELECT pg_relation_size(‘mytable’); VACUUM mytable; } * Version 8.3 and later
  • 26. shared_buffers: optimizations VACUUM Safe during OVERSIMPLIFICATION WARNING! X X X X X X X ... base/file1 base/file2 base/file1 base/file3 base/file1 base/file1 base/file2 ... 456 12 100 612 32 101 891 ... PINNED DIRTY OTHER OTHER OTHER PINNED OTHER ... 0 4 4 3 2 0 1 ... 256K ring buffer allocated! { SELECT pg_relation_size(‘mytable’); VACUUM mytable; } * Version 8.3 and later