Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to content

Commit 5364b35

Browse files
committed
Increase maximum number of clog buffers.
Benchmarking has shown that the current number of clog buffers limits scalability. We've previously increased the number in 33aaa13, but that's not sufficient with a large number of clients. We've benchmarked the cost of increasing the limit by benchmarking worst case scenarios; testing showed that 128 buffers don't cause a regression, even in contrived scenarios, whereas 256 does There are a number of more complex patches flying around to address various clog scalability problems, but this is simple enough that we can get it into 9.6; and is beneficial even after those patches have been applied. It is a bit unsatisfactory to increase this in small steps every few releases, but a better solution seems to require a rewrite of slru.c; not something done quickly. Author: Amit Kapila and Andres Freund Discussion: CAA4eK1+-=18HOrdqtLXqOMwZDbC_15WTyHiFruz7BvVArZPaAw@mail.gmail.com
1 parent 25fe8b5 commit 5364b35

File tree

1 file changed

+11
-18
lines changed
  • src/backend/access/transam

1 file changed

+11
-18
lines changed

src/backend/access/transam/clog.c

+11-18
Original file line numberDiff line numberDiff line change
@@ -417,30 +417,23 @@ TransactionIdGetStatus(TransactionId xid, XLogRecPtr *lsn)
417417
/*
418418
* Number of shared CLOG buffers.
419419
*
420-
* Testing during the PostgreSQL 9.2 development cycle revealed that on a
421-
* large multi-processor system, it was possible to have more CLOG page
422-
* requests in flight at one time than the number of CLOG buffers which existed
423-
* at that time, which was hardcoded to 8. Further testing revealed that
424-
* performance dropped off with more than 32 CLOG buffers, possibly because
425-
* the linear buffer search algorithm doesn't scale well.
420+
* On larger multi-processor systems, it is possible to have many CLOG page
421+
* requests in flight at one time which could lead to disk access for CLOG
422+
* page if the required page is not found in memory. Testing revealed that we
423+
* can get the best performance by having 128 CLOG buffers, more than that it
424+
* doesn't improve performance.
426425
*
427-
* Unconditionally increasing the number of CLOG buffers to 32 did not seem
428-
* like a good idea, because it would increase the minimum amount of shared
429-
* memory required to start, which could be a problem for people running very
430-
* small configurations. The following formula seems to represent a reasonable
426+
* Unconditionally keeping the number of CLOG buffers to 128 did not seem like
427+
* a good idea, because it would increase the minimum amount of shared memory
428+
* required to start, which could be a problem for people running very small
429+
* configurations. The following formula seems to represent a reasonable
431430
* compromise: people with very low values for shared_buffers will get fewer
432-
* CLOG buffers as well, and everyone else will get 32.
433-
*
434-
* It is likely that some further work will be needed here in future releases;
435-
* for example, on a 64-core server, the maximum number of CLOG requests that
436-
* can be simultaneously in flight will be even larger. But that will
437-
* apparently require more than just changing the formula, so for now we take
438-
* the easy way out.
431+
* CLOG buffers as well, and everyone else will get 128.
439432
*/
440433
Size
441434
CLOGShmemBuffers(void)
442435
{
443-
return Min(32, Max(4, NBuffers / 512));
436+
return Min(128, Max(4, NBuffers / 512));
444437
}
445438

446439
/*

0 commit comments

Comments
 (0)