Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to content

Commit 90ebcc3

Browse files
committed
Don't spuriously report FD_SETSIZE exhaustion on Windows.
Starting on 2023-08-03, this intermittently terminated a "pgbench -C" test in CI. It could affect a high-client-count "pgbench" without "-C". While parallel reindexdb and vacuumdb reach the same problematic check, sufficient client count and/or connection turnover is less plausible for them. Given the lack of examples from the buildfarm or from manual builds, reproducing this must entail rare operating system configurations. Also correct the associated error message, which was wrong for non-Windows. Back-patch to v12, where the pgbench check first appeared. While v11 vacuumdb has the problematic check, reaching it with typical vacuumdb usage is implausible. Reviewed by Thomas Munro. Discussion: https://postgr.es/m/CA+hUKG+JwvTNdcyJTriy9BbtzF1veSRQ=9M_ZKFn9_LqE7Kp7Q@mail.gmail.com
1 parent fcdd668 commit 90ebcc3

File tree

2 files changed

+49
-7
lines changed

2 files changed

+49
-7
lines changed

src/bin/pgbench/pgbench.c

Lines changed: 14 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -7837,14 +7837,23 @@ clear_socket_set(socket_set *sa)
78377837
static void
78387838
add_socket_to_set(socket_set *sa, int fd, int idx)
78397839
{
7840+
/* See connect_slot() for background on this code. */
7841+
#ifdef WIN32
7842+
if (sa->fds.fd_count + 1 >= FD_SETSIZE)
7843+
{
7844+
pg_log_error("too many concurrent database clients for this platform: %d",
7845+
sa->fds.fd_count + 1);
7846+
exit(1);
7847+
}
7848+
#else
78407849
if (fd < 0 || fd >= FD_SETSIZE)
78417850
{
7842-
/*
7843-
* Doing a hard exit here is a bit grotty, but it doesn't seem worth
7844-
* complicating the API to make it less grotty.
7845-
*/
7846-
pg_fatal("too many client connections for select()");
7851+
pg_log_error("socket file descriptor out of range for select(): %d",
7852+
fd);
7853+
pg_log_error_hint("Try fewer concurrent database clients.");
7854+
exit(1);
78477855
}
7856+
#endif
78487857
FD_SET(fd, &sa->fds);
78497858
if (fd > sa->maxfd)
78507859
sa->maxfd = fd;

src/fe_utils/parallel_slot.c

Lines changed: 35 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -295,8 +295,41 @@ connect_slot(ParallelSlotArray *sa, int slotno, const char *dbname)
295295
slot->connection = connectDatabase(sa->cparams, sa->progname, sa->echo, false, true);
296296
sa->cparams->override_dbname = old_override;
297297

298-
if (PQsocket(slot->connection) >= FD_SETSIZE)
299-
pg_fatal("too many jobs for this platform");
298+
/*
299+
* POSIX defines FD_SETSIZE as the highest file descriptor acceptable to
300+
* FD_SET() and allied macros. Windows defines it as a ceiling on the
301+
* count of file descriptors in the set, not a ceiling on the value of
302+
* each file descriptor; see
303+
* https://learn.microsoft.com/en-us/windows/win32/api/winsock2/nf-winsock2-select
304+
* and
305+
* https://learn.microsoft.com/en-us/windows/win32/api/winsock/ns-winsock-fd_set.
306+
* We can't ignore that, because Windows starts file descriptors at a
307+
* higher value, delays reuse, and skips values. With less than ten
308+
* concurrent file descriptors, opened and closed rapidly, one can reach
309+
* file descriptor 1024.
310+
*
311+
* Doing a hard exit here is a bit grotty, but it doesn't seem worth
312+
* complicating the API to make it less grotty.
313+
*/
314+
#ifdef WIN32
315+
if (slotno >= FD_SETSIZE)
316+
{
317+
pg_log_error("too many jobs for this platform: %d", slotno);
318+
exit(1);
319+
}
320+
#else
321+
{
322+
int fd = PQsocket(slot->connection);
323+
324+
if (fd >= FD_SETSIZE)
325+
{
326+
pg_log_error("socket file descriptor out of range for select(): %d",
327+
fd);
328+
pg_log_error_hint("Try fewer jobs.");
329+
exit(1);
330+
}
331+
}
332+
#endif
300333

301334
/* Setup the connection using the supplied command, if any. */
302335
if (sa->initcmd)

0 commit comments

Comments
 (0)