Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to content

Commit 2a0c81a

Browse files
hlinnakahlinnaka
committed
Add support for include_dir in config file.
This allows easily splitting configuration into many files, deployed in a directory. Magnus Hagander, Greg Smith, Selena Deckelmann, reviewed by Noah Misch.
1 parent ce9eee3 commit 2a0c81a

File tree

4 files changed

+290
-62
lines changed

4 files changed

+290
-62
lines changed

‎doc/src/sgml/config.sgml

+117-33
Original file line numberDiff line numberDiff line change
@@ -79,38 +79,6 @@ shared_buffers = 128MB
7979
value, write either two quotes (preferred) or backslash-quote.
8080
</para>
8181

82-
<para>
83-
<indexterm>
84-
<primary><literal>include</></primary>
85-
<secondary>in configuration file</secondary>
86-
</indexterm>
87-
In addition to parameter settings, the <filename>postgresql.conf</>
88-
file can contain <firstterm>include directives</>, which specify
89-
another file to read and process as if it were inserted into the
90-
configuration file at this point. This feature allows a configuration
91-
file to be divided into physically separate parts.
92-
Include directives simply look like:
93-
<programlisting>
94-
include 'filename'
95-
</programlisting>
96-
If the file name is not an absolute path, it is taken as relative to
97-
the directory containing the referencing configuration file.
98-
Inclusions can be nested.
99-
</para>
100-
101-
<para>
102-
<indexterm>
103-
<primary><literal>include_if_exists</></primary>
104-
<secondary>in configuration file</secondary>
105-
</indexterm>
106-
There is also an <literal>include_if_exists</> directive, which acts
107-
the same as the <literal>include</> directive, except for the behavior
108-
when the referenced file does not exist or cannot be read. A regular
109-
<literal>include</> will consider this an error condition, but
110-
<literal>include_if_exists</> merely logs a message and continues
111-
processing the referencing configuration file.
112-
</para>
113-
11482
<para>
11583
<indexterm>
11684
<primary>SIGHUP</primary>
@@ -213,7 +181,123 @@ SET ENABLE_SEQSCAN TO OFF;
213181
</para>
214182

215183
</sect2>
216-
</sect1>
184+
185+
<sect2 id="config-includes">
186+
<title>Configuration File Includes</title>
187+
188+
<para>
189+
<indexterm>
190+
<primary><literal>include</></primary>
191+
<secondary>in configuration file</secondary>
192+
</indexterm>
193+
In addition to parameter settings, the <filename>postgresql.conf</>
194+
file can contain <firstterm>include directives</>, which specify
195+
another file to read and process as if it were inserted into the
196+
configuration file at this point. This feature allows a configuration
197+
file to be divided into physically separate parts.
198+
Include directives simply look like:
199+
<programlisting>
200+
include 'filename'
201+
</programlisting>
202+
If the file name is not an absolute path, it is taken as relative to
203+
the directory containing the referencing configuration file.
204+
Inclusions can be nested.
205+
</para>
206+
207+
<para>
208+
<indexterm>
209+
<primary><literal>include_if_exists</></primary>
210+
<secondary>in configuration file</secondary>
211+
</indexterm>
212+
There is also an <literal>include_if_exists</> directive, which acts
213+
the same as the <literal>include</> directive, except for the behavior
214+
when the referenced file does not exist or cannot be read. A regular
215+
<literal>include</> will consider this an error condition, but
216+
<literal>include_if_exists</> merely logs a message and continues
217+
processing the referencing configuration file.
218+
</para>
219+
220+
<para>
221+
<indexterm>
222+
<primary><literal>include_dir</></primary>
223+
<secondary>in configuration file</secondary>
224+
</indexterm>
225+
The <filename>postgresql.conf</> file can also contain
226+
<firstterm>include_dir directives</>, which specify an entire directory
227+
of configuration files to include. It is used similarly:
228+
<programlisting>
229+
include_dir 'directory'
230+
</programlisting>
231+
Non-absolute directory names follow the same rules as single file include
232+
directives: they are relative to the directory containing the referencing
233+
configuration file. Within that directory, only non-directory files whose
234+
names end with the suffix <literal>.conf</literal> will be included. File
235+
names that start with the <literal>.</literal> character are also excluded,
236+
to prevent mistakes as they are hidden on some platforms. Multiple files
237+
within an include directory are processed in filename order. The filenames
238+
are ordered by C locale rules, ie. numbers before letters, and uppercase
239+
letters before lowercase ones.
240+
</para>
241+
242+
<para>
243+
Include files or directories can be used to logically separate portions
244+
of the database configuration, rather than having a single large
245+
<filename>postgresql.conf</> file. Consider a company that has two
246+
database servers, each with a different amount of memory. There are likely
247+
elements of the configuration both will share, for things such as logging.
248+
But memory-related parameters on the server will vary between the two. And
249+
there might be server specific customizations, too. One way to manage this
250+
situation is to break the custom configuration changes for your site into
251+
three files. You could add this to the end of your
252+
<filename>postgresql.conf</> file to include them:
253+
<programlisting>
254+
include 'shared.conf'
255+
include 'memory.conf'
256+
include 'server.conf'
257+
</programlisting>
258+
All systems would have the same <filename>shared.conf</>. Each server
259+
with a particular amount of memory could share the same
260+
<filename>memory.conf</>; you might have one for all servers with 8GB of RAM,
261+
another for those having 16GB. And finally <filename>server.conf</> could
262+
have truly server-specific configuration information in it.
263+
</para>
264+
265+
<para>
266+
Another possibility is to create a configuration file directory and
267+
put this information into files there. For example, a <filename>conf.d</>
268+
directory could be referenced at the end of<filename>postgresql.conf</>:
269+
<screen>
270+
include_dir 'conf.d'
271+
</screen>
272+
Then you could name the files in the <filename>conf.d</> directory like this:
273+
<screen>
274+
00shared.conf
275+
01memory.conf
276+
02server.conf
277+
</screen>
278+
This shows a clear order in which these files will be loaded. This is
279+
important because only the last setting encountered when the server is
280+
reading its configuration will be used. Something set in
281+
<filename>conf.d/02server.conf</> in this example would override a value
282+
set in <filename>conf.d/01memory.conf</>.
283+
</para>
284+
285+
<para>
286+
You might instead use this configuration directory approach while naming
287+
these files more descriptively:
288+
<screen>
289+
00shared.conf
290+
01memory-8GB.conf
291+
02server-foo.conf
292+
</screen>
293+
This sort of arrangement gives a unique name for each configuration file
294+
variation. This can help eliminate ambiguity when several servers have
295+
their configurations all stored in one place, such as in a version
296+
control repository. (Storing database configuration files under version
297+
control is another good practice to consider).
298+
</para>
299+
</sect2>
300+
</sect1>
217301

218302
<sect1 id="runtime-config-file-locations">
219303
<title>File Locations</title>

‎src/backend/utils/misc/guc-file.l

+157-29
Original file line numberDiff line numberDiff line change
@@ -362,6 +362,39 @@ ProcessConfigFile(GucContext context)
362362
}
363363
}
364364

365+
/*
366+
* Given a configuration file or directory location that may be a relative
367+
* path, return an absolute one. We consider the location to be relative to
368+
* the directory holding the calling file.
369+
*/
370+
static char *
371+
AbsoluteConfigLocation(const char *location, const char *calling_file)
372+
{
373+
char abs_path[MAXPGPATH];
374+
375+
if (is_absolute_path(location))
376+
return pstrdup(location);
377+
else
378+
{
379+
if (calling_file != NULL)
380+
{
381+
strlcpy(abs_path, calling_file, sizeof(abs_path));
382+
get_parent_directory(abs_path);
383+
join_path_components(abs_path, abs_path, location);
384+
canonicalize_path(abs_path);
385+
}
386+
else
387+
{
388+
/*
389+
* calling_file is NULL, we make an absolute path from $PGDATA
390+
*/
391+
join_path_components(abs_path, data_directory, location);
392+
canonicalize_path(abs_path);
393+
}
394+
return pstrdup(abs_path);
395+
}
396+
}
397+
365398
/*
366399
* Read and parse a single configuration file. This function recurses
367400
* to handle "include" directives.
@@ -378,7 +411,6 @@ ParseConfigFile(const char *config_file, const char *calling_file, bool strict,
378411
{
379412
bool OK = true;
380413
FILE *fp;
381-
char abs_path[MAXPGPATH];
382414

383415
/*
384416
* Reject too-deep include nesting depth. This is just a safety check
@@ -394,31 +426,7 @@ ParseConfigFile(const char *config_file, const char *calling_file, bool strict,
394426
return false;
395427
}
396428

397-
/*
398-
* If config_file is a relative path, convert to absolute. We consider
399-
* it to be relative to the directory holding the calling file.
400-
*/
401-
if (!is_absolute_path(config_file))
402-
{
403-
if (calling_file != NULL)
404-
{
405-
strlcpy(abs_path, calling_file, sizeof(abs_path));
406-
get_parent_directory(abs_path);
407-
join_path_components(abs_path, abs_path, config_file);
408-
canonicalize_path(abs_path);
409-
config_file = abs_path;
410-
}
411-
else
412-
{
413-
/*
414-
* calling_file is NULL, we make an absolute path from $PGDATA
415-
*/
416-
join_path_components(abs_path, data_directory, config_file);
417-
canonicalize_path(abs_path);
418-
config_file = abs_path;
419-
}
420-
}
421-
429+
config_file = AbsoluteConfigLocation(config_file,calling_file);
422430
fp = AllocateFile(config_file, "r");
423431
if (!fp)
424432
{
@@ -563,20 +571,35 @@ ParseConfigFp(FILE *fp, const char *config_file, int depth, int elevel,
563571
}
564572

565573
/* OK, process the option name and value */
566-
if (guc_name_compare(opt_name, "include_if_exists") == 0)
574+
if (guc_name_compare(opt_name, "include_dir") == 0)
567575
{
568576
/*
569-
* An include_if_exists directive isn't a variable and should be
577+
* An include_dir directive isn't a variable and should be
570578
* processed immediately.
571579
*/
572-
if (!ParseConfigFile(opt_value, config_file, false,
580+
if (!ParseConfigDirectory(opt_value, config_file,
573581
depth + 1, elevel,
574582
head_p, tail_p))
575583
OK = false;
576584
yy_switch_to_buffer(lex_buffer);
585+
ConfigFileLineno = save_ConfigFileLineno;
577586
pfree(opt_name);
578587
pfree(opt_value);
579588
}
589+
else if (guc_name_compare(opt_name, "include_if_exists") == 0)
590+
{
591+
/*
592+
* An include_if_exists directive isn't a variable and should be
593+
* processed immediately.
594+
*/
595+
if (!ParseConfigFile(opt_value, config_file, false,
596+
depth + 1, elevel,
597+
head_p, tail_p))
598+
OK = false;
599+
yy_switch_to_buffer(lex_buffer);
600+
pfree(opt_name);
601+
pfree(opt_value);
602+
}
580603
else if (guc_name_compare(opt_name, "include") == 0)
581604
{
582605
/*
@@ -665,6 +688,111 @@ cleanup:
665688
return OK;
666689
}
667690

691+
/*
692+
* Read and parse all config files in a subdirectory in alphabetical order
693+
*/
694+
bool
695+
ParseConfigDirectory(const char *includedir,
696+
const char *calling_file,
697+
int depth, int elevel,
698+
ConfigVariable **head_p,
699+
ConfigVariable **tail_p)
700+
{
701+
char *directory;
702+
DIR *d;
703+
struct dirent *de;
704+
char **filenames = NULL;
705+
int num_filenames = 0;
706+
int size_filenames = 0;
707+
bool status;
708+
709+
directory = AbsoluteConfigLocation(includedir, calling_file);
710+
d = AllocateDir(directory);
711+
if (d == NULL)
712+
{
713+
ereport(elevel,
714+
(errcode_for_file_access(),
715+
errmsg("could not open configuration directory \"%s\": %m",
716+
directory)));
717+
return false;
718+
}
719+
720+
/*
721+
* Read the directory and put the filenames in an array, so we can sort
722+
* them prior to processing the contents.
723+
*/
724+
while ((de = ReadDir(d, directory)) != NULL)
725+
{
726+
struct stat st;
727+
char filename[MAXPGPATH];
728+
729+
/*
730+
* Only parse files with names ending in ".conf". Explicitly reject
731+
* files starting with ".". This excludes things like "." and "..",
732+
* as well as typical hidden files, backup files, and editor debris.
733+
*/
734+
if (strlen(de->d_name) < 6)
735+
continue;
736+
if (de->d_name[0] == '.')
737+
continue;
738+
if (strcmp(de->d_name + strlen(de->d_name) - 5, ".conf") != 0)
739+
continue;
740+
741+
join_path_components(filename, directory, de->d_name);
742+
canonicalize_path(filename);
743+
if (stat(filename, &st) == 0)
744+
{
745+
if (!S_ISDIR(st.st_mode))
746+
{
747+
/* Add file to list, increasing its size in blocks of 32 */
748+
if (num_filenames == size_filenames)
749+
{
750+
size_filenames += 32;
751+
if (num_filenames == 0)
752+
/* Must initialize, repalloc won't take NULL input */
753+
filenames = palloc(size_filenames * sizeof(char *));
754+
else
755+
filenames = repalloc(filenames, size_filenames * sizeof(char *));
756+
}
757+
filenames[num_filenames] = pstrdup(filename);
758+
num_filenames++;
759+
}
760+
}
761+
else
762+
{
763+
/*
764+
* stat does not care about permissions, so the most likely reason
765+
* a file can't be accessed now is if it was removed between the
766+
* directory listing and now.
767+
*/
768+
ereport(elevel,
769+
(errcode_for_file_access(),
770+
errmsg("could not stat file \"%s\": %m",
771+
filename)));
772+
return false;
773+
}
774+
}
775+
776+
if (num_filenames > 0)
777+
{
778+
int i;
779+
qsort(filenames, num_filenames, sizeof(char *), pg_qsort_strcmp);
780+
for (i = 0; i < num_filenames; i++)
781+
{
782+
if (!ParseConfigFile(filenames[i], NULL, true,
783+
depth, elevel, head_p, tail_p))
784+
{
785+
status = false;
786+
goto cleanup;
787+
}
788+
}
789+
}
790+
status = true;
791+
792+
cleanup:
793+
FreeDir(d);
794+
return status;
795+
}
668796

669797
/*
670798
* Free a list of ConfigVariables, including the names and the values

0 commit comments

Comments
 (0)