diff --git a/doc/src/sgml/datatype.sgml b/doc/src/sgml/datatype.sgml index 09309ba0390b..53fd7fe3fa11 100644 --- a/doc/src/sgml/datatype.sgml +++ b/doc/src/sgml/datatype.sgml @@ -5389,7 +5389,7 @@ WHERE ... unknown Identifies a not-yet-resolved type, e.g., of an undecorated - string literal. + string literal. Also, the null value. diff --git a/doc/src/sgml/ddl.sgml b/doc/src/sgml/ddl.sgml index fcd1cb85352f..d88db25b0d0b 100644 --- a/doc/src/sgml/ddl.sgml +++ b/doc/src/sgml/ddl.sgml @@ -168,6 +168,8 @@ DROP TABLE products; + &nullvalues; + Default Values diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml index fef9584f908e..53b70f31e9bd 100644 --- a/doc/src/sgml/filelist.sgml +++ b/doc/src/sgml/filelist.sgml @@ -21,6 +21,7 @@ + diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 574a544d9fa4..d6a5f71bf6f7 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -23790,7 +23790,8 @@ MERGE INTO products p This section describes the SQL-compliant subquery expressions available in PostgreSQL. All of the expression forms documented in this section return - Boolean (true/false) results. + three-valued typed + results (true, false, or null). @@ -23852,19 +23853,17 @@ WHERE EXISTS (SELECT 1 FROM tab2 WHERE col2 = tab1.col2); The right-hand side is a parenthesized - subquery, which must return exactly one column. The left-hand expression + subquery, which must return exactly one column. The result of IN + is false if the subquery returns no rows, otherwise the left-hand expression is evaluated and compared to each row of the subquery result. - The result of IN is true if any equal subquery row is found. - The result is false if no equal row is found (including the - case where the subquery returns no rows). + The result is true if any equal subquery row is found. + The result is false if no equal row is found. - Note that if the left-hand expression yields null, or if there are - no equal right-hand values and at least one right-hand row yields - null, the result of the IN construct will be null, not false. - This is in accordance with SQL's normal rules for Boolean combinations - of null values. + As explained in , it is not possible to see + a false result in the presence of both rows and null values since the multiple equality + tests are AND'd together. @@ -23881,21 +23880,18 @@ WHERE EXISTS (SELECT 1 FROM tab2 WHERE col2 = tab1.col2); as described in . The right-hand side is a parenthesized subquery, which must return exactly as many columns as there are - expressions in the left-hand row. The left-hand expressions are + expressions in the left-hand row. + The result of IN is false if the subquery returns no rows, + otherwise the left-hand expressions are evaluated and compared row-wise to each row of the subquery result. - The result of IN is true if any equal subquery row is found. - The result is false if no equal row is found (including the - case where the subquery returns no rows). + The result is true if any equal subquery row is found. + The result is false if no equal row is found. - As usual, null values in the rows are combined per - the normal rules of SQL Boolean expressions. Two rows are considered - equal if all their corresponding members are non-null and equal; the rows - are unequal if any corresponding members are non-null and unequal; - otherwise the result of that row comparison is unknown (null). - If all the per-row results are either unequal or null, with at least one - null, then the result of IN is null. + As explained in , it is not possible to see + a false result in the presence of both rows and null values since the multiple equality + tests are OR'd together. @@ -23907,20 +23903,17 @@ WHERE EXISTS (SELECT 1 FROM tab2 WHERE col2 = tab1.col2); - The right-hand side is a parenthesized - subquery, which must return exactly one column. The left-hand expression - is evaluated and compared to each row of the subquery result. - The result of NOT IN is true if only unequal subquery rows - are found (including the case where the subquery returns no rows). + The right-hand side is a parenthesized subquery, which must return exactly one column. + The result of NOT IN is true if the subquery returns no rows, + otherwise the left-hand expression is evaluated and compared to each row of the subquery result. + The result is true if only unequal subquery rows are found. The result is false if any equal row is found. - Note that if the left-hand expression yields null, or if there are - no equal right-hand values and at least one right-hand row yields - null, the result of the NOT IN construct will be null, not true. - This is in accordance with SQL's normal rules for Boolean combinations - of null values. + As explained in , it is not possible to see + a true result in the presence of both rows and null values since the multiple inequality + tests are OR'd together. @@ -23937,21 +23930,18 @@ WHERE EXISTS (SELECT 1 FROM tab2 WHERE col2 = tab1.col2); as described in . The right-hand side is a parenthesized subquery, which must return exactly as many columns as there are - expressions in the left-hand row. The left-hand expressions are + expressions in the left-hand row. + The result of NOT IN is true if the subquery returns no rows, + otherwise the left-hand expressions are evaluated and compared row-wise to each row of the subquery result. - The result of NOT IN is true if only unequal subquery rows - are found (including the case where the subquery returns no rows). + The result is true if only unequal subquery rows are found. The result is false if any equal row is found. - As usual, null values in the rows are combined per - the normal rules of SQL Boolean expressions. Two rows are considered - equal if all their corresponding members are non-null and equal; the rows - are unequal if any corresponding members are non-null and unequal; - otherwise the result of that row comparison is unknown (null). - If all the per-row results are either unequal or null, with at least one - null, then the result of NOT IN is null. + As explained in , it is not possible to see + a true result in the presence of both rows and null values since the multiple inequality + tests are OR'd together. @@ -23965,13 +23955,13 @@ WHERE EXISTS (SELECT 1 FROM tab2 WHERE col2 = tab1.col2); The right-hand side is a parenthesized - subquery, which must return exactly one column. The left-hand expression + subquery, which must return exactly one column. The result of ANY is + false if the subquery returns no rows, otherwise the left-hand expression is evaluated and compared to each row of the subquery result using the given operator, which must yield a Boolean result. - The result of ANY is true if any true result is obtained. - The result is false if no true result is found (including the - case where the subquery returns no rows). + The result is true if any true result is obtained. + The result is false if no true result is found. @@ -23980,11 +23970,10 @@ WHERE EXISTS (SELECT 1 FROM tab2 WHERE col2 = tab1.col2); - Note that if there are no successes and at least one right-hand row yields - null for the operator's result, the result of the ANY construct - will be null, not false. - This is in accordance with SQL's normal rules for Boolean combinations - of null values. + If operator can produce null valued booleans then, as explained in + , it is not possible to see + a false result in the presence of both rows and null values since the multiple equality + tests are AND'd together. Note that IS DISTINCT FROM is not an operator. @@ -24002,16 +23991,19 @@ WHERE EXISTS (SELECT 1 FROM tab2 WHERE col2 = tab1.col2); as described in . The right-hand side is a parenthesized subquery, which must return exactly as many columns as there are - expressions in the left-hand row. The left-hand expressions are + expressions in the left-hand row. The result of ANY is + false if the subquery returns no rows, otherwise the left-hand expressions are evaluated and compared row-wise to each row of the subquery result, using the given operator. - The result of ANY is true if the comparison - returns true for any subquery row. - The result is false if the comparison returns false for every - subquery row (including the case where the subquery returns no - rows). - The result is NULL if no comparison with a subquery row returns true, - and at least one comparison returns NULL. + The result is true if the comparison returns true for any subquery row. + The result is false if the comparison returns false for every subquery row. + + + + If operator can produce null valued booleans then, as explained in + , it is not possible to see + a false result in the presence of both rows and null values since the multiple equality + tests are OR'd together. Note that IS DISTINCT FROM is not an operator. @@ -24029,15 +24021,20 @@ WHERE EXISTS (SELECT 1 FROM tab2 WHERE col2 = tab1.col2); The right-hand side is a parenthesized - subquery, which must return exactly one column. The left-hand expression + subquery, which must return exactly one column. The result of ALL is + true if the subquery returns no rows, otherwise the left-hand expression is evaluated and compared to each row of the subquery result using the given operator, which must yield a Boolean result. - The result of ALL is true if all rows yield true - (including the case where the subquery returns no rows). + The result is true if all rows yield true. The result is false if any false result is found. - The result is NULL if no comparison with a subquery row returns false, - and at least one comparison returns NULL. + + + + If operator can produce null valued booleans then, as explained in + , it is not possible to see + a true result in the presence of both rows and null values since the multiple equality + tests are AND'd together. Note that IS DISTINCT FROM is not an operator. @@ -24058,22 +24055,21 @@ WHERE EXISTS (SELECT 1 FROM tab2 WHERE col2 = tab1.col2); as described in . The right-hand side is a parenthesized subquery, which must return exactly as many columns as there are - expressions in the left-hand row. The left-hand expressions are + expressions in the left-hand row. The result of ALL is + true if the subquery returns no rows, otherwise the left-hand expressions are evaluated and compared row-wise to each row of the subquery result, using the given operator. - The result of ALL is true if the comparison - returns true for all subquery rows (including the - case where the subquery returns no rows). - The result is false if the comparison returns false for any - subquery row. - The result is NULL if no comparison with a subquery row returns false, - and at least one comparison returns NULL. + The result is true if the comparison returns true for all subquery rows. + The result is false if the comparison returns false for any subquery row. - See for details about the meaning - of a row constructor comparison. + If operator can produce null valued booleans then, as explained in + , it is not possible to see + a true result in the presence of both rows and null values since the multiple equality + tests are AND'd together. Note that IS DISTINCT FROM is not an operator. + @@ -24098,6 +24094,14 @@ WHERE EXISTS (SELECT 1 FROM tab2 WHERE col2 = tab1.col2); compared row-wise to the single subquery result row. + + If operator can produce null valued booleans then, as explained in + , the result cannot be true in the + presence of null valued fields in either the row constructor or the subquery result row, as + the individual field tests are AND'd together. + Note that IS DISTINCT FROM is not an operator. + + See for details about the meaning of a row constructor comparison. @@ -24165,7 +24169,8 @@ WHERE EXISTS (SELECT 1 FROM tab2 WHERE col2 = tab1.col2); PostgreSQL extensions; the rest are SQL-compliant. All of the expression forms documented in this section return - Boolean (true/false) results. + three-valued boolean typed + results (true, false, or null). @@ -24178,24 +24183,13 @@ WHERE EXISTS (SELECT 1 FROM tab2 WHERE col2 = tab1.col2); The right-hand side is a parenthesized list of expressions. The result is true if the left-hand expression's - result is equal to any of the right-hand expressions. This is a shorthand - notation for - - -expression = value1 -OR -expression = value2 -OR -... - + result is equal to any of the right-hand expressions. - Note that if the left-hand expression yields null, or if there are - no equal right-hand values and at least one right-hand expression yields - null, the result of the IN construct will be null, not false. - This is in accordance with SQL's normal rules for Boolean combinations - of null values. + As explained in , it is not possible to see + a false result in the presence null values since the multiple equality + tests are OR'd together. @@ -24209,35 +24203,15 @@ OR The right-hand side is a parenthesized list of expressions. The result is true if the left-hand expression's - result is unequal to all of the right-hand expressions. This is a shorthand - notation for - - -expression <> value1 -AND -expression <> value2 -AND -... - + result is unequal to all of the right-hand expressions. - Note that if the left-hand expression yields null, or if there are - no equal right-hand values and at least one right-hand expression yields - null, the result of the NOT IN construct will be null, not true - as one might naively expect. - This is in accordance with SQL's normal rules for Boolean combinations - of null values. + As explained in , it is not possible to see + a true result in the presence of null values since the multiple inequality + tests are OR'd together. - - - x NOT IN y is equivalent to NOT (x IN y) in all - cases. However, null values are much more likely to trip up the novice when - working with NOT IN than when working with IN. - It is best to express your condition positively if possible. - - @@ -24250,30 +24224,26 @@ AND The right-hand side is a parenthesized expression, which must yield an - array value. - The left-hand expression + array value. The result of ANY is + false if the array has zero element, otherwise + the left-hand expression is evaluated and compared to each element of the array using the given operator, which must yield a Boolean result. - The result of ANY is true if any true result is obtained. - The result is false if no true result is found (including the - case where the array has zero elements). + The result is true if any true result is obtained. + The result is false if no true result is found. - If the array expression yields a null array, the result of - ANY will be null. If the left-hand expression yields null, - the result of ANY is ordinarily null (though a non-strict - comparison operator could possibly yield a different result). - Also, if the right-hand array contains any null elements and no true - comparison result is obtained, the result of ANY - will be null, not false (again, assuming a strict comparison operator). - This is in accordance with SQL's normal rules for Boolean combinations - of null values. + If operator can produce null valued booleans then, as explained in + , it is not possible to see + a false result in the presence of both elements and null values since the multiple equality + tests are AND'd together. Note that IS DISTINCT FROM is not an operator. SOME is a synonym for ANY. + IN is equivalent to = ANY. @@ -24287,26 +24257,27 @@ AND The right-hand side is a parenthesized expression, which must yield an array value. - The left-hand expression + The result of ALL is + true if the array has zero elements, otherwise + the left-hand expression is evaluated and compared to each element of the array using the given operator, which must yield a Boolean result. - The result of ALL is true if all comparisons yield true - (including the case where the array has zero elements). + The result is true if all comparisons yield true. The result is false if any false result is found. - If the array expression yields a null array, the result of - ALL will be null. If the left-hand expression yields null, - the result of ALL is ordinarily null (though a non-strict - comparison operator could possibly yield a different result). - Also, if the right-hand array contains any null elements and no false - comparison result is obtained, the result of ALL - will be null, not true (again, assuming a strict comparison operator). - This is in accordance with SQL's normal rules for Boolean combinations - of null values. + If operator can produce null valued booleans then, as explained in + , it is not possible to see + a true result in the presence of both elements and null values since the multiple equality + tests are AND'd together. Note that IS DISTINCT FROM is not an operator. + + + NOT IN is equivalent to <> ALL. + + @@ -24357,6 +24328,11 @@ AND considered. + + See + and surrounding content for additional details and examples. + + row_constructor IS DISTINCT FROM row_constructor @@ -24391,20 +24367,11 @@ AND - The SQL specification requires row-wise comparison to return NULL if the - result depends on comparing two NULL values or a NULL and a non-NULL. - PostgreSQL does this only when comparing the - results of two row constructors (as in - ) or comparing a row constructor - to the output of a subquery (as in ). - In other contexts where two composite-type values are compared, two - NULL field values are considered equal, and a NULL is considered larger - than a non-NULL. This is necessary in order to have consistent sorting - and indexing behavior for composite types. - - - - Each side is evaluated and they are compared row-wise. Composite type + Each side is evaluated and they are compared row-wise. + As discussed and shown in , + null values are treated as being equal to other null values and greater + than all non-null values. + Composite type comparisons are allowed when the operator is =, <>, diff --git a/doc/src/sgml/json.sgml b/doc/src/sgml/json.sgml index 206eadb8f7ba..0f39d4bf21a0 100644 --- a/doc/src/sgml/json.sgml +++ b/doc/src/sgml/json.sgml @@ -129,6 +129,11 @@ the corresponding PostgreSQL types. + + null value + within JSON + + JSON Primitive Types and Corresponding <productname>PostgreSQL</productname> Types @@ -162,7 +167,7 @@ null (none) - SQL NULL is a different concept + An SQL null value is similar, but see for differences. diff --git a/doc/src/sgml/nullvalues.sgml b/doc/src/sgml/nullvalues.sgml new file mode 100644 index 000000000000..29ece221ffb8 --- /dev/null +++ b/doc/src/sgml/nullvalues.sgml @@ -0,0 +1,1124 @@ + + Null Values Overview + + + null value + + + + This section first introduces the concept of null values and then + explains how different parts of the system behave when provided + one or more null value inputs. Examples throughout this section + can be executed so long as the following table and rows are created first. + + + + Throughout this section, the discussion of null values will be limited to + the SQL language unless otherwise noted. The JSON-related data types, and the + non-SQL procedural languages, have their own behaviors documented in their + respective areas. + + + + The following CREATE TABLE and INSERT + SQL commands can be executed in any SQL client to create and populate + the persistent table used in the examples below. The \pset + commands require the use of psql as the client program; + they make the resulting output a bit easier to read and do not impact any behaviors + described herein. Note, the examples below have been manually edited to show + true and false instead of + t and f. They also omit any transactional + command output when transactions are used. Instead, each transaction gets its own + display block. + + + +CREATE TABLE null_examples ( + id bigint PRIMARY KEY, + value integer NULL +); +INSERT INTO null_examples +VALUES (1, 1), (2, NULL), (3, 4); + +-- This makes null values print as \N in the output instead of the empty string. +\pset null '\\N' +-- Removes the row count footer that prints by default. +\pset footer off + + + + Meaning + + Generally, a null value is assumed to mean "unknown", but other interpretations + are common. A data model design may state that a null value + is to be used to represent "not applicable" - i.e., that a value is not + even possible. The null value also takes on a literal meaning of "not found" + when produced as the result of an outer join. + + + In different programming languages, some of which are accessible in the server + as procedural languages, the null value is represented in other ways. + Of those included in PostgreSQL, + these are: + None in Python, + undefined in Perl, + and the empty string in TCL. + + + + + Usage + + A null value, like all values, must have a data type, and is valid for all data types. + It must also be printed as text. This section discusses null values at the boundaries + of the system, as well as how they can come into existence due to the design of a query. + + + Null Value Input + + A null value can be used as input to any function, operator, or expression. + The system will then decide how to behave based on the rules described in the + rest of this section. + + + As noted in , + a null value literal is written using the NULL keyword. + Its type is the pseudo-type unknown + but can be cast to any concrete data type. + + + +SELECT + NULL AS "Literal Null Value", + pg_typeof(null) AS "Type of Null", + pg_typeof(NULL::text) AS "Type of Cast Null", + cast(null as text) AS "Cast Null Value"; + + + Literal Null Value | Type of Null | Type of Cast Null | Cast Null Value +--------------------+--------------+-------------------+----------------- + \N | unknown | text | \N + + + + +SELECT text NULL; + + +ERROR: column "text" does not exist +LINE 1: select text NUll; + + + + The COPY ... FROM command, + including its psql counter-part meta-command + \copy, + must deal with input files containing textual representations of the null value. + The lack of consistency in real-world data requires having a few options to the + command related to null handling. See the documentation in + for more information. + But, in short, for CSV input it expects text to be quoted and interprets an unquoted + empty string as the null value. + + + + Null Values in Tables + + From a semantics perspective a table treats a null value like any other value. + However, the SQL Language recognizes its uniqueness by defining a column-scoped + NOT NULL constraint + as syntactic sugar. At present one would expect that a column having a + domain data type with a NOT NULL constraint would likewise be + incapable of having a null value stored. This is not the case. See the commentary + below in for more information. + + + + Derived Null Values + + Even if all data stored in tables are known to be non-null, null values can still + be produced while executing a query. The most common way this happens is by + introducing a (left) outer join to the query and having left side data without + corresponding data on the right side of the join. + +SELECT + countries.country, + flagships.flagship +FROM ( + VALUES ('Spain'), ('Switzerland') +) as countries (country) +LEFT JOIN ( + VALUES ('Spain', 'Ship') +) as flagships (country, flagship) +ON countries.country = flagships.country; + + + country | flagship +-------------+---------- + Spain | Ship + Switzerland | \N + + + + + Null Value Output + + As evidenced above, the "absence of value" aspect of the null value results + in its secondary textual representation being an empty string + (its primary representation is just NULL). + This can be problematic if the empty string is expected to be a valid value. + Therefore, places that deal with possible null values as input and text as + output need some means to give the user a way to specify how to print + the null value. + + + Generally, the primary representation is used when the value is part of a multi-element value. + If the value is being displayed by itself the secondary (blank) representation is used. The settings + discussed herein typically control the secondary representation. The null value representation when it + is within a container type (composite, array, etc...) is controlled by the input and output rules of the + container type. It is when the container value itself is the null value that these generalities then apply. + + + No matter how the null value got into the result when presenting results to the user it is + necessary to present null values using text. This is the responsibility of the client application. + The psql client program has the + \pset null meta-command to specify the textual output of null values + it encounters in query results. + + + When the final output of the result is a text file instead of a user additional + considerations come into play. While the option to take the user presentation + and send it to a text file always exists PostgreSQL also + provides a facility to output a structured text file. + The COPY ... TO command, + including its psql counter-part meta-command + \copy, + has the NULL option (and some modifier options) to specify + the string to print to the output for null values it encounters in the query result. + As with input file processing, for the CSV format it will, by default, + produce an unquoted empty string for the null value. + + + + Null Value Handling + + The presence of null values in the system results in three-valued logic. + In conventional two-valued (binary) logic every outcome is either true or false. + In three-valued logic the concept of unknown, represented using the null value, is + also an outcome. This results in falsifying the common-sense notion + that "p OR NOT p" is always true. + +SELECT + NULL OR NOT NULL AS "N OR !N"; + + + N OR !N +--------- + \N + + (See for more explanation.) + + + When dealing with null values it is often useful to explicitly to convert + data to and from a null value given a known non-null representation + (e.g., the empty string, the numbers 0 or 1, or boolean false). + The COALESCE and + NULLIF functions are useful + for this purpose. + + + + + + Distinctness - Overcoming the Cardinal Rule of Null Values + + The cardinal rule, a null value is + + neither equal nor unequal + + to any value, including other null values. + +SELECT + NULL = NULL AS "N = N", + NULL != NULL AS "N != N", + 1 = NULL AS "1 = N", + 1 != NULL AS "1 != N", + 1 = 1 AS "1 = 1", + 1 != 1 AS "1 != 1"; + + + N = N | N != N | 1 = N | 1 != N | 1 = 1 | 1 != 1 +-------+--------+-------+--------+-------+-------- + \N | \N | \N | \N | true | false + + However, as with many rules, there are exceptions, as noted in + . + Particularly, when the two compared values are part of a larger multi-element value. + +SELECT + array[1,2]=array[1,null] AS "Array Equals"; + + + Array Equals +-------------- + false + + + + Because of this SQL standard rule, checking for a null value has an + explicit IS NULL predicate. Additionally, there are comparison + predicates that consider a null value equal to other null values but unequal + to any other value (e.g., IS DISTINCT, and IS TRUE.) + These, and other predicates, are described in + + +SELECT id, value, + value IS NULL AS "IS NULL", + value IS DISTINCT FROM id AS "IS DIST", + value != id AS "IS !=" +FROM null_examples; + + + id | value | IS NULL | IS DIST | IS != +----+-------+---------+---------+------- + 1 | 1 | false | false | false + 2 | \N | true | true | \N + 3 | 4 | false | true | true + + + + On the other hand, the SQL standard is largely alone in taking this approach to comparing + values to the null value. For example, when working within the JSON data types the use of equals + produces true or false and so the concept of distinctness is neither present nor required. + Additional details and links are provided in . + For the non-SQL procedural languages, please consult the appropriate documentation. + + + There is also a cardinal warning: when dealing with + composite types in + expressions, composite IS NULL + and composite IS NOT NULL + are not the opposites of each other in the case where some, + but not all, of the composite's fields are null values. + (The case where all fields are null is indistinguishable + from the composite as a whole being null.) + Write NOT(composite IS NULL) instead. + +SELECT + c, + c IS NULL AS "c IS N", + NOT(c IS NULL) AS "NOT c IS N", + c IS NOT NULL AS "c IS NOT N", + ROW(value, value) IS NULL AS "ROW(v,v) IS N", + ROW(value, value) IS NOT NULL AS "ROW(v,v) IS NOT N" +FROM null_examples AS c; + + + c | c IS N | NOT c IS N | c IS NOT N | ROW(v,v) IS N | ROW(v,v) IS NOT N +-------+--------+------------+------------+---------------+------------------- + (1,1) | false | true | true | false | true + (2,) | false | true | false | true | false + (3,4) | false | true | true | false | true + + See below for an explanation. + + + + + Null-Valued Operands + + As a general expectation, operator invocation expressions where one of the inputs + is a null value will result in a null-valued output. + +SELECT + 1 + null AS "Add", + 'text' || null AS "Concatenate"; + + + Add | Concatenate +-----+------------- + \N | \N + + Operators that behave otherwise should document their deviation from this norm. + + + A notable example is the IN operator, which + uses equality, not distinctness, for testing. + +SELECT + 1 IN (1, null) AS "In Present", + 1 IN (2, null) AS "In Missing", + null IN (1, 2) AS "N In Non-N", + null IN (null, 2) AS "N In N"; + + + In Present | In Missing | N In Non-N | N In N +------------+------------+------------+-------- + true | \N | \N | \N + + This is just an extension of the multi-element testing behavior described in + . + + + Experience shows that CASE expressions are also prone + to bugs since their format encourages binary logic thinking while a + WHEN test will not consider a null value to be a match. + +SELECT id, value, + CASE WHEN id = value THEN 'Equal' ELSE 'Not Equal' END AS "Affirm", + CASE WHEN id != value THEN 'Not Equal' ELSE 'Equal' END AS "Negate", + CASE WHEN value IS NULL THEN 'Null' + WHEN id = value THEN 'Equal' + ELSE 'Not Equal' END AS "Safe Affirm", + CASE WHEN value IS NULL THEN 'Null' + WHEN id != value THEN 'Not Equal' + ELSE 'Equal' END AS "Safe Negate" +FROM null_examples; + + + id | value | Affirm | Negate | Safe Affirm | Safe Negate +----+-------+-----------+-----------+-------------+------------- + 1 | 1 | Equal | Equal | Equal | Equal + 2 | \N | Not Equal | Equal | Null | Null + 3 | 4 | Not Equal | Not Equal | Not Equal | Not Equal + + + + The boolean operators AND and OR + will ignore the null value input if the other input is sufficient to + determine the outcome. + +SELECT + true OR null AS "T or N", + false OR null AS "F or N", + true AND null AS "T and N", + false AND null AS "F and N"; + + + T or N | F or N | T and N | F and N +--------+--------+---------+--------- + true | \N | \N | false + + + + + + Null Values in Domains + + A domain is a user-defined data type that can have a NOT NULL + constraint. However, some usages of domains can still cause a column to be of the + domain type but some value may be null. The common way this happens is by including + the domain column's table on the right side of a left join. + +BEGIN; +CREATE DOMAIN domain_example AS integer NOT NULL; +CREATE TABLE domain_examples (de_id bigint PRIMARY KEY, de_value domain_example); +INSERT INTO domain_examples VALUES (1, 1), (2, 2); +SELECT *, pg_typeof(de_value) +FROM null_examples AS ne +LEFT JOIN domain_examples AS de ON ne.id = de.de_id; +ROLLBACK; + + +BEGIN +CREATE DOMAIN +CREATE TABLE +INSERT 0 2 + id | value | de_id | de_value | pg_typeof +----+-------+-------+----------+---------------- + 1 | 1 | 1 | 1 | domain_example + 2 | \N | 2 | 2 | domain_example + 3 | 4 | \N | \N | domain_example + +ROLLBACK + + Please see the details in + for another example, as well as commentary on why this non-standard behavior exists. + + + + + Testing Multi-Element Values with Null-Valued Elements + + Arrays and composite types are multi-element types. Here we also consider non-empty + subquery results + and the list of values (i.e., the multiset) specified in the + IN test. + + + When a test is performed on one of these multi-element values + the system will iterate over each element, (or pair of elements if the test is + comparing two row constructors to each other), + left-to-right, combining the results using the boolean operations described in + . For tests that + require an exhaustive search, (e.g., ALL, NOT IN) + the search effectively ends when a false result is found (AND combiners). + For tests that require a true result, (e.g., ANY, + IN) the search effectively ends when a true result is found + (OR combiners). Therefore: + + + IN and ANY + (OR) cannot produce a false result in the presence of null, and + + + NOT IN and ALL + (AND) cannot produce a true result in the presence of null. + + + This is because any exhaustive search will produce at least one null value result + that cannot be ignored. + + + The SQL standard requires that non-exhaustive + (i.e., IN and ANY) subquery tests + return false when there are no rows in the subquery result, and return true + for the exhaustive tests (i.e., NOT IN and ALL). + + + Note that the cardinal warning + noted in above is just the application of this behavior to the + IS NULL and IS NOT NULL + tests, which are both exhaustive search tests guaranteed to produce at least one false result + when the composite has a mix of null and non-null values. + + + In the next section, the rules above are discussed. + + discusses situations where a predicate or a scalar value + are being compared to a multi-element value. + In + the rules when two multi-element values are compared + to each other are discussed + (including the two row constructor comparison case.) + + + + + Multi-Element Predicates and Scalars + + Composite Fields + + When a composite typed value is created, a null value can be assigned to any + of its fields (see for how to do this). + So long as at least one field is non-null the composite value + as a whole exists and an IS NULL predicate will return false. + + + Applying the IS NOT NULL predicate to a composite value performs + checks on whether all fields of the composite have non-null values. This is not the same + as a non-null composite value. Specifically, if the composite value has + a null-valued field then both the IS NOT NULL predicate and the + IS NULL predicate will return false. + +SELECT + ROW(1,2) IS NULL AS "Row Is Null", + ROW(1,2) IS NOT NULL AS "Row Is Not Null", + ROW(1,NULL) IS NULL AS "Row Is Null", + ROW(1,NULL) IS NOT NULL AS "Row Is Not Null"; + + + Row Is Null | Row Is Not Null | Row Is Null | Row Is Not Null +-------------+-----------------+-------------+----------------- + false | true | false | false + + + + Please read for a complete treatment + on how PostgreSQL handles row-wise comparison. The + next two multi-element related items in this section discuss those comparisons in the + presence of null-valued fields, and also in terms of the SQL standard. + + + + Array Elements and IN Multiset Members + + Examples of applying the behavior discussed in + to arrays, and IN and NOT IN multisets, using the + operators defined in . The following examples produce + the same results when swapping IN/ANY + and also NOT IN/ALL, plus transforming the multiset/array format. + I.e., the exhaustive and non-exhaustive pairs noted in . + + + +SELECT + 1 = ANY(array[1, 1, NULL]) AS "Any-Null-Match", + 1 = ANY(array[1, 1]) AS "Any-NoNull-Match", + 1 = ALL(array[1, 1, NULL]) AS "ALL-Null-Match", + 1 = ALL(array[1, 1]) AS "All-NoNull-Match"; +SELECT + 2 IN (1, 1, NULL) AS "IN-Null-Negative", + 2 IN (1, 1) AS "IN-NoNull-Negative", + 2 NOT IN (2, 2, NULL) AS "NotIN-Null-Negative", + 2 NOT IN (2, 2) AS "NotIN-NoNull-Negative"; + + + Any-Null-Match | Any-NoNull-Match | ALL-Null-Match | All-NoNull-Match +----------------+------------------+----------------+------------------ + true | true | \N | true + + IN-Null-Negative | IN-NoNull-Negative | NotIN-Null-Negative | NotIN-NoNull-Negative +------------------+--------------------+---------------------+----------------------- + \N | false | false | false + + + + + Single-Column Subquery Rows + + The following examples demonstrate the behavior discussed in + + applied to subqueries using the operators defined in . + Here we cover the case where the multiple elements being checked are rows, each having one column. + If the column itself is multi-element then the thing being searched for must be a compatible + multi-element value, and the corresponding comparison behavior described in + will also be applied. + + + +SELECT + 1 = ANY(SELECT unnest(array[1, 1, NULL])) AS "Any-Null-Match", + 1 = ANY(SELECT unnest(array[1, 1])) AS "Any-NoNull-Match", + 1 = ALL(SELECT unnest(array[1, 1, NULL])) AS "ALL-Null-Match", + 1 = ALL(SELECT unnest(array[1, 1])) AS "All-NoNull-Match"; +SELECT + 2 = ANY(SELECT unnest(array[1, 1, NULL])) AS "Any-Null-NoMatch", + 2 = ANY(SELECT unnest(array[1, 1])) AS "Any-NoNull-NoMatch", + 2 = ALL(SELECT unnest(array[1, 1, NULL])) AS "ALL-Null-NoMatch", + 2 = ALL(SELECT unnest(array[1, 1])) AS "All-NoNull-NoMatch"; + + + Any-Null-Match | Any-NoNull-Match | ALL-Null-Match | All-NoNull-Match +----------------+------------------+----------------+------------------ + true | true | \N | true + + Any-Null-NoMatch | Any-NoNull-NoMatch | ALL-Null-NoMatch | All-NoNull-NoMatch +------------------+--------------------+------------------+-------------------- + \N | false | false | false + + + + + + + Multi-Element Comparisons + + The previous section, , discussed applying + a predicate or a scalar value check element-wise across a multi-element value. + This section moves the discussion over to comparing two multi-element values to each other. + As both array and composite typed values + can be stored within an index, and comparing two values in that context must not produce + a null-valued result, considerations are made to adhere to the SQL standard where + possible while still making indexes, which the specification is silent on, functional. + Specifically, except when comparing two row constructors, null values are considered + equal to other null values and greater than all non-null values. + + + There are five pair-wise comparison situations to consider: + element-wise when the inputs are arrays, and row-wise when the inputs can be either + row constructors or composite typed values. While these four later combinations seem similar, + the fact that row constructors are query literals, while composite typed values can be stored, + brings about important differences in how they are treated. Please read + for a fuller treatment of this topic. Here + we briefly recap the five situations in the presence of null values. + + + Element-wise Comparisons + + In this first situation, null values within an array compare as equal to each other and greater + than all non-null values, regardless of whether the comparison involves + array constructors or array-typed values. + +SELECT + array[1,2]=array[1,null] AS "Constructors", + s, t, + s = t AS "Stored Equality", + t > s AS "Stored Ordering" +FROM +(values (array[1,2])) AS sv (s), +(values (array[1,null::integer])) AS st (t); + + + Constructors | s | t | Stored Equality | Stored Ordering +--------------+-------+----------+-----------------+----------------- + false | {1,2} | {1,NULL} | false | true + + + + + Row-wise Mutual Row Constructor Comparisons + + In this situation, null values produce unknown when compared to all values. + +SELECT + (1,2)=(1,null) AS "NonNull=Null", + (1,null::integer)=(1,null) AS "Null=Null"; + + + NonNull=Null | Null=Null +--------------+----------- + \N | \N + + + + + Row-wise Composite Involved Comparisons + + In these three situations, null values are considered equal to each other and greater than + all non-null valueS. + + +SELECT s, t, + s = t AS "Stored Equals Stored", + t < (1,2) AS "Stored LT Constructor", + t = (1,null::integer) AS "Stored Equals Constructor" +FROM + (values (1,2)) AS s, + (values (1,null::integer)) AS t; + + + s | t | Stored Equals Stored | Stored LT Constructor | Stored Equals Constructor +-------+------+----------------------+-----------------------+--------------------------- + (1,2) | (1,) | false | false | true + + + + SQL Standard Conformance + + The SQL standard requires row-wise comparison to return NULL if the + result depends on comparing two NULL values or a NULL and a non-NULL. + PostgreSQL does this only when comparing the + results of two row constructors (as in + ) or comparing a row constructor + to the output of a subquery (as in ). + + + + + + Null-Valued Arguments in Normal Function Calls + + Function specifications + have a "strictness" attribute (pg_proc.proisstrict) that, + when set to "strict" (true) will tell the executor to return a null value for any + function call having at least one null-valued input, without executing the + function. + + + Most functions, especially single argument functions, are defined with strict because without + non-null values to act upon they cannot produce a meaningful result. However, for multi-argument + functions, especially variadic functions + like concatenate, null values often are ignored. + This can be different than the choice made by a binary operator performing the same function, + like for concatenating text, but not always, like concatenating an element onto an array. + +SELECT + lower(null::text) AS "Lower", + left('text', null) AS "Left", + 'one' || null AS "|| Text Op", + concat('one', null) AS "concat Text Func", + array_append(array[1], null) AS "append([], null)", + array[1]::integer[] || null::integer AS "[] || null", + array[1]::integer[] || null::integer[] AS "[] || null[]"; + + + Lower | Left | || Text Op | concat Text Func | append([], null) | [] || null | [] || null[] +-------+------+------------+------------------+------------------+------------+-------------- + \N | \N | \N | one | {1,NULL} | {1,NULL} | {1} + + In short, please read the documentation for the functions you use if they may receive null inputs + to understand how they will behave. Send a documentation comment pointing out any functions + that do not behave strictly but whose actual behavior in the presence of null-valued input + is not described or readily inferred. + + + + + Null-Valued Arguments in Aggregate and Window Functions + + When executing an aggregate or window function the state tracking component + (which may be initialized to a non-null value, e.g., 0 for the count function) + will remain unchanged even if the underlying processing + function returns a null value, whether from being defined strict + or it returns a null value upon execution. The aggregation + routine will usually ignore the null value and continue processing, + as demonstrated in count(value) below. + +SELECT + count(*) AS "Count", + count(value) AS "Count Value", + count(null_examples) AS "Count Composite", + count(row(value, value)) AS "Count Row" +FROM null_examples; + + + Count | Count Value | Count Composite | Count Row +-------+-------------+-----------------+----------- + 3 | 2 | 3 | 3 + + Notice the "Count Row" outcome, though. While we noted in the cardinal warning + that a composite whose fields are all null values is indistinguishable from + a null value of composite type, the count aggregate does indeed distinguish them, + recognizing and counting the non-null composite value produced by the + row constructor + row(null, null). + + + + + Null Values When Filtering + + A WHERE clause that evaluates to a null value for a given row will exclude that row. + Note below that, due to tri-valued logic described in , + the row with an id of 2 is not included in either of the first two results. The third result, using + IS NULL, finds that row. + +SELECT id, value AS "Equals 1" +FROM null_examples +WHERE value = 1; + +SELECT id, value AS "Not Equal to 1" +FROM null_examples +WHERE value != 1; + +SELECT id, value AS "IS NULL" +FROM null_examples +WHERE value IS NULL; + + + id | Equals 1 +----+---------- + 1 | 1 + + id | Not Equal to 1 +----+---------------- + 3 | 4 + + id | IS NULL +----+--------- + 2 | \N + + + + + + Null Values in Table Constraints + + It is possible to define + check constraint + expressions on tables to ensure only values passing those expressions are inserted. + While this seems like it would behave the same as a where clause, the choice here, + when an expression evaluates to a null value, is to allow the row to be inserted + - the same as a true result. + +BEGIN; +ALTER TABLE null_examples ADD CONSTRAINT value_not_1 CHECK (value != 1); +ROLLBACK; + + +ERROR: check constraint "value_not_1" of relation "null_examples" is violated by some row + + +BEGIN; +ALTER TABLE null_examples ADD CONSTRAINT value_not_10 CHECK (value != 10); +ROLLBACK; + + +ALTER TABLE + + We are using a transaction (BEGIN and ROLLBACK) and + the ALTER TABLE command to add two constraints to our null_examples table. + The first constraint prohibits rows with a value of 1, which our row with an id of 1 violates. + Prohibiting the value 10 definitely allows rows with ids 1 and 3 to exist, and since we are + not told that some row violates our constraint the null value in the row with id 2 is being + accepted as well. + + + The NOT NULL column constraint + produces the same answer as a column IS NOT NULL check constraint but is + more concise to write. + + + + + Null Values When Grouping + + In the context of both DISTINCT and GROUP BY + it is necessary that all inputs resolve to being either equal to or not equal to all + other values. These features use distinctness + instead of simple equality in order to handle a null value like a definite value equal to + another null value and unequal to all other values. + +WITH vals (value) AS (VALUES (1), (NULL), (1), (2), (NULL)) +SELECT + value, + count(*) AS "Count" +FROM vals +GROUP BY value +ORDER BY value; + + + value | Count +-------+------- + 1 | 2 + 2 | 1 + \N | 2 + + +WITH vals (value) AS (VALUES (1), (NULL), (1), (2), (NULL)) +SELECT DISTINCT value +FROM vals +ORDER BY value NULLS FIRST; + + + value +------- + \N + 1 + 2 + + + + + + Null Values When Ordering + + In the context of ORDER BY, distinctness rules also apply, + though this is insufficient since it must be determined whether or not to + present null values before or after all non-null values. To handle + this, the ORDER BY clause will let you specify either + NULLS FIRST or NULLS LAST. + +WITH vals (value) AS (VALUES (1), (NULL), (1), (2), (NULL)) +SELECT value FROM vals +ORDER BY value DESC NULLS FIRST; + + + value +------- + \N + \N + 2 + 1 + 1 + + + + Note that when dealing with multi-element values the comparison behavior described in + applies: + if the comparison determination rests upon comparing a null value to a non-null value, + the multi-element value with the null-valued component will sort greater than the one + with a non-null component. + + + + + Null Values in Indexes + + The uniqueness and relative ordering rules applied to null values + are defined when creating an index. For the default + NULLS DISTINCT uniqueness, equality rules are applied. + Specifying NULLS NOT DISTINCT will result in + IS DISTINCT FROM rules being applied whereby all null + values are equal to each other. This setting applies to all columns in the index. + + + +BEGIN; +CREATE UNIQUE INDEX value_nulls_distinct_implicit ON null_examples (value); +CREATE UNIQUE INDEX value_nulls_distinct_explicit ON null_examples (value) NULLS DISTINCT; +INSERT INTO null_examples VALUES (4, NULL); +ROLLBACK; + + +CREATE INDEX +CREATE INDEX +INSERT 0 1 + + +BEGIN; +CREATE UNIQUE INDEX value_nulls_not_distinct_explicit ON null_examples (value) NULLS NOT DISTINCT; +INSERT INTO null_examples VALUES (4, NULL); +ROLLBACK; + + +CREATE INDEX +ERROR: duplicate key value violates unique constraint "value_nulls_not_distinct_explicit" +DETAIL: Key (value)=(null) already exists. + + + + For ordering, each column in the index gets its own specification of + direction and null value placement similar to that found in the + ORDER BY clause. + + + Note that when dealing with multi-element values the comparison behavior described in + applies, + if the comparison determination rests upon comparing a null value to a non-null value + the multi-element value with the null-valued component will sort greater than the one + with a non-null component. + + + + + Null Values in Partition Keys + + Presently, PostgreSQL requires that all the columns of a + partition key be included in the primary key. Furthermore, all columns used in a primary + key must have a not-null column constraint applied to them. Therefore, any partitioned table + with a primary key will only have non-null values in the partition key columns. + + + However, should you set up a situation where a partition key column can both: have a null value + and, null values in that key go to a specific partition, list-based routing will work as expected. + There is presently no way to direct rows having null values in partition keys away from the + default partition for range and hash partitioning. + + + + + Null-Valued Settings + + The value of a setting known to the system will never be null. There is a bit of confusion + because the current_setting function has an operating mode where instead + of provoking an error when retrieving the value of a setting not known to the system it will + instead return a null value. This null value should not be considered the value of the setting + but an error indicator. + +SELECT current_setting('example.string', false); +SELECT current_setting('example.string', true); + + +unrecognized configuration parameter "example.string" + current_setting +----------------- + \N + + The next paragraph discusses the corner case behavior when this + suggestion is not heeded. + + + The corner case mentioned above is only meaningful for + custom settings, + thus this section focuses on SQL interaction. + Unlike settings created by extensions, custom settings can only be textual and the default + value for text here is the empty string. + +-- The transaction markers are left here to emphasize the rollback behavior. +SHOW example.string; +BEGIN; +SELECT set_config('example.string', NULL, true); +SELECT current_setting('example.string') IS NULL AS "Setting Is Null"; +ROLLBACK; +SHOW example.string; +RESET example.string; +SHOW example.string; + + +ERROR: unrecognized configuration parameter "example.string" +BEGIN + set_config +------------ + + + Setting Is Null +----------------- + false + +ROLLBACK + example.string +---------------- + + +RESET + example.string +---------------- + + + Notice two important behaviors: first, even though we passed in a null value to + the set_config function, the current_setting + function returned a non-null value, specifically the empty string. Second, after ROLLBACK the + setting is still present (i.e., the error seen before creating the setting no longer appears), + and in fact will remain so until the session ends + (i.e., RESET does not restore the non-existence state.) + + + The other ways to specify settings do allow for null values; + a specific non-null value is required as part of the setting specification. + + + + + Null Values in JSON + + As noted in , the JSON specification's + null value is assigned its own type having a single constant value which can be + compared to all other JSON types with the expected non-null boolean result. + A consequence of this definition is that an SQL json or jsonb type containing + a JSON null value is seen as non-null in SQL. + (Note, while in SQL the capitalization of NULL is unimportant - + all-caps is just convention - JSON requires lowercase.) + +SELECT 'null'::json IS NULL AS "JSON null is NULL"; + + + JSON null is NULL +------------------- + false + + Additionally, the SQL operators and functions involving JSON key or array element selection, + or construction from literals, require that a valid number or text value be supplied as an operand + and so an SQL null value cannot be targeted by those operators and functions. + + SELECT to_json(null::text); + + + to_json +--------- + \N + + That all said, the system will convert an SQL null value to a JSON null value when in a + composite type context. + +SELECT json_build_object('value', value) +FROM null_examples; + + + json_build_object +------------------- + {"value" : 1} + {"value" : null} + {"value" : 4} + + And vice versa. + +SELECT * +FROM jsonb_to_recordset('[{"value":1},{"value":null},{"value":4}]'::jsonb) AS jtr (value integer); + + + value +------- + 1 + \N + 4 + + Or when a simple scalar JSON null is cast to an SQL type. + +SELECT 'null'::jsonb::numeric IS NULL AS "Cast jsonb NULL to SQL NULL"; + + + Cast jsonb NULL to SQL NULL +----------------------------- + true + + + + Aspects of null value handling within the internals of the JSON-related types are discussed + in , + particularly in . + This section is focused on how SQL null values are related to JSON null values. + + + diff --git a/doc/src/sgml/ref/create_domain.sgml b/doc/src/sgml/ref/create_domain.sgml index c111285a69c6..0240f75f3cf7 100644 --- a/doc/src/sgml/ref/create_domain.sgml +++ b/doc/src/sgml/ref/create_domain.sgml @@ -197,9 +197,10 @@ CREATE DOMAIN name [ AS ] NOT NULL, are checked when converting a value to the domain type. It is possible for a column that is nominally of the domain type to read as null despite there being such - a constraint. For example, this can happen in an outer-join query, if - the domain column is on the nullable side of the outer join. A more - subtle example is + a constraint. For example, this can happen in + an outer-join query, if + the domain column is on the nullable side of the outer join. + A more subtle example is INSERT INTO tab (domcol) VALUES ((SELECT domcol FROM tab WHERE false)); diff --git a/doc/src/sgml/syntax.sgml b/doc/src/sgml/syntax.sgml index 916189a7d68c..c41687b22447 100644 --- a/doc/src/sgml/syntax.sgml +++ b/doc/src/sgml/syntax.sgml @@ -281,9 +281,9 @@ U&"d!0061t!+000061" UESCAPE '!' - There are three kinds of implicitly-typed + There are four kinds of implicitly-typed constants in PostgreSQL: - strings, bit strings, and numbers. + strings, bit strings, numbers, and the null value. Constants can also be specified with explicit types, which can enable more accurate representation and more efficient handling by the system. These alternatives are discussed in the following @@ -834,6 +834,25 @@ CAST ( 'string' AS type ) usage, as is the function-call syntax. + + + The Null Value Constant + + null value + constant + + + The null value represents an unknown value and its constant, the keyword NULL, + when evaluated in an expression, likewise yields a value of unknown type. + See for an overview of how the system behaves in the presence + of a null value in various contexts. + + + Due to the typing of a null value as unknown it is often necessary to use + a cast, as described in the previous section, to convert it to the specific type needed. + However, implicit casting is performed when contextual information is available. + +