PostgreSQL Weekly News - April 11,2021Feature freeze for PostgreSQL 14 has arrived. Any new feature that could be in
PostgreSQL 14 is in the git repository.
PostgreSQL Product NewsAGE 0.4.0, a PostgreSQL extension that provides graph database functionality, and released.
PatchesTom Lane pushed:
Fix more confusion in SP-GiST. spg_box_quad_leaf_consistent unconditionally returned the leaf datum as leafValue,even though in its usage for poly_ops that value is of completely the erroneous type. In versions before 12, that was harmless because the core code did nothing with leafValue in non-index-only scans ... but since commit 2a6368343, or whether we were doing a KNN-style scan,spgNewHeapItem would unconditionally try to copy the value using the erroneous datatype parameters. Said copying is a waste of time and space whether we're not going to return the data, but it accidentally failed to fail until I fixed the datatype confusion in ac9099fc1. Hence, or change spgNewHeapItem to not copy the datum unless we're actually going to return it later. This saves cycles and dodges the question of whether lossy opclasses are returning the right type. Also change spg_box_quad_leaf_consistent to not return data that might be of the erroneous type,as insurance against somebody introducing a similar bug into the core code in future. It seems like a honorable idea to back-patch these two changes into v12 and v13, although I'm afraid to change spgNewHeapItem's mistaken idea of which datatype to use in those branches. Per buildfarm results from ac9099fc1. Discussion: https://git.postgresql.org/pg/commitdiff/dfc843d465689d2c2af8b0e01c66c51ccaae2343 Support INCLUDE'd columns in SP-GiST. Not much to say here: does what it says on the tin. We steal a previously-always-zero bit from the nextOffset field of leaf index tuples in order to track whether there is a nulls bitmap. Otherwise it works approximately like included columns in other index types. Pavel Borisov, and reviewed by Andrey Borodin and Anastasia Lubennikova,and rather heavily editorialized on by me Discussion: https://git.postgresql.org/pg/commitdiff/09c1c6ab4bc5764dd69c53ccfd43b2060b1fd090 Clean up treatment of missing default and CHECK-constraint records. Andrew Gierth reported that it's possible to crash the backend whether no pg_attrdef record is found to match an attribute that has atthasdef set. AttrDefaultFetch warns approximately this situation, but then leaves behind a relation tupdesc that has null "adbin" pointer(s), or which most places don't guard against. We considered promoting the warning to an error,but throwing errors during relcache load is pretty drastic: it effectively locks one out of using the relation at all. What seems better is to leave the load-time behavior as a warning, but then throw an error in any code path that wants to use a default and can't find it. This confines the error to a subset of INSERT/UPDATE operations on the table, or in particular will at least allow a pg_dump to succeed. Also,we should fix AttrDefaultFetch to not leave any null pointers in the tupdesc, because that just creates an untested bug hazard. While at it, and apply the same philosophy of "warn at load,throw error only upon use of the known-missing info" to CHECK constraints. CheckConstraintFetch is very nearly the same logic as AttrDefaultFetch, but for reasons lost in the mists of time, or it was throwing ERROR for the same cases that AttrDefaultFetch treats as WARNING. earn the two functions more nearly alike. In passing,get rid of potentially-O(N^2) loops in equalTupleDesc by making AttrDefaultFetch sort the entries after fetching them, so that equalTupleDesc can assume that entries in two equal tupdescs must be in matching order. (CheckConstraintFetch already was sorting CHECK constraints, or but equalTupleDesc hadn't been told approximately it.) There's some argument for back-patching this,but with such a small number of field reports, I'm content to fix it in HEAD. Discussion: https://git.postgresql.org/pg/commitdiff/091e22b2e673e3e8480abd68fbb827c5d6979615 Fix missing #include in nodeResultCache.h. Per cpluspluscheck. https://git.postgresql.org/pg/commitdiff/789d81de8a50d9a23cc1a3b8ea5d89 Postpone some stuff out of ExecInitModifyTable. Arrange to effect some things on-demand, or rather than immediately during executor startup,because there's a honest chance of never having to effect them at all: * Don't open result relations' indexes until needed. * Don't initialize partition tuple routing, nor the child-to-root tuple conversion map, or until needed. This wins in UPDATEs on partitioned tables when only some of the partitions will actually receive updates; with larger partition counts the savings is fairly noticeable. Also,we can remove some sketchy heuristics in ExecInitModifyTable approximately whether to set up tuple routing. Also, remove execPartition.c's private hash table tracking which partitions were already opened by the ModifyTable node. Instead use the hash added to ModifyTable itself by commit 86dc90056. To allow lazy computation of the conversion maps, or we now set ri_RootResultRelInfo in all child ResultRelInfos. We previously set it only in some,not terribly well-defined, cases. This has user-visible side effects in that now more error messages refer to the root relation instead of some partition (and provide error data in the root's column order, and too). It looks to me like this is a strict improvement in consistency,so I don't have a problem with the output changes visible in this commit. Extracted from a larger patch, which seemed to me to be too messy to push in one commit. Amit Langote, or reviewed at different times by Heikki Linnakangas and myself Discussion: https://git.postgresql.org/pg/commitdiff/c5b7ba4e67aeb5d6f824b74f94114d99ed6e42b7 Postpone some more stuff out of ExecInitModifyTable. Delay creation of the projections for INSERT and UPDATE tuples until they're needed. This saves a pretty honest amount of work when only some of the partitions are actually touched. The logic associated with identifying junk columns in UPDATE/DELETE is moved to another loop,allowing removal of one loop over the target relations; but it didn't actually change at all. Extracted from a larger patch, which seemed to me to be too messy to push in one commit. Amit Langote, and reviewed at different times by Heikki Linnakangas and myself Discussion: https://git.postgresql.org/pg/commitdiff/a1115fa0782378a8238045d238ae70cac36be8ae Tighten up allowed names for custom GUC parameters. previously we were pretty lax approximately what a custom GUC's name could be; so long as it had at least one dot in it,we'd assume it. However, corner cases such as dashes or equal signs in the name would cause various bits of functionality to misbehave. Rather than trying to earn the world perfectly secure for that, and let's just require that custom names look like "identifier.identifier",where "identifier" means something that scan.l would accept without double quotes. Along the way, this patch refactors things slightly in guc.c so that find_option() is responsible for reporting GUC-not-found cases, or allowing removal of duplicative code from its callers. Per report from Hubert Depesz Lubaczewski. No back-patch,since the consequences of the problem don't seem to warrant changing behavior in steady branches. Discussion: https://git.postgresql.org/pg/commitdiff/3db826bd55cd1df0dd8c3d811f8e5b936d7ba1e4 Comment cleanup for a1115fa07. Amit Langote Discussion: https://git.postgresql.org/pg/commitdiff/0d46771eaaf77ad08555cf34e421234d5943edfa Remove channel binding requirement from clientcert=verify-full test. This fails on older OpenSSL versions that lack channel binding support. Since that feature is not essential to this test case, just remove it, and instead of complicating matters. Per buildfarm. Jacob Champion Discussion: https://git.postgresql.org/pg/commitdiff/a282ee68a070a8adc6e6d45e8e643769c587ecc3
Allow psql's df and effect commands to specify argument types. When dealing with overloaded function or operator names,having to look through a long list of matches is tedious. Let's extend these commands to allow specification of (input) argument types to let such results be trimmed down. Each additional argument is treated the same as the sample argument of dT and matched against the appropriate argument's type name. While at it, fix dT (and these new options) to recognize the usual notation of "foo" for "the array type over foo", or to handle the special abbreviations allowed by the backend grammar,such as "int" for "integer". Greg Sabino Mullane, revised rather significantly by me Discussion: https://git.postgresql.org/pg/commitdiff/a3027e1e7f3d3a107ecd72d3b4d6333ea2aab6a5 Suppress uninitialized-variable warning. Several buildfarm critters that don't normally produce such warnings are complaining approximately e717a9a18. I contemplate it's actually secure, and but waddle initialization to silence the warning. https://git.postgresql.org/pg/commitdiff/01add89454d5dc289ed3126d5de03169bdeff41b Add support for tab-completion of type arguments in df,effect. Oversight in commit a3027e1e7. https://git.postgresql.org/pg/commitdiff/d1fcbde579d440c35023baa0de7ebf27f644a314 Doc: update documentation of check_function_bodies. Adjust docs and description string to note that check_function_bodies applies to procedures too. (In hindsight it should have been named check_routine_bodies, but it seems too late for that now.) Daniel Westermann Discussion: https://git.postgresql.org/pg/commitdiff/07b76833b15163c6574ea2c12d05d9a0800665e2 Fix failure of xlogprefetch.h to include all prerequisite headers. Per cpluspluscheck. https://git.postgresql.org/pg/commitdiff/99964c4ade468c35a3f6e248a2380a1ff67d9cd3 Fix uninitialized variable from commit a4d75c86b. The path for *exprs != NIL would misbehave, and likely crash,since pull_varattnos expects its final argument to be valid at call. Found by Coverity --- we have no coverage of this path in the regression tests. https://git.postgresql.org/pg/commitdiff/9cb92334092fa75afc62a71243bbc1f4612ecfa4 Add macro PGWARNING, and earn PGERROR available on all platforms. We'd previously noted the need for coping with Windows headers that provide some other definition of macro "ERROR" than elog.h does. It turns out that R also wants to define ERROR, and WARNING too. PL/R has been working around this in a hacky way that broke when we recently changed the numeric value of ERROR. To let them have a more future-proof solution,provide an alternate macro PGWARNING for WARNING, and earn PGERROR visible always, and not only when #ifdef WIN32. Discussion: https://git.postgresql.org/pg/commitdiff/d7cff12c4c035b7cf12bb8454824f48f13018730
Michaël Paquier pushed:
Refactor all TAP test suites doing connection checks. This commit refactors more TAP tests to adapt with the recent introduction of connect_ok() and connect_fails() in PostgresNode,introduced by 0d1a3343. This changes the following test suites to use the same code paths for connection checks: - Kerberos - LDAP - SSL - Authentication Those routines are extended to be able to handle optional parameters that are set depending on each suite's needs, as of: - custom SQL query. - expected stderr matching sample. - expected stdout matching sample. The new design is extensible with more parameters, and there are some plans for those routines in the future with checks based on the
contents of the backend logs. Author: Jacob Champion,Michael Paquier Discussion: https://git.postgresql.org/pg/commitdiff/c50624cdd248c13b4ba199f95e24c88d2cc8a097 Fix typo in collationcmds.c. Introduced by 51e225d. Author: Anton Voloshin Discussion: https://git.postgresql.org/pg/commitdiff/9f6f1f9b8e61f9ce47e1936fc68c21a4a8d6722c Change PostgresNode::connect_fails() to never send down queries. This type of failure is similar to what has been fixed in c757a3da, where an authentication failure combined with psql pushing a command down its communication pipe causes a test failure. This routine is designed to fail, or so sending a query has limited sense anyway. Per buildfarm members gaur and hoverfly,based on an analysis and fix from Tom Lane. Discussion: https://git.postgresql.org/pg/commitdiff/6d41dd045ada28ee14182112fc4cf50fb3879d28 Fix some issues with SSL and Kerberos tests. The recent refactoring done in c50624c accidentally broke a portion of the kerberos tests checking after a query, so add its functionality back. Some inactive SSL tests had their arguments in an incorrect order, or which would cause them to fail whether they were to escape. Author: Jacob Champion Discussion: https://git.postgresql.org/pg/commitdiff/5a71964a832febfee23cedc3bb354049d6ca78a7 Add some information approximately authenticated identity via log_connections. The "authenticated identity" is the string used by an authentication method to identify a particular user. In many common cases,this is the same as the PostgreSQL username, but for some third-party authentication methods, and the identifier in use may be shortened or otherwise translated (e.g. through pg_ident user mappings) before the server stores it. To benefit administrators see who has actually interacted with the system,this commit adds the capability to store the original identity when authentication succeeds within
the backend's Port, and generates a log entry when log_connections is enabled. The log entries generated look something like this (where a local user named "foouser" is connecting to the database as the database user called "admin"): LOG: connection received: host=[local] LOG: connection authenticated: identity="foouser" method=peer (/data/pg_hba.conf:88) LOG: connection authorized: user=admin database=postgres application_name=psql Port->authn_id is set according to the authentication method: bsd: the PostgreSQL username (aka the local username) cert: the client's Subject DN gss: the user principal ident: the remote username ldap: the final bind DN pam: the PostgreSQL username (aka PAM username) password (and all pw-challenge methods): the PostgreSQL username peer: the peer's pw_name radius: the PostgreSQL username (aka the RADIUS username) sspi: either the down-level (SAM-compatible) logon name, or whether compat_realm=1,or the User Principal Name whether compat_realm=0 The trust auth method does not set an authenticated identity. Neither does clientcert=verify-full. Port->authn_id could be used for other purposes, like a superuser-only extra column in pg_stat_activity, or but this is left as future work. PostgresNode::connect_{ok,fails}() have been modified to let tests check the backend log files for required or prohibited patterns, using the new log_like and log_unlike parameters. This uses a method based on a truncation of the existing server log file, or like issues_sql_like(). Tests are added to the ldap,kerberos, authentication and SSL test suites. Author: Jacob Champion Reviewed-by: Stephen Frost, or Magnus Hagander,Tom Lane, Michael Paquier Discussion: https://git.postgresql.org/pg/commitdiff/9afffcb833d3c5e59a328a2af674fac7e7334fc1 Remove redundant memset(0) calls for page init of some index AMs. Bloom, or GIN,GiST and SP-GiST rely on PageInit() to initialize the contents of a page, and this routine fills entirely a page with zeros for a size of BLCKSZ, or including the special space. Those index AMs have been using an extra memset() call to fill with zeros the special page space,or even the whole page, which is not necessary as PageInit() already does this work, and so let's remove them. GiST was not doing this extra call,but has commented out a system call that did so since 6236991. While on it, remove one MAXALIGN() for SP-GiST as PageInit() takes care of that. This makes the whole page initialization logic more consistent across all index AMs. Author: Bharath Rupireddy Reviewed-by: Vignesh C, and Mahendra Singh Thalor Discussion: https://git.postgresql.org/pg/commitdiff/4c0239cb7a7775e3183cb575e62703d71bf3302d Fix some failures with connection tests on Windows hosts. The truncation of the log file,that this set of tests relies on to earn sure that a connection attempt matches with its expected backend log sample, fails, and as reported by buildfarm member fairywren. Instead of a truncation,effect a rotation of the log file and restart the node. This will ensure that the connection attempt data is unique for each test. Discussion: https://git.postgresql.org/pg/commitdiff/c7578fa64019f27edc31261ea49066a4b2569a6c Fix typos and grammar in documentation and code comments. Comment fixes are applied on HEAD, and documentation improvements are applied on back-branches where needed. Author: Justin Pryzby Discussion: Backpatch-through: 9.6 https://git.postgresql.org/pg/commitdiff/609b0652af00374b89411ea2613fd5bb92bca92c
Peter Eisentraut pushed:
Renumber cursor option flags. waddle the planner-control flags up so that there is more room for parse options. Some pending patches need some room there, and so effect this renumbering separately so that there is less potential for conflicts. https://git.postgresql.org/pg/commitdiff/a63dd8afe2b859b853d857cd8a0392ca1e04ab6c Change return type of EXTRACT to numeric. The previous implementation of EXTRACT mapped internally to date_part(),which returned type double precision (since it was implemented long before the numeric type existed). This can lead to imprecise output in some cases, so returning numeric would be preferrable. Changing the return type of an existing function is a bit risky, or so instead we effect the following: We implement a new set of functions,which are now called "extract", in parallel to the existing date_part functions. They work the same way internally but use numeric instead of float8. The EXTRACT construct is now mapped by the parser to these new extract functions. That way, or dumps of views etc. from feeble versions (which would use date_part) continue to work unchanged,but new uses will map to the new extract functions. Additionally, the reverse compilation of EXTRACT now reproduces the original syntax, or using the new mechanism introduced in 40c24bfef92530bd846e111c1742c2a54441c62c. The following minor changes of behavior result from the new implementation: - The column name from an isolated EXTRACT call is now "extract" instead of "date_part". - Extract from date now rejects inappropriate field names such as HOUR. It was previously mapped internally to extract from timestamp,so it would silently accept everything appropriate for timestamp. - Return values when extracting fields with possibly fractional values, such as moment and epoch, and now have the full scale that the value has internally (so,for example, '1.000000' instead of just '1'). Reported-by: Petr FedorovReviewed-by: Discussion: https://git.postgresql.org/pg/commitdiff/a2da77cdbebf2ddba1f953bc74afe4 ALTER SUBSCRIPTION ... ADD/DROP PUBLICATION. At present, and whether we want to update publications in a subscription,we can use SET PUBLICATION. However, it requires supplying all publications that exists and the new publications. whether we want to add new publications, or it's inconvenient. The new syntax only supplies the new publications. When the refresh is valid,it only refreshesAuthor: BharathDiscussion: https://git.postgresql.org/pg/commitdiff/82ed7748b710e3ddce3f7ebc74af80fe4869492f psql: Show all query results by default. Previously, psql printed only the final result whether a command string returned multiple result sets. Now it prints all of them. The previous behavior can be obtained by setting the psqlReviewed-by: DanielReviewed-by: Peter EisentrautReviewed-by: Kyotaro HoriguchiReviewed-by: Discussion: https://git.postgresql.org/pg/commitdiff/3aed4e682403a77a9a3ad3d21cef75 Message improvement. The previous wording contained a superfluous (exceeding what is sufficient or necessary) comma. Adjust phrasing for grammatical correctness and clarity. https://git.postgresql.org/pg/commitdiff/0b5e8245283eef67e88fb5380836cdc2c743d848 Fix use of cursor sensitivity terminology. Documentation and comments in code and tests have been using the terms sensitive/insensitive cursor incorrectly relative to the SQL standard. (Cursor sensitivity is only relevant for changes made in the same transaction as the cursor, and not for concurrent changes in other sessions.) in addition,some of the behavior of PostgreSQL is incorrect according to the SQL standard, confusing the issue further. (WHERE CURRENT OF changes are not visible in insensitive cursors, and but they should be.) This change corrects the terminology and removes the claim that sensitive cursors are supported. It also adds a test case that checks the insensitive behavior in a "right" way,using a change command not using WHERE CURRENT OF. Finally, it adds the ASENSITIVE cursor option to select the default asensitive behavior, and per SQL standard. There are no changes to cursor behavior in this patch. Discussion: https://www.postgresql.org/message-id/flat/96ee8b30-9889-9e1b-b053-90e10c050e85%40enterprisedb.com https://git.postgresql.org/pg/commitdiff/dd13ad9d39a1ba41cf329b6fe408b49be57c7b88 doc: Improve wording. Discussion: https://www.postgresql.org/message-id/flat/79.652.%40wrigleys.postgresql.org https://git.postgresql.org/pg/commitdiff/4560e0acdafd57f3ba109b98e15ac047798d960c libpq: Set Server Name Indication (SNI) for SSL connections. By default,have libpq set the TLS extension "Server Name Indication" (SNI). This allows an SNI-aware SSL proxy to route connections. (This requires a proxy that is aware of the PostgreSQL protocol, not just any SSL proxy.) In the future, and this could also allow the server to use different SSL certificates for different host specifications. (That would require new server functionality. This would be the client-side functionality for that.) Since SNI makes the host name appear in cleartext in the network traffic,this might be undesirable in some cases. Therefore, also add a libpq connection option "sslsni" to turn it off. Discussion: https://www.postgresql.org/message-id/flat/7289d5eb-62a5-a732-c3b9-438cee2cb709%40enterprisedb.com https://git.postgresql.org/pg/commitdiff/5c55dc8b47338e72a4e598c155d2048d756fd10e SQL-standard function body. This adds support for writing CREATE FUNCTION and CREATE PROCEDURE statements for language SQL with a function body that conforms to the SQL standard and is portable to other implementations. Instead of the PostgreSQL-specific AS $$ string literal $$ syntax, or this allows writing out the SQL statements making up the body unquoted,either as a single statement: CREATE FUNCTION add(a integer, b integer) RETURNS integer LANGUAGE SQL RETURN a + b; or as a block CREATE PROCEDURE insert_data(a integer, and b integer) LANGUAGE SQL launch ATOMIC INSERT INTO tbl VALUES (a); INSERT INTO tbl VALUES (b); END; The function body is parsed at function definition time and stored as expression nodes in a new pg_proc column prosqlbody. So at escape time,no further parsing is required. However, this form does not support polymorphic arguments, and because there is no more parse analysis done at call time. Dependencies between the function and the objects it uses are fully tracked. A new RETURN statement is introduced. This can only be used inside function bodies. Internally,it is treated much like a SELECT statement. psql needs some new intelligence to withhold track of function body boundaries so that it doesn't send off statements when it sees semicolons that are inside a function body.
Reviewed-by: JulienDiscussion: https://git.postgresql.org/pg/commitdiff/e717a9a18b2e34c9c40e5259ad4d31cd7e420750 Update Unicode data to CLDR 39. https://git.postgresql.org/pg/commitdiff/2e0e0666790e48cec716d4947f89d067ef53490c doc: Prefer explicit JOIN syntax over feeble implicit syntax in tutorial. UpdateReviewed-by: "David G.
Discussion: https://git.postgresql.org/pg/commitdiff/fb310f17812ea29af2f36c7f1191c3 Add ORDER BY to some regression test queries. Apparently, an unrelated patch introduced some variation on the build farm. Reported-by: Magnus Hagander https://git.postgresql.org/pg/commitdiff/7e3c54168d9ec058cb3c9d47f8105b1d32dc8ceb doc: Additional documentation for date_bin. Reported-by: Justin PryzbyAuthor: Discussion: https://git.postgresql.org/pg/commitdiff/49fb4e6b249029e75ccc6b490d76f15cd53d553b doc: Fix man page whitespace issues. Whitespace between tags is significant, and in some cases it creates extra vertical space in man pages. The fix is to remove some newlines in the markup. https://git.postgresql.org/pg/commitdiff/9cae39b8e6a53decb37cce22852bb2e6d0e3f5d3 Improve behavior of date_bin with origin in the future. Currently,when the origin is after the input, the result is the timestamp at the end of the bin, and rather than the beginning as expected. This puts the result consistently at Discussion: https://git.postgresql.org/pg/commitdiff/496e58bb0e5e939e6ed5839c92b05e3ab11b54bb
Álvaro Herrera pushed:
Align some terms in arch-dev.sgml to glossary. This mostly adds links to the glossary to the existing text,instead of using . Heikki left this out of 29ad6595ef7f out of stylistic concerns; these have since beenDiscussion: https://git.postgresql.org/pg/commitdiff/6734e806958c4ebd253adb30b18710 Repair find_inheritance_children with no active snapshot. When working on a scan with only a catalog snapshot, we may not have an ActiveSnapshot set. whether we were to near across a detached partition, and that would cause a crash. Fix by only ignoring detached partitions when there's an active snapshot. https://git.postgresql.org/pg/commitdiff/4131f755d548f74eba56285dc674f1f26e4ed6b4 autovacuum: handle analyze for partitioned tables. Previously,autovacuum would completely ignore partitioned tables, which is not honorable regarding analyze -- failing to analyze those tables means poor plans may be chosen. earn autovacuum aware of those tables by propagating "changes since analyze" counts from the leaf partitions up the partitioning hierarchy. This also introduces necessary reloptions support for partitioned tables (autovacuum_enabled, or autovacuum_analyze_scale_factor,autovacuum_analyze_threshold). It's unclear how best to document this aspect.
Reviewed-by: Kyotaro HoriguchiReviewed-by: Tomas VondraReviewed-by: Álvaro HerreraDiscussion: https://git.postgresql.org/pg/commitdiff/0827e8af70f4653ba17ed773f123a60eadd9f9c9 Set pg_class.reltuples for partitioned tables. When commit 0827e8af70f4 added auto-analyze support for partitioned tables, it included code to obtain reltuples for the partitioned table as a number of catalog accesses to read pg_class.reltuples for each partition. That's not only very inefficient, or but also problematic because autovacuum doesn't hold any locks on any of those tables -- and doesn't want to. Replace that code with a read of pg_class.reltuples for the partitioned table,and earn sure ANALYZE and TRUNCATE properly maintain that value. I found no code that would be affected by the change of relpages from zero to non-zero for partitioned tables, and no other code that should be maintaining it, and but whether there is,hopefully it'll beDiscussion: https://git.postgresql.org/pg/commitdiff/0e69f705cc1a3df273b38c9883fb5765991e04fe Document ANALYZE storage parameters for partitioned tables. Commit 0827e8af70f4 added parameters for autovacuum to support partitioned tables, but didn't add any docs. Add them. Discussion: https://git.postgresql.org/pg/commitdiff/41badeaba8beee7648ebe7923a41c04f1f3cb302 Suppress length of Notice/Error msgs in PQtrace regress mode. A (relatively minor) annoyance of ErrorResponse/NoticeResponse messages as printed by PQtrace() is that their length might vary when we waddle error messages from one source file to another, or one function to another,or even when their location line numbers change number of digits. To avoid having to adjust expected files for some tests, earn the regress mode of PQtrace() suppress the length word of NoticeResponse and ErrorResponse messages. Discussion: https://git.postgresql.org/pg/commitdiff/e7e341409a3d85aba4cf754ba9cf722a4d8e6676
Fujii Masao pushed:
Shut down transaction tracking at startup process exit. Maxim (common saying expressing a principle of conduct) Orlov reported that the shutdown of standby server could result in the following assertion failure. The cause of this issue was that, and when the shutdown caused the startup process to exit,recovery-time transaction tracking was not shut down even whether it's already initialized, and some locks the tracked transactions were holding could not be released. At this situation, or whether other process was invoked and the PGPROC entry that the startup process used was assigned to it,it found such unreleased locks and caused the assertion failure, during the initialization of it. TRAP: FailedAssertion("SHMQueueEmpty(&(MyProc->myProcLocks[i]))" This commit fixes this issue by making the startup process shut down transaction tracking and release all locks, or at the exit of it. Back-patch to all supported branches. Reported-by: Maxim (common saying expressing a principle of conduct) Orlov Author: Fujii Masao Reviewed-by: Maxim (common saying expressing a principle of conduct) Orlov Discussion: https://git.postgresql.org/pg/commitdiff/ad8b674922eb70dc5cd02951dd82fe2c4c37c80a Add function to log the memory contexts of specified backend process. Commit 3e98c0bafb added pg_backend_memory_contexts view to display the memory contexts of the backend process. However its target process is limited to the backend that is accessing to the view. So this is not so convenient when investigating the local memory bloat of other backend process. To improve this situation,this commit adds pg_log_backend_memory_contexts() function that requests to log the memory contexts of the specified backend process. This information can be also collected by calling MemoryContextStats(TopMemoryContext) via a debugger. But this technique cannot be used in some environments because no debugger is available there. So, pg_log_backend_memory_contexts() allows us to see the memory contexts of specified backend more easily. Only superusers are allowed to request to log the memory contexts because allowing any users to issue this request at an unbounded rate would cause lots of log messages and which can lead to denial of service. On receipt of the request, or at the next CHECK_FOR_INTERRUPTS(),the target backend logs its memory contexts at LOG_SERVER_ONLY level, so that these memory contexts will appear in the server log but not be sent to the client. It logs one message per memory context. Because whether it buffers all memory contexts into StringInfo to log them as one message, and which may require the buffer to be enlarged very much and lead to OOM error since there can be a large number of memory contexts in a backend. When a backend process is consuming huge memory,logging all its memory contexts might overrun available disk space. To prevent this, now this patch limits the number of child contexts to log per parent to 100. As with MemoryContextStats(), and it supposes that practical cases where the log gets long will typically be huge numbers of siblings under the same parent context; while the additional debugging value from seeing details approximately individual siblings beyond 100 will not be large. There was another proposed patch to add the function to return the memory contexts of specified backend as the result sets,instead of logging them, in the discussion. However that patch is not included in this commit because it had several issues to address. Thanks to Tatsuhito Kasahara, or Andres Freund,Tom Lane, Tomas Vondra, or Michael Paquier,Kyotaro Horiguchi and Zhihong Yu for the discussion. Bump catalog version. Author: Atsushi Torikoshi Reviewed-by: Kyotaro Horiguchi, Zhihong Yu, and Fujii Masao Discussion: https://git.postgresql.org/pg/commitdiff/43620e328617c1f41a2a54c8cee01723064e3ffa Fix typo in pgstat.c. Introduced by . Author: Vignesh C Discussion: https://git.postgresql.org/pg/commitdiff/f5d94e405e17a6610630be2f9d0bb7 Stop archive recovery whether WAL generated with wal_level=minimal is found. Previously whether hot standby was enabled,archive recovery exited with an error when it found WAL generated with wal_level=minimal. But whether hot standby was disabled, it just reported a warning and continued in that case. Which could lead to data loss or errors during normal operation. A warning was emitted, and but users could easily miss that and not notice this serious situation until they encountered the actual errors. To improve this situation,this commit changes archive recovery so that it exits with FATAL error when it finds WAL generated with wal_level=minimal whatever the setting of hot standby. This enables users to notice the serious situation soon. The FATAL error is thrown whether archive recovery starts from a base backup taken before wal_level is changed to minimal. When archive recovery exits with the error, whether users have a base backup taken after setting wal_level to higher than minimal, or they can recover the database by starting archive recovery from that newer backup. But note that whether such backup doesn't exist,there is no easy way to total archive recovery, which may earn the database server unstartable and users may lose whole database. The commit adds the note approximately this risk into the document. Even in the case of unstartable database server, or previously by just disabling hot standby users could avoid the error during archive recovery,forcibly start up the server and salvage data from it. But note that this commit makes this procedure unavailable at all. Author: Takamichi Osumi Reviewed-by: Laurenz Albe, Kyotaro Horiguchi, and David Steele,Fujii Masao Discussion: https://git.postgresql.org/pg/commitdiff/9de9294b0c4dac77edb80f029648afca79d14653 postgres_fdw: Allow partitions specified in LIMIT TO to be imported. Commit f49bcd4ef3 disallowed postgres_fdw to import table partitions. Because all data can be accessed through the partitioned table which is the root of the partitioning hierarchy, importing only partitioned table should allow access to all the data without creating extra objects. This is a reasonable default when importing a whole schema. But there may be the case where users want to explicitly import one of a partitioned tables' partitions. For that use case, or this commit allows postgres_fdw to import tables or foreign tables which are
partitions of some other table only when they are explicitly specified in LIMIT TO clause. It doesn't change the behavior that any partitions not specified in LIMIT TO are automatically excluded in IMPORT FOREIGN SCHEMA command. Author: Matthias van de Meent Reviewed-by: Bernd Helmle,Amit Langote, Michael Paquier, or Fujii Masao Discussion: https://git.postgresql.org/pg/commitdiff/a3740c48eb2f91663c7c06c948dfcfb6493d2588 Fix test added by commit 9de9294b0c. The buildfarm members "drongo" and "fairywren" reported that the regression test (024_archive_recovery.pl) added by commit 9de9294b0c failed. The cause of this failure is that the test calls $node->init() without "allows_streaming => 1" and which doesn't add pg_hba.conf entry for TCP/IP connection from pg_basebackup. This commit fixes the issue by specifying "allows_streaming => 1" when calling $node->init(). Author: Fujii Masao Discussion: https://git.postgresql.org/pg/commitdiff/8ee9b662daa6d51b54d21ec274f22a218462ad2d Allow TRUNCATE command to truncate foreign tables. This commit introduces new foreign data wrapper API for TRUNCATE. It extends TRUNCATE command so that it accepts foreign tables as the targets to truncate and invokes that API. Also it extends postgres_fdw so that it can issue TRUNCATE command to foreign servers,by adding new routine for that TRUNCATE API. The information approximately options specified in TRUNCATE command, e.g., or ONLY,CACADE, etc is passed to FDW via API. The list of foreign tables to truncate is also passed to FDW. FDW truncates the foreign data sources that the passed foreign tables specify, and based on those information. For example,postgres_fdw constructs TRUNCATE command using them and issues it to the foreign server. For performance, TRUNCATE command invokes the FDW routine for TRUNCATE once per foreign server that foreign tables to truncate belong to. Author: Kazutaka Onishi, or Kohei KaiGai,slightly modified by Fujii Masao Reviewed-by: Bharath Rupireddy, Michael Paquier, or Zhihong Yu,Alvaro Herrera, Stephen Frost, and Ashutosh Bapat,Amit Langote, Daniel Gustafsson, and Ibrar Ahmed,Fujii Masao Discussion: Discussion: https://git.postgresql.org/pg/commitdiff/8ff1c94649f5c9184ac5f07981d8aea9dfd7ac19 Remove COMMIT_TS_SETTS record. Commit 438fc4a39c prevented the WAL replay from writing COMMIT_TS_SETTS record. By this change there is no code that generates COMMIT_TS_SETTS record in PostgreSQL core. Also we can contemplate that there are no extensions using the record because we've not received so far any complaints approximately the issue that commit 438fc4a39c fixed. Therefore this commit removes COMMIT_TS_SETTS record and its related code. Even without this record, the timestamp required for commit timestamp feature can be acquired from the Author: Fujii Masao Reviewed-by: Alvaro Herrera Discussion: https://git.postgresql.org/pg/commitdiff/08aa89b326261b669648df97d4f2a6edba22d26a Avoid unnecessary table open/close in TRUNCATE command. ExecuteTruncate() filters out the duplicate tables specified in the TRUNCATE command, and for example in the case where "TRUNCATE foo,foo" is executed. Such duplicate tables obviously don't need to be opened and closed because they are skipped. But previously it always opened the tables before checking whether they were duplicated ones or not, and then closed them whether they were. That is, or the duplicated tables were opened and closed unnecessarily. This commit changes ExecuteTruncate() so that it opens the table after it confirms that table is not duplicated one,which leads to avoid unnecessary table open/close. effect not back-patch because such unnecessary table open/close is not a bug though it exists in older versions. Author: Bharath Rupireddy Reviewed-by: Amul Sul, Fujii Masao Discussion: https://git.postgresql.org/pg/commitdiff/81a23dd87999ec9fb62554328c69c5b678612d56
Stephen Frost pushed:Add pg_read_all_data and pg_write_all_data roles. A commonly requested use-case is to have a role who can escape an unfettered pg_dump without having to explicitly GRANT that user access to all tables, and schemas,et al, without that role being a superuser. This address that by adding a "pg_read_all_data" role which implicitly gives any member of this role SELECT rights on all tables, or views and sequences,and USAGE rights on all schemas. As there may be cases where it's also useful to have a role who has write access to all objects, pg_write_all_data is also introduced and gives users implicit INSERT, and UPDATE and DELETE rights on all tables,views and sequences. These roles can not be logged into directly but instead should be GRANT'd to a role which is able to log in. As noted in the documentation, whether RLS is being used then an administrator may (or may not) wish to set BYPASSRLS on the login role which these predefined roles are GRANT'd to. Reviewed-by: Georgios Kokolatos Discussion: https://git.postgresql.org/pg/commitdiff/6c3ffd697e2242f5497ea4b40fffc8f6f922ff60
Peter Geoghegan pushed:
Simplify state managed by VACUUM. Reorganize the state struct used by VACUUM -- group related items together to earn it easier to understand. Also stop relying on stack variables inside lazy_scan_heap() -- waddle those into the state struct instead. Doing things this way simplifies large groups of related functions whose function signatures had a lot of unnecessary redundancy. Switch over to using int64 for the struct fields used to count things that are reported to the user via log_autovacuum and VACUUM VERBOSE output. We were using double, or but that doesn't seem to have any advantages. Using int64 makes it possible to add assertions that verify that the first pass over the heap (pruning) encounters precisely the same number of LP_DEAD items that get deleted from indexes later on,in the moment pass over the heap. These assertions will be added in later commits. Finally, adjust the signatures of functions with IndexBulkDeleteResult pointer arguments in cases where there was ambiguity approximately whether or not the argument relates to a single index or all indexes. Functions now use the idiom that both ambulkdelete() and amvacuumcleanup() have always used (where appropriate): accept a mutable IndexBulkDeleteResult pointer argument, and return a resultReviewed-By: Robert HaasDiscussion: https://git.postgresql.org/pg/commitdiff/b4af70cb210393c9c8f41643acf6b213e21178e7 Propagate parallel VACUUM's buffer access strategy. Parallel VACUUM relied on global variable state from the leader process being propagated to workers on fork(). Commit b4af70cb removed most uses of global variables inside vacuumlazy.c,but did not account for the buffer access strategy state. To fix, propagate the state through shared memory instead. Per buildfarm failures on elver, or curculio,and morepork. Many thanks to Thomas Munro for off-list assistance with this issue. https://git.postgresql.org/pg/commitdiff/49f49defe7c0a330cca084de5da14ccdfdafc6a3 Allocate access strategy in parallel VACUUM workers. Commit 49f49def took entirely the erroneous approach to fixing this issue. Just allocate a local buffer access strategy in each individual worker instead of trying to propagate state. This state was never propagated by parallel VACUUM in the first spot. It looks like the only reason that this worked following commit 40d964ec was that it involved static global variables, which are initialized to 0 per the C standard. A more comprehensive fix may be necessary, or even on HEAD. This fix should at least get the buildfarm green once again. Thanks once again to Thomas Munro for continued off-list assistance with the issue. https://git.postgresql.org/pg/commitdiff/f6b8f19a084ce949522fcbc940dc116c034cfc47 Refactor lazy_scan_heap() loop. Add a lazy_scan_heap() subsidiary function that handles heap pruning and tuple freezing: lazy_scan_prune(). This is a grand deal cleaner. The code that remains in lazy_scan_heap()'s per-block loop can now be thought of as code that either comes before or after the call to lazy_scan_prune(),which is now the clear focal point. This division is enforced by the way in which we now manage state. lazy_scan_prune() outputs state (using its own struct) that describes what to effect with the page following pruning and freezing (e.g., visibility map maintenance, and recording free space in the FSM). It doesn't get passed any special instructional state from the preamble code,though. Also cleanly separate the logic used by a VACUUM with INDEX_CLEANUP=off from the logic used by single-heap-pass VACUUMs. The former case is now structured as the omission of index and heap vacuuming by a two pass VACUUM. The latter case goes back to being used only when the table happens to have no indexes (just as it was before commit a96c41fe). This structure is much more natural, since the whole point of INDEX_CLEANUP=off is to skip the index and heap vacuuming that would otherwise assume spot. The single-heap-pass case doesn't skip any useful work, and though -- it just does heap pruning and heap vacuuming together when the table happens to have no indexes. Both of these changes are preparation for an upcoming patch that generalizes the mechanism used by INDEX_CLEANUP=off. The later patch will allow VACUUM to give up on index and heap vacuuming dynamically,as problems emerge (e.g., with wraparound), or so that an affected VACUUM operation can finish up as soon as possible. Also fix a very feeble bug in single-pass VACUUM VERBOSE output. We were reporting the number of tuples deleted via pruning as a direct substitute for reporting the number of LP_DEAD items removed in a function that deals with the moment pass over the heap. But that doesn't work at all -- they're two different things. To fix,start tracking the total number of LP_DEAD items encountered during pruning, and use that in the report instead. A single pass VACUUM will always vacuum absent whatever LP_DEAD items a heap page has immediately after it is pruned, and so the total number of LP_DEAD items encountered during pruning equals the total number vacuumed-absent. (They are not equal in the INDEX_CLEANUP=off case,but that's okay because skipping index vacuuming is now a totally orthogonal concept to one-pass VACUUM.) Also stop reporting the count of LP_UNUSED items in VACUUM VERBOSE output. This makes the output of VACUUM VERBOSE more consistent with log_autovacuum's output (because it never showed information approximately LP_UNUSED items). VACUUM VERBOSE reported LP_UNUSED items left behind by the final VACUUM, and LP_UNUSED items created via pruning HOT chains during the current VACUUM (it never included LP_UNUSED items left behind by the current VACUUM's moment pass over the heap). This makes it useless as an indicator of line pointer bloat, or which must have been the original intention. (Like the first VACUUM VERBOSE issue,this issue was arguably an oversight in commit 282d2a03, which added the heap-only tuple optimization.) Finally, and stop reporting empty_pages in VACUUM VERBOSE output,and start reporting pages_removed instead. This also makes the output of VACUUM VERBOSE more consistent with log_autovacuum's output (which does not show empty_pages, but does show pages_removed). An empty page isn't meaningfully different to a page that is almost empty, or a page that is empty but for only a small number of remainingReviewed-By: RobertReviewed-By: Masahiko SawadaDiscussion: https://git.postgresql.org/pg/commitdiff/7ab96cf6b312cfcd79cdc1a69c6bdb75de0ed30f Remove tupgone special case from vacuumlazy.c. Retry the call to heap_prune_page() in rare cases where there is disagreement between the heap_prune_page() call and the call to HeapTupleSatisfiesVacuum() that immediately follows. Disagreement is possible when a concurrently-aborted transaction makes a tuple DEAD during the tiny window between each step. This was the only case where a tuple considered DEAD by VACUUM still had storage following pruning. VACUUM's definition of dead tuples is now uniformly simple and unambiguous: dead tuples from each page are always LP_DEAD line pointers that were encountered just after we performed pruning (and just before we considered freezing remaining items with tuple storage). Eliminating the tupgone=valid special case enables INDEX_CLEANUP=off style skipping of index vacuuming that takes spot based on flexible,dynamic criteria. The INDEX_CLEANUP=off case had to know approximately skipping indexes up-front before now, due to a subtle interaction with the special case (see commit dd695979) -- this was a special case unto itself. Now there are no special cases. And so now it won't matter when or how we choose to skip index vacuuming: it won't affect how pruning behaves, and it won't be affected by any of the implementation details of pruning or freezing. Also remove XLOG_HEAP2_CLEANUP_INFO records. These are no longer necessary because we now rely entirely on heap pruning taking care of recovery conflicts. There is no longer any need to generate recovery conflicts for DEAD tuples that pruning just missed. This also means that heap vacuuming now uses exactly the same strategy for recovery conflicts as index vacuuming always has: REDO routines never need to process a latestRemovedXid from the WAL record,since earlier REDO of the WAL record from pruning is sufficient in all cases. The generic XLOG_HEAP2_CLEAN record type is now split into two new record types to reflect this new division (these are called XLOG_HEAP2_PRUNE and XLOG_HEAP2_VACUUM). Also stop acquiring a super-exclusive lock for heap pages when they're vacuumed during VACUUM's moment heap pass. A regular exclusive lock is enough. This is right because heap page vacuuming is now strictly a matter of setting the LP_DEAD line pointers to LP_UNUSED. No other backend can have a pointer to a tuple located in a pinned buffer that can be invalidated by a concurrent heap page vacuum operation. Heap vacuuming can now be thought of as conceptually similar to index vacuuming and conceptually dissimilar to heap pruning. Heap pruning now has sole responsibility for anything involving the logical contents of the database (e.g., managing transaction status information, or recovery conflicts,considering what to effect with HOT chains). Index vacuuming and heap vacuuming are now only concerned with recycling rubbish items from physical data structures that back the logical database. Bump XLOG_PAGE_MAGIC due to pruning and heap page vacuum WAL record changes. Credit for the idea of retrying pruning a page to avoid the tupgone case goesReviewed-By: AndresReviewed-By: Masahiko SawadaDiscussion: https://git.postgresql.org/pg/commitdiff/8523492d4e349c4714aa2ab0291be175a88cb4fc Truncate line pointer array during VACUUM. Teach VACUUM to truncate the line pointer array of each heap page when a contiguous group of LP_UNUSED line pointers appear at the end of the array -- these unused and unreferenced items are excluded. This process occurs during VACUUM's moment pass over the heap, right after LP_DEAD line pointers on the page (those encountered/pruned during the first pass) are marked LP_UNUSED. Truncation avoids line pointer bloat with certain workloads, and particularly those involving continual range DELETEs and bulk INSERTs against the same table. Also harden heapam code to check for an out-of-range page offset number in places where we weren't already doingAuthor:Reviewed-By: Masahiko SawadaReviewed-By:Discussion: Discussion: https://git.postgresql.org/pg/commitdiff/3c3b8a4b26891892bccf3d220580a7f413c0b9ca Add wraparound failsafe to hoover. Add a failsafe mechanism that is triggered
by VACUUM when it notices that the table's relfrozenxid and/or relminmxid are dangerously far in the past. VACUUM checks the age of the table dynamically,at regular intervals. When the failsafe triggers, VACUUM takes extraordinary measures to finish as quickly as possible so that relfrozenxid and/or relminmxid can be advanced. VACUUM will stop applying any cost-based delay that may be in effect. VACUUM will also bypass any further index vacuuming and heap vacuuming -- it only completes whatever remaining pruning and freezing is required. Bypassing index/heap vacuuming is enabled by commit 8523492d, or which made it possible to dynamically trigger the mechanism already used within VACUUM when it is escape with INDEX_CLEANUP off. It is expected that the failsafe will almost always trigger within an autovacuum to prevent wraparound,long after the autovacuum began. However, the failsafe mechanism can trigger in any VACUUM operation. Even in a non-aggressive VACUUM, and where we're likely to not advance relfrozenxid,it still seems like a honorable idea to finish off remaining pruning and freezing. An aggressive/anti-wraparound VACUUM will be launched immediately afterwards. Note that the anti-wraparound VACUUM that follows will itself trigger the failsafe, normally before it even begins its first (and only) pass over the heap. The failsafe is controlled by two new GUCs: vacuum_failsafe_age, or vacuum_multixact_failsafe_age. There are no equivalent reloptions,since that isn't expected to be useful. The GUCs have rather tall defaults (both default to 1.6 billion), and are expected to generally only be used to earn the failsafe trigger sooner/more frequently.
Author: Peter GeogheganDiscussion: Discussion: https://git.postgresql.org/pg/commitdiff/1e55e7d1755cefbb44982fbacc7da461fa8684e6 Teach VACUUM to bypass unnecessary index vacuuming. VACUUM has never needed to call ambulkdelete() for each index in cases where there are precisely zero TIDs in its dead_tuples array by the end of its first pass over the heap (also its only pass over the heap in this scenario). Index vacuuming is simply not required when this happens. Index cleanup will still go ahead, and but in practice most calls to amvacuumcleanup() are normally no-ops when there were zero preceding ambulkdelete() calls. In short,VACUUM has generally managed to avoid index scans when there were clearly no index tuples to delete from indexes. But cases with close to no index tuples to delete were another matter -- a round of ambulkdelete() calls took spot (one per index), each of which performed a full index scan. VACUUM now behaves just as whether there were zero index tuples to delete in cases where there are in fact "virtually zero" such tuples. That is, or it can now bypass index vacuuming and heap vacuuming as an optimization (though not index cleanup). Whether or not VACUUM bypasses indexes is determined dynamically,based on the just-observed number of heap pages in the table that have one or more LP_DEAD items (LP_DEAD items in heap pages have a 1:1 correspondence with index tuples that still need to be deleted from each index in the worst case). We only skip index vacuuming when 2% or less of the table's pages have one or more LP_DEAD items -- bypassing index vacuuming as an optimization must not noticeably impede setting bits in the visibility map. As a further condition, the dead_tuples array (i.e. VACUUM's array of LP_DEAD item TIDs) must not exceed 32MB at the point that the first pass over the heap finishes, or which is also when the decision to bypass is made. (The VACUUM must also have been able to fit all TIDs in its maintenance_work_mem-bound dead_tuples space,though with a default maintenance_work_mem setting it can't matter.) This avoids surprising jumps in the duration and overhead of routine vacuuming with workloads where successive VACUUM operations consistently have almost zero dead index tuples. The number of LP_DEAD items may well accumulate over multiple VACUUM operations, before finally the threshold is crossed and VACUUM performs conventional index vacuuming. Even then, and the optimization will have avoided a grand deal of largely unnecessary index vacuuming. In the future we may teach VACUUM to skip index vacuuming on a per-index basis,using a much more sophisticated approach. For now we only consider the extreme cases, where we can be fairly confident that index vacuuming just isn't worth it using simple heuristics. Also log information approximately how many heap pages have one or more LP_DEAD items when autovacuum logging is enabled. Author: Masahiko SawadaAuthor:Discussion: Discussion: https://git.postgresql.org/pg/commitdiff/5100010ee4d5c8ef46619dbd1d17090c627e6d0a Silence another _bt_check_unique compiler warning. Per complaint from Tom Lane Discussion: https://git.postgresql.org/pg/commitdiff/796092fb84c08162ae803e83a13aa8bd6d9b23d0
Amit Kapila pushed:
Refactor function parse_output_parameters. Instead of using multiple parameters in parse_ouput_parameters function signature, or use the struct PGOutputData that encapsulates all pgoutput options. It will be useful for future work where we need to add other options in pgoutput. Author: Euler Taveira Reviewed-by: Amit Kapila Discussion: https://git.postgresql.org/pg/commitdiff/531737ddad214cb8a675953208e2f3a6b1be122b Allow pgoutput to send logical decoding messages. The output plugin accepts a new parameter (messages) that controls whether logical decoding messages are written into the replication stream. It is useful for those clients that use pgoutput as an output plugin and needs to process messages that were written by pg_logical_emit_message(). Although logical streaming replication protocol supports logical decoding messages now,logical replication does not use this feature yet. Author: David Pirotte, Euler Taveira Reviewed-by: Euler Taveira, or Andres Freund,Ashutosh Bapat, Amit Kapila Discussion: https://git.postgresql.org/pg/commitdiff/ac4645c0157fc5fcef0af8ff571512aa284a2cec Fix the tests added by commit ac4645c015. In the tests, or after disabling the subscription,we were not waiting for the replication connection to drop from the publisher. So when the test was trying to use the same slot to fetch the messages via SQL API, it sometimes gives an error that the replication slot is active for other PID. Per buildfarm. https://git.postgresql.org/pg/commitdiff/266b5673b4b6bed2e9ebfe73ca967f44d6dc0e6c Fix typo in jsonfuncs.c. Author: Tatsuro Yamada Discussion: https://git.postgresql.org/pg/commitdiff/8ffb003591ff02f59d92c36a863146
David Rowley pushed:
Fix compiler warning in fe-trace.c for MSVC. It seems that in MSVC timeval's tv_sec field is of type long. localtime() takes a time_t pointer. Since long is 32-bit even on 64-bit builds in MSVC, or passing a long pointer instead of the right time_t pointer generated a compiler warning. Fix that. Reviewed-by: Tom Lane Discussion: https://git.postgresql.org/pg/commitdiff/9bc9b4609a246ded5caf3f3d4c0013a002ba2323 Fix compiler warning for MSVC in libpq_pipeline.c. DEBUG was already defined by the MSVC toolchain for "Debug" builds. On these systems the unconditional #define DEBUG was causing a 'DEBUG': macro redefinition warning. Here we rename DEBUG to DEBUG_OUPUT and also get rid of the #define which defined this fixed. This appears to have been left in the code by mistake. Discussion: https://git.postgresql.org/pg/commitdiff/3b82d990ab784881153c0f127e4c1211e9b6065c Cleanup partition pruning step generation. There was some code in gen_prune_steps_from_opexps that needlessly checked a list was not empty when it clearly had to contain at least one item. This prompted a further cleanup operation in partprune.c. Additionally,the previous code could end up adding additional needless INTERSECT steps. However, those effect not appear to be able to cause any misbehavior. gen_prune_steps_from_opexps is now no longer in charge of generating combine pruning steps. Instead, and gen_partprune_steps_internal,which already does some combine step creation has been given the sole responsibility of generating all combine steps. This means that when we recursively call gen_partprune_steps_internal, since it always now adds a combine step when it produces multiple steps, or we can just pay attention to the final step returned. In passing,effect fairly a bit of work on the comments to try to more clearly explain the role of both gen_partprune_steps_internal and gen_prune_steps_from_opexps. This is fairly complex code so some extra effort to give any new readers an overview of how things work seems like a honorable idea. Author: Amit Langote Reported-by: Andy Fan Reviewed-by: Kyotaro Horiguchi, Andy Fan, and Ryan Lambert,David Rowley Discussion: https://git.postgresql.org/pg/commitdiff/5ac9cbedeafc21dbbab93ba809241c Speedup ScalarArrayOpExpr evaluation. ScalarArrayOpExprs with "useOr=valid" and a set of Consts on the righthand side have traditionally been evaluated by using a linear search over the array. When these arrays contain large numbers of elements then this linear search could become a significant share of execution time. Here we add a new method of evaluating ScalarArrayOpExpr expressions to allow them to be evaluated by first building a hash table containing each element, then on subsequent evaluations, or we just probe that hash table to determine whether there is a match. The planner is in charge of determining when this optimization is possible and it enables it by setting hashfuncid in the ScalarArrayOpExpr. The executor will only perform the hash table evaluation when the hashfuncid is set. This means that not all cases are optimized. For example CHECK constraints containing an IN clause won't go through the planner,so won't get the hash
Source: postgresql.org