summaryrefslogtreecommitdiff
path: root/src/bin
AgeCommit message (Collapse)Author
2022-03-24Add additional filtering options to pg_waldump.Thomas Munro
Allow filtering by RelFileNode, BlockNumber, ForkNum and FPW. Author: David Christensen <[email protected]> Reviewed-by: Japin Li <[email protected]> Reviewed-by: Bharath Rupireddy <[email protected]> Reviewed-by: Cary Huang <[email protected]> Reviewed-by: Thomas Munro <[email protected]> Discussion: https://postgr.es/m/lzzgmgm6e5.fsf%40veeddrois.attlocal.net
2022-03-24Fix pgbench TAP test.Tatsuo Ishii
Bildfarm member prairiedog reported a pgbench TAP test failure after commit: 4a39f87acd6e681e5ded1239391d8a92645b43d6. This commit attempts to fix some copy&paste errors introduced in the previous commit. Author: Yugo Nagata Reported-by: Tom Lane Discussion: https://postgr.es/m/2775989.1648060014%40sss.pgh.pa.us
2022-03-23pg_upgrade: Upgrade an Assert to a real 'if' testAlvaro Herrera
It seems possible for the condition being tested to be true in production, and nobody would never know (except when some data eventually becomes corrupt?). Author: Álvaro Herrera <[email protected]> Discussion: https://postgr.es/m//[email protected]
2022-03-23pg_basebackup: Try to fix some failures on Windows.Robert Haas
Commit ffd53659c46a54a6978bcb8c4424c1e157a2c0f1 messed up the mechanism that was being used to pass parameters to LogStreamerMain() on Windows. It worked on Linux because only Windows was using threads. Repair by moving the additional parameters added by that commit into the 'logstreamer_param' struct. Along the way, fix a compiler warning on builds without HAVE_LIBZ. Discussion: http://postgr.es/m/CA+TgmoY5=AmWOtMj3v+cySP2rR=Bt6EGyF_joAq4CfczMddKtw@mail.gmail.com
2022-03-23Replace BASE_BACKUP COMPRESSION_LEVEL option with COMPRESSION_DETAIL.Robert Haas
There are more compression parameters that can be specified than just an integer compression level, so rename the new COMPRESSION_LEVEL option to COMPRESSION_DETAIL before it gets released. Introduce a flexible syntax for that option to allow arbitrary options to be specified without needing to adjust the main replication grammar, and common code to parse it that is shared between the client and the server. This commit doesn't actually add any new compression parameters, so the only user-visible change is that you can now type something like pg_basebackup --compress gzip:level=5 instead of writing just pg_basebackup --compress gzip:5. However, it should make it easy to add new options. If for example gzip starts offering fries, we can support pg_basebackup --compress gzip:level=5,fries=true for the benefit of users who want fries with that. Along the way, this fixes a few things in pg_basebackup so that the pg_basebackup can be used with a server-side compression algorithm that pg_basebackup itself does not understand. For example, pg_basebackup --compress server-lz4 could still succeed even if only the server and not the client has LZ4 support, provided that the other options to pg_basebackup don't require the client to decompress the archive. Patch by me. Reviewed by Justin Pryzby and Dagfinn Ilmari Mannsåker. Discussion: http://postgr.es/m/CA+TgmoYvpetyRAbbg1M8b3-iHsaN4nsgmWPjOENu5-doHuJ7fA@mail.gmail.com
2022-03-23Allow pgbench to retry in some cases.Tatsuo Ishii
When serialization or deadlock errors are reported by backend, allow to retry and continue the benchmarking. For this purpose new options "--max-tries", "--failures-detailed" and "--verbose-errors" are added. Transactions with serialization errors or deadlock errors will be repeated after rollbacks until they complete successfully or reach the maximum number of tries (specified by the --max-tries option), or the maximum time of tries (specified by the --latency-limit option). These options can be specified at the same time. It is not possible to use an unlimited number of tries (--max-tries=0) without the --latency-limit option or the --time option. By default the option --max-tries is set to 1, which means transactions with serialization/deadlock errors are not retried. If the last try fails, this transaction will be reported as failed, and the client variables will be set as they were before the first run of this transaction. Statistics on retries and failures are printed in the progress, transaction / aggregation logs and in the end with other results (all and for each script). Also retries and failures are printed per-command with average latency by using option (--report-per-command, -r). Option --failures-detailed prints group failures by basic types (serialization failures / deadlock failures). Option --verbose-errors prints distinct reports on errors and failures (errors without retrying) by type with detailed information like which limit for retries was violated and how far it was exceeded for the serialization/deadlock failures. Patch originally written by Marina Polyakova then Yugo Nagata inherited the discussion and heavily modified the patch to make it commitable. Authors: Yugo Nagata, Marina Polyakova Reviewed-by: Fabien Coelho, Tatsuo Ishii, Alvaro Herrera, Kevin Grittner, Andres Freund, Arthur Zakirov, Alexander Korotkov, Teodor Sigaev, Ildus Kurbangaliev Discussion: https://postgr.es/m/flat/72a0d590d6ba06f242d75c2e641820ec%40postgrespro.ru
2022-03-22Add missing dependency of pg_dumpall to WIN32RES.Andres Freund
When cross-building to windows, or building with mingw on windows, the build could fail with x86_64-w64-mingw32-gcc: error: win32ver.o: No such file or director because pg_dumpall didn't depend on WIN32RES, but it's recipe references it. The build nevertheless succeeded most of the time, due to pg_dump/pg_restore having the required dependency, causing win32ver.o to be built. Reported-By: Thomas Munro <[email protected]> Discussion: https://postgr.es/m/CA+hUKGJeekpUPWW6yCVdf9=oBAcCp86RrBivo4Y4cwazAzGPng@mail.gmail.com Backpatch: 10-, omission present on all live branches
2022-03-22Add ALTER SUBSCRIPTION ... SKIP.Amit Kapila
This feature allows skipping the transaction on subscriber nodes. If incoming change violates any constraint, logical replication stops until it's resolved. Currently, users need to either manually resolve the conflict by updating a subscriber-side database or by using function pg_replication_origin_advance() to skip the conflicting transaction. This commit introduces a simpler way to skip the conflicting transactions. The user can specify LSN by ALTER SUBSCRIPTION ... SKIP (lsn = XXX), which allows the apply worker to skip the transaction finished at specified LSN. The apply worker skips all data modification changes within the transaction. Author: Masahiko Sawada Reviewed-by: Takamichi Osumi, Hou Zhijie, Peter Eisentraut, Amit Kapila, Shi Yu, Vignesh C, Greg Nancarrow, Haiying Tang, Euler Taveira Discussion: https://postgr.es/m/CAD21AoDeScrsHhLyEPYqN3sydg6PxAPVBboK=30xJfUVihNZDA@mail.gmail.com
2022-03-21Fix bogus dependency handling for GENERATED expressions.Tom Lane
For GENERATED columns, we record all dependencies of the generation expression as AUTO dependencies of the column itself. This means that the generated column is silently dropped if any dependency is removed, even if CASCADE wasn't specified. This is at least a POLA violation, but I think it's actually based on a misreading of the standard. The standard does say that you can't drop a dependent GENERATED column in RESTRICT mode; but that's buried down in a subparagraph, on a different page from some pseudocode that makes it look like an AUTO drop is being suggested. Change this to be more like the way that we handle regular default expressions, ie record the dependencies as NORMAL dependencies of the pg_attrdef entry. Also, make the pg_attrdef entry's dependency on the column itself be INTERNAL not AUTO. That has two effects: * the column will go away, not just lose its default, if any dependency of the expression is dropped with CASCADE. So we don't need any special mechanism to make that happen. * it provides an additional cross-check preventing someone from dropping the default expression without dropping the column. catversion bump because of change in the contents of pg_depend (which also requires a change in one information_schema view). Per bug #17439 from Kevin Humphreys. Although this is a longstanding bug, it seems impractical to back-patch because of the need for catalog contents changes. Discussion: https://postgr.es/m/[email protected]
2022-03-21Remove workarounds for avoiding [U]INT64_FORMAT in translatable strings.Tom Lane
Further code simplification along the same lines as d914eb347 and earlier patches. Aleksander Alekseev, Japin Li Discussion: https://postgr.es/m/CAJ7c6TMSKi3Xs8h5MP38XOnQQpBLazJvVxVfPn++roitDJcR7g@mail.gmail.com
2022-03-20psql: handle tab completion of timezone names after "SET TIMEZONE TO".Tom Lane
Dagfinn Ilmari Mannsåker and Tom Lane Discussion: https://postgr.es/m/[email protected]
2022-03-20Fix global ICU collations for ICU < 54Peter Eisentraut
createdb() didn't check for collation attributes validity, which has to be done explicitly on ICU < 54. It also forgot to close the ICU collator opened during the check which leaks some memory. To fix both, add a new check_icu_locale() that does all the appropriate verification and close the ICU collator. initdb also had some partial check for ICU < 54. To have consistent error reporting across major ICU versions, and get rid of the need to include ucol.h, remove the partial check there. The backend will report an error if needed during the post-boostrap iniitialization phase. Author: Julien Rouhaud <[email protected]> Discussion: https://www.postgresql.org/message-id/20220319041459.qqqiqh335sga5ezj@jrouhaud
2022-03-19Improve handling of SET ACCESS METHOD for ALTER MATERIALIZED VIEWMichael Paquier
b048326 has added support for SET ACCESS METHOD in ALTER TABLE, but it has missed a few things for materialized views: - No documentation for this clause on the ALTER MATERIALIZED VIEW page. - psql tab completion missing. - No regression tests. This commit closes the gap on all the points listed above. Author: Yugo Nagata Discussion: https://postgr.es/m/[email protected]
2022-03-18Remove workarounds for avoiding [U]INT64_FORMAT in translatable strings.Tom Lane
Update pg_backup_tar.c along the same lines as 62aa2bb29 and other previous cleanup: we can now rely on %lld or %llu as long as we explicitly cast to long long or unsigned long long. Japin Li Discussion: https://postgr.es/m/MEYP282MB16694F7CC1B119B4ECDD8CBEB6139@MEYP282MB1669.AUSP282.PROD.OUTLOOK.COM
2022-03-18Add circular WAL decoding buffer, take II.Thomas Munro
Teach xlogreader.c to decode the WAL into a circular buffer. This will support optimizations based on looking ahead, to follow in a later commit. * XLogReadRecord() works as before, decoding records one by one, and allowing them to be examined via the traditional XLogRecGetXXX() macros and certain traditional members like xlogreader->ReadRecPtr. * An alternative new interface XLogReadAhead()/XLogNextRecord() is added that returns pointers to DecodedXLogRecord objects so that it's now possible to look ahead in the WAL stream while replaying. * In order to be able to use the new interface effectively while streaming data, support is added for the page_read() callback to respond to a new nonblocking mode with XLREAD_WOULDBLOCK instead of waiting for more data to arrive. No direct user of the new interface is included in this commit, though XLogReadRecord() uses it internally. Existing code doesn't need to change, except in a few places where it was accessing reader internals directly and now needs to go through accessor macros. Reviewed-by: Julien Rouhaud <[email protected]> Reviewed-by: Tomas Vondra <[email protected]> Reviewed-by: Andres Freund <[email protected]> (earlier versions) Discussion: https://postgr.es/m/CA+hUKGJ4VJN8ttxScUFM8dOKX0BrBiboo5uz1cq=AovOddfHpA@mail.gmail.com
2022-03-18Fix header inclusion order in pg_receivewal.cMichael Paquier
lz4frame.h was getting declared after the headers specific to Postgres, but it needs to be included between postgres_fe.h and the internal headers. Issue introduced by babbbb5. Reported-by: Justin Prysby Discussion: https://postgr.es/m/[email protected]
2022-03-17Add option to use ICU as global locale providerPeter Eisentraut
This adds the option to use ICU as the default locale provider for either the whole cluster or a database. New options for initdb, createdb, and CREATE DATABASE are used to select this. Since some (legacy) code still uses the libc locale facilities directly, we still need to set the libc global locale settings even if ICU is otherwise selected. So pg_database now has three locale-related fields: the existing datcollate and datctype, which are always set, and a new daticulocale, which is only set if ICU is selected. A similar change is made in pg_collation for consistency, but in that case, only the libc-related fields or the ICU-related field is set, never both. Reviewed-by: Julien Rouhaud <[email protected]> Discussion: https://www.postgresql.org/message-id/flat/5e756dd6-0e91-d778-96fd-b1bcb06c161a%402ndquadrant.com
2022-03-15Change HAVE_LIBLZ4 and HAVE_LIBZSTD tests to USE_LZ4 and USE_ZSTD.Robert Haas
These tests were added recently, but older code tests USE_LZ4 rathr than HAVE_LIBLZ4, so let's follow the established precedent. It also seems more consistent with the intent of the configure tests, since I think that the USE_* symbols are intended to correspond to what the user requested, and the HAVE_* symbols to what configure found while probing. Discussion: http://postgr.es/m/CA+Tgmoap+hTD2-QNPJLH4tffeFE8MX5+xkbFKMU3FKBy=ZSNKA@mail.gmail.com
2022-03-15Fix collection of typos in the code and the documentationMichael Paquier
Some words were duplicated while other places were grammatically incorrect, including one variable name in the code. Author: Otto Kekalainen, Justin Pryzby Discussion: https://postgr.es/m/[email protected]
2022-03-14Optionally disable subscriptions on error.Amit Kapila
Logical replication apply workers for a subscription can easily get stuck in an infinite loop of attempting to apply a change, triggering an error (such as a constraint violation), exiting with the error written to the subscription server log, and restarting. To partially remedy the situation, this patch adds a new subscription option named 'disable_on_error'. To be consistent with old behavior, this option defaults to false. When true, both the tablesync worker and apply worker catch any errors thrown and disable the subscription in order to break the loop. The error is still also written in the logs. Once the subscription is disabled, users can either manually resolve the conflict/error or skip the conflicting transaction by using pg_replication_origin_advance() function. After resolving the conflict, users need to enable the subscription to allow apply process to proceed. Author: Osumi Takamichi and Mark Dilger Reviewed-by: Greg Nancarrow, Vignesh C, Amit Kapila, Wang wei, Tang Haiying, Peter Smith, Masahiko Sawada, Shi Yu Discussion : https://postgr.es/m/DB35438F-9356-4841-89A0-412709EBD3AB%40enterprisedb.com
2022-03-13Fix bogus tab-completion queries.Tom Lane
My (tgl's) thinko in commit 02b8048ba: I forgot that the first argument of COMPLETE_WITH_QUERY_PLUS is a format string, and hence failed to double a literal %. These two places seem to be the only ones that are wrong, though. Vignesh C Discussion: https://postgr.es/m/CALDaNm0hBO+tZqBWhBjTVxyET1GWANq5K9XpQ07atSxnFXbG7w@mail.gmail.com
2022-03-11pg_basebackup: Clean up some bogus file extension tests.Robert Haas
Justin Pryzby Discussion: http://postgr.es/m/[email protected]
2022-03-11pg_basebackup: Avoid unclean failure with server-compression and -D -.Robert Haas
Fail with a suitable error message instead. We can't inject the backup manifest into the output tarfile without decompressing it, and if we did that, we'd have to recompress the tarfile afterwards to produce the result the user is expecting. While we have enough infrastructure in pg_basebackup now to accomplish that whole series of steps without much additional code, it seems like excessively surprising behavior. The user probably did not select server-side compression with the idea that the client was going to end up decompressing it and then recompressing. Report from Justin Pryzby. Fix by me. Discussion: http://postgr.es/m/CA+Tgmob6Rnjz-Qv32h3yJn8nnUkLhrtQDAS4y5AtsgtorAFHRA@mail.gmail.com
2022-03-11Add support for zstd with compression of full-page writes in WALMichael Paquier
wal_compression gains a new value, "zstd", to allow the compression of full-page images using the compression method of the same name. Compression is done using the default level recommended by the library, as of ZSTD_CLEVEL_DEFAULT = 3. Some benchmarking has shown that it could make sense to use a level lower for the FPI compression, like 1 or 2, as the compression rate did not change much with a bit less CPU consumed, but any tests done would only cover few scenarios so it is hard to come to a clear conclusion. Anyway, there is no reason to not use the default level instead, which is the level recommended by the library so it should be fine for most cases. zstd outclasses easily pglz, and is better than LZ4 where one wants to have more compression at the cost of extra CPU but both are good enough in their own scenarios, so the choice between one or the other of these comes to a study of the workload patterns and the schema involved, mainly. This commit relies heavily on 4035cd5, that reshaped the code creating and restoring full-page writes to be aware of the compression type, making this integration straight-forward. This patch borrows some early work from Andrey Borodin, though the patch got a complete rewrite. Author: Justin Pryzby Discussion: https://postgr.es/m/[email protected]
2022-03-09Fix double declaration for check_ok() in pg_upgrade.hPeter Eisentraut
Author: Pavel Borisov <[email protected]>
2022-03-08Fix LZ4 tests for remaining buffer space.Robert Haas
We should flush the buffer when the remaining space is less than the maximum amount that we might need, not when it is less than or equal to the maximum amount we might need. Jeevan Ladhe, per an observation from me. Discussion: http://postgr.es/m/CANm22CgVMa85O1akgs+DOPE8NSrT1zbz5_vYfS83_r+6nCivLQ@mail.gmail.com
2022-03-08Add support for zstd base backup compression.Robert Haas
Both client-side compression and server-side compression are now supported for zstd. In addition, a backup compressed by the server using zstd can now be decompressed by the client in order to accommodate the use of -Fp. Jeevan Ladhe, with some edits by me. Discussion: http://postgr.es/m/CA+Tgmobyzfbz=gyze2_LL1ZumZunmaEKbHQxjrFkOR7APZGu-g@mail.gmail.com
2022-03-05Introduce PG_TEST_TIMEOUT_DEFAULT for TAP suite non-elapsing timeouts.Noah Misch
Slow hosts may avoid load-induced, spurious failures by setting environment variable PG_TEST_TIMEOUT_DEFAULT to some number of seconds greater than 180. Developers may see faster failures by setting that environment variable to some lesser number of seconds. In tests, write $PostgreSQL::Test::Utils::timeout_default wherever the convention has been to write 180. This change raises the default for some briefer timeouts. Back-patch to v10 (all supported versions). Discussion: https://postgr.es/m/[email protected]
2022-03-04psql: Make SSL info display more compactPeter Eisentraut
Remove the bits display, since that can be derived from the cipher suite. Reviewed-by: Daniel Gustafsson <[email protected]> Discussion: https://www.postgresql.org/message-id/flat/aee28ee7-0ab3-c2e2-5bed-109feb0c089b%40enterprisedb.com
2022-03-04Add some additional tests for row filters in logical replication.Amit Kapila
Commit 52e4f0cd47 didn't add tests for pg_dump support, so add a few tests for it. Additionally, verify that catalogs are updated after few ALTER PUBLICATION commands that modify row filters by using \d. Reported-by: Tomas Vondra Author: Shi yu, based on initial by Tomas Vondra Reviewed-by: Euler Taveira and Amit Kapila Discussion: https://postgr.es/m/[email protected]
2022-03-01Fix typo in pgbench messages.Tatsuo Ishii
Author: KAWAMOTO Masaya Reviewed-by: Fabien COELHO Discussion: https://postgr.es/m/20220224115622.41e671e3449ebd8c270e9103%40sraoss.co.jp
2022-03-01Fix check for PGHOST[ADDR] in pg_upgrade with Windows and temporary pathsMichael Paquier
The checks currently done at the startup of pg_upgrade on PGHOST and PGHOSTADDR to avoid any attempts to access to an external cluster fail setting those parameters to Windows paths or even temporary paths prefixed by an '@', as it only considers as a valid path strings beginning with a slash. As mentioned by Andres, is_unixsock_path() is designed to detect such cases, so, like any other code paths dealing with the same problem (psql and libpq), use it rather than assuming that all valid paths are prefixed with just a slash. This issue has been found while testing the TAP tests of pg_upgrade through the CI on Windows. This is a bug, but nobody has complained about it since pg_upgrade exists so no backpatch is done, at least for now. Analyzed-by: Andres Freund, Michael Paquier Discussion: https://postgr.es/m/YeYj4DU5qY/[email protected]
2022-03-01psql: Additional testsPeter Eisentraut
Add a few TAP tests for things that happen while a user query is being sent: - \timing - client encoding handling - notifications Discussion: https://www.postgresql.org/message-id/[email protected]
2022-03-01Rework internal command generation of pg_rewindMichael Paquier
pg_rewind generates and executes internally up to two commands to work on the target cluster, depending on the options given by its caller: - postgres -C to retrieve the value of restore_command, when using -c/--restore-target-wal. - postgres --single to enforce recovery once and get the target cluster in a clean shutdown state. Both commands have been applying incorrect quoting rules, which could lead to failures when for example using a target data directory with unexpected characters like CRLFs. Those commands are now generated with PQExpBuffer, making use of string_utils.h to quote those commands as they should. We may extend those commands in the future with more options, so this makes any upcoming additions easier. This is arguably a bug fix, but nobody has complained about the existing code being a problem either, so no backpatch is done. Extracted from a larger patch by the same author. Author: Gunnar "Nick" Bluth Discussion: https://postgr.es/m/[email protected]
2022-02-28Improve some psql test codePeter Eisentraut
Split psql_like() into two functions psql_like() and psql_fails_like() and make them mirror the existing command_like() and command_fails_like() more closely. In particular, follow the universal convention that the test name is the last argument. Discussion: https://www.postgresql.org/message-id/[email protected]
2022-02-27Fix use of wrong variable in pg_receivewal's get_destination_dir().Andres Freund
The global variable wrongly used is always the one passed to get_destination_dir(), so there currently are no negative consequences. Author: Bharath Rupireddy <[email protected]> Discussion: https://postgr.es/m/CALj2ACUT0C2LQwhyLXTQdj8T9SxZa5j7cmu-UOz0cZ8_D5edjg@mail.gmail.com
2022-02-25pg_waldump: Fix error message for WAL files smaller than XLOG_BLCKSZ.Andres Freund
When opening a WAL file smaller than XLOG_BLCKSZ (e.g. 0 bytes long) while determining the wal_segment_size, pg_waldump checked errno, despite errno not being set by the short read. Resulting in a bogus error message. Author: Kyotaro Horiguchi <[email protected]> Discussion: https://postgr.es/m/[email protected] Backpatch: 11-, the bug was introducedin fc49e24fa
2022-02-23Quick exit on log stream child exit in pg_basebackupDaniel Gustafsson
If the log streaming child process (thread on Windows) dies during backup then the whole backup will be aborted at the end of the backup. Instead, trap ungraceful termination of the log streaming child and exit early. This also adds a TAP test for simulating this by terminating the responsible backend. Reviewed-by: Michael Paquier <[email protected]> Reviewed-by: Bharath Rupireddy <[email protected]> Reviewed-by: Magnus Hagander <[email protected]> Discussion: https://postgr.es/m/[email protected] Discussion: https://postgr.es/m/VI1PR83MB0189818B82C19059CB62E26199A89@VI1PR83MB0189.EURPRD83.prod.outlook.com
2022-02-23Use test functions in pg_rewind test moduleDaniel Gustafsson
Commit 61081e75c introduced pg_rewind along with the test suite, which ensured that subroutines didn't incur more than one test to plan. Now that we no longer explicitly plan tests (since 549ec201d), we can use the usual Test::More functions. Reviewed-by: Andrew Dunstan <[email protected]> Discussion: https://postgr.es/m/[email protected]
2022-02-23Add temporary debug info to help debug 019_replslot_limit.pl failures.Andres Freund
I have not been able to reproduce the occasional failures of 019_replslot_limit.pl we are seeing in the buildfarm and not for lack of trying. The additional logging and increased log level will hopefully help. Will be reverted once the cause is identified. Discussion: https://postgr.es/m/[email protected]
2022-02-22Allow specifying row filters for logical replication of tables.Amit Kapila
This feature adds row filtering for publication tables. When a publication is defined or modified, an optional WHERE clause can be specified. Rows that don't satisfy this WHERE clause will be filtered out. This allows a set of tables to be partially replicated. The row filter is per table. A new row filter can be added simply by specifying a WHERE clause after the table name. The WHERE clause must be enclosed by parentheses. The row filter WHERE clause for a table added to a publication that publishes UPDATE and/or DELETE operations must contain only columns that are covered by REPLICA IDENTITY. The row filter WHERE clause for a table added to a publication that publishes INSERT can use any column. If the row filter evaluates to NULL, it is regarded as "false". The WHERE clause only allows simple expressions that don't have user-defined functions, user-defined operators, user-defined types, user-defined collations, non-immutable built-in functions, or references to system columns. These restrictions could be addressed in the future. If you choose to do the initial table synchronization, only data that satisfies the row filters is copied to the subscriber. If the subscription has several publications in which a table has been published with different WHERE clauses, rows that satisfy ANY of the expressions will be copied. If a subscriber is a pre-15 version, the initial table synchronization won't use row filters even if they are defined in the publisher. The row filters are applied before publishing the changes. If the subscription has several publications in which the same table has been published with different filters (for the same publish operation), those expressions get OR'ed together so that rows satisfying any of the expressions will be replicated. This means all the other filters become redundant if (a) one of the publications have no filter at all, (b) one of the publications was created using FOR ALL TABLES, (c) one of the publications was created using FOR ALL TABLES IN SCHEMA and the table belongs to that same schema. If your publication contains a partitioned table, the publication parameter publish_via_partition_root determines if it uses the partition's row filter (if the parameter is false, the default) or the root partitioned table's row filter. Psql commands \dRp+ and \d <table-name> will display any row filters. Author: Hou Zhijie, Euler Taveira, Peter Smith, Ajin Cherian Reviewed-by: Greg Nancarrow, Haiying Tang, Amit Kapila, Tomas Vondra, Dilip Kumar, Vignesh C, Alvaro Herrera, Andres Freund, Wei Wang Discussion: https://www.postgresql.org/message-id/flat/CAHE3wggb715X%2BmK_DitLXF25B%3DjE6xyNCH4YOwM860JR7HarGQ%40mail.gmail.com
2022-02-21pg_upgrade: Don't print progress status when output is not a tty.Andres Freund
Until this change pg_upgrade with output redirected to a file / pipe would end up printing all files in the cluster. This has made check-world output exceedingly verbose. Author: Andres Freund <[email protected]> Reviewed-By: Justin Pryzby <[email protected]> Reviewed-By: Daniel Gustafsson <[email protected]> Discussion: https://postgr.es/m/CA+hUKGKjrV61ZVJ8OSag+3rKRmCZXPc03bDyWMqhXg3rdZ=fOw@mail.gmail.com
2022-02-20Remove most msys special processing in TAP testsAndrew Dunstan
Following migration of Windows buildfarm members running TAP tests to use of ucrt64 perl for those tests, special processing for msys perl is no longer necessary and so is removed. Backpatch to release 10 Discussion: https://postgr.es/m/[email protected]
2022-02-20Remove PostgreSQL::Test::Utils::perl2host completelyAndrew Dunstan
Commit f1ac4a74de disabled this processing, and as nothing has broken (as expected) here we proceed to remove the routine and adjust all the call sites. Backpatch to release 10 Discussion: https://postgr.es/m/[email protected] Discussion: https://postgr.es/m/[email protected]
2022-02-17Avoid dangling-pointer usage in pg_basebackup progress reports.Tom Lane
Ill-considered refactoring in 23a1c6578 led to progress_filename sometimes pointing to data that had gone out of scope. The most bulletproof fix is to hang onto a copy of whatever's passed in. Compared to the work spent elsewhere per file, that's not very expensive, plus we can skip it except in verbose logging mode. Per buildfarm. Discussion: https://postgr.es/m/[email protected]
2022-02-17Add missing binary-upgrade guard.Robert Haas
Commit 9a974cbcba005256a19991203583a94b4f9a21a9 arranged for pg_dumpall to preserve tablespace OIDs, but it should only do that in binary upgrade mode, not all the time. Reported by Christoph Berg. Discussion: http://postgr.es/m/[email protected]
2022-02-15Remove command checks in tests of pg_basebackup and pg_receivewalMichael Paquier
The TAP tests of those commands have been checking if commands of "gzip" and "lz4" existed by launching them with an extra --version. Based on the buildfarm, this is not required for "gzip" as the command always exists. Since 1d084fb, "lz4" has a ./configure check doing the same thing. Reported-by: Andres Freund Discussion: https://postgr.es/m/[email protected] Discussion: https://postgr.es/m/[email protected]
2022-02-15Fix thinko with subdirectories generated by pg_upgrade for internal filesMichael Paquier
38bfae3 has mixed the "dump/" and "log/" subdirectories generated in "pg_upgrade_output.d/", causing the internal dump files to be generated in "log/" and the log files to be in "dump/", but the opposite should be done. This was not directly an issue for pg_upgrade runs, as the internal dump files were still picked up at the location of their creation, but the newest version of the buildfarm client would have reported the dump files instead of the log files on failures of pg_upgrade. Issue spotted while testing the TAP tests of pg_upgrade.
2022-02-14Move scanint8() to numutils.cPeter Eisentraut
Move scanint8() to numutils.c and rename to pg_strtoint64(). We already have a "16" and "32" version of that, and the code inside the functions was aligned, so this move makes all three versions consistent. The API is also changed to no longer provide the errorOK case. Users that need the error checking can use strtoi64(). Reviewed-by: John Naylor <[email protected]> Discussion: https://www.postgresql.org/message-id/flat/[email protected]
2022-02-14Database-level collation version trackingPeter Eisentraut
This adds to database objects the same version tracking that collation objects have. There is a new pg_database column datcollversion that stores the version, a new function pg_database_collation_actual_version() to get the version from the operating system, and a new subcommand ALTER DATABASE ... REFRESH COLLATION VERSION. This was not originally added together with pg_collation.collversion, since originally version tracking was only supported for ICU, and ICU on a database-level is not currently supported. But we now have version tracking for glibc (since PG13), FreeBSD (since PG14), and Windows (since PG13), so this is useful to have now. Reviewed-by: Julien Rouhaud <[email protected]> Discussion: https://www.postgresql.org/message-id/flat/f0ff3190-29a3-5b39-a179-fa32eee57db6%40enterprisedb.com