summaryrefslogtreecommitdiff
path: root/src/backend/commands/copy.c
AgeCommit message (Collapse)Author
2004-07-31Restructure error handling as recently discussed. It is now reallyTom Lane
possible to trap an error inside a function rather than letting it propagate out to PostgresMain. You still have to use AbortCurrentTransaction to clean up, but at least the error handling itself will cooperate.
2004-06-16Represent type-specific length coercion functions as pg_cast entries,Tom Lane
eliminating the former hard-wired convention about their names. Allow pg_cast entries to represent both type coercion and length coercion in a single step --- this is represented by a function that takes an extra typmod argument, just like a length coercion function. This nicely merges the type and length coercion mechanisms into something at least a little cleaner than we had before. Make use of the single- coercion-step behavior to fix integer-to-bit coercion so that coercing to bit(n) yields the rightmost n bits of the integer instead of the leftmost n bits. This should fix recurrent complaints about the odd behavior of this coercion. Clean up the documentation of the bit string functions, and try to put it where people might actually find it. Also, get rid of the unreliable heuristics in ruleutils.c about whether to display nested coercion steps; instead require parse_coerce.c to label them properly in the first place.
2004-06-06Infrastructure for I/O of composite types: arrange for the I/O routinesTom Lane
of a composite type to get that type's OID as their second parameter, in place of typelem which is useless. The actual changes are mostly centralized in getTypeInputInfo and siblings, but I had to fix a few places that were fetching pg_type.typelem for themselves instead of using the lsyscache.c routines. Also, I renamed all the related variables from 'typelem' to 'typioparam' to discourage people from assuming that they necessarily contain array element types.
2004-06-05Tweak palloc/repalloc to allow zero bytes to be requested, as per recentTom Lane
proposal. Eliminate several dozen now-unnecessary hacks to avoid palloc(0). (It's likely there are more that I didn't find.)
2004-05-26Reimplement the linked list data structure used throughout the backend.Neil Conway
In the past, we used a 'Lispy' linked list implementation: a "list" was merely a pointer to the head node of the list. The problem with that design is that it makes lappend() and length() linear time. This patch fixes that problem (and others) by maintaining a count of the list length and a pointer to the tail node along with each head node pointer. A "list" is now a pointer to a structure containing some meta-data about the list; the head and tail pointers in that structure refer to ListCell structures that maintain the actual linked list of nodes. The function names of the list API have also been changed to, I hope, be more logically consistent. By default, the old function names are still available; they will be disabled-by-default once the rest of the tree has been updated to use the new API names.
2004-04-21Change COPY CSV keyword to be:Bruce Momjian
FORCE QUOTE to force quotes FORCE NOT NULL to quote null input values
2004-04-19Prevent doubling of escapes when not in quote mode for CSV.Bruce Momjian
2004-04-19Complete TODO item:Bruce Momjian
o -Allow dump/load of CSV format This adds new keywords to COPY and \copy: CSV - enable CSV mode (comma separated variable) QUOTE - specify quote character ESCAPE - specify escape character FORCE - force quoting of specified column LITERAL - suppress null comparison for columns Doc changes included. Regression updates coming from Andrew.
2004-04-15Modify COPY for() loop to use attnum as a variable name, not 'i'.Bruce Momjian
2004-04-06The attached applied patch throws an error if the delimiter appears inBruce Momjian
the COPY NULL string: test=> copy pg_language to '/tmp/x' with delimiter '|'; COPY test=> copy pg_language to '/tmp/x' with delimiter '|' null '|x'; ERROR: COPY delimiter must not appear in the NULL specification test=> copy pg_language from '/tmp/x' with delimiter '|' null '|x'; ERROR: COPY delimiter must not appear in the NULL specification It also throws an error if it conflicts with the default NULL string: test=> copy pg_language to '/tmp/x' with delimiter '\\'; ERROR: COPY delimiter must not appear in the NULL specification test=> copy pg_language to '/tmp/x' with delimiter '\\' NULL 'x'; COPY
2004-02-10Restructure smgr API as per recent proposal. smgr no longer depends onTom Lane
the relcache, and so the notion of 'blind write' is gone. This should improve efficiency in bgwriter and background checkpoint processes. Internal restructuring in md.c to remove the not-very-useful array of MdfdVec objects --- might as well just use pointers. Also remove the long-dead 'persistent main memory' storage manager (mm.c), since it seems quite unlikely to ever get resurrected.
2004-01-28Review uses of IsUnderPostmaster, change some tests to look atTom Lane
whereToSendOutput instead because they are really inquiring about the correct client communication protocol. Update some comments. This is pointing towards supporting regular FE/BE client protocol in a standalone backend, per discussion a month or so back.
2004-01-26Ensure that close() and fclose() are checked for errors, at least inTom Lane
cases involving writes. Per recent discussion about the possibility of close-time failures on some filesystems. There is a TODO item for this, too.
2004-01-18Don't use %s-with-precision format spec to truncate data being displayedTom Lane
in a COPY error message. It seems that glibc gets indigestion if it is asked to truncate strings that contain invalid UTF-8 encoding sequences. vsnprintf will return -1 in such cases, leading to looping and eventual memory overflow in elog.c. Instead use our own, more robust pg_mbcliplen routine. I believe this problem accounts for several recent reports of unexpected 'out of memory' errors during COPY IN.
2003-11-29$Header: -> $PostgreSQL Changes ...PostgreSQL Daemon
2003-10-06Modify COPY FROM to match the null-value string against the column valueTom Lane
before it is de-backslashed, not after. This allows the null string \N to be reliably distinguished from the data value \N (which must be represented as \\N). Per bug report from Manfred Koizar ... but it's amazing this hasn't been reported before ... Also, be consistent about encoding conversion for null string: the form specified in the command is in the server encoding, but what is sent to/from client must be in client encoding. This never worked quite right before either.
2003-09-29Improve context display for failures during COPY IN, as recentlyTom Lane
discussed on pghackers.
2003-09-25Message editing: remove gratuitous variations in message wording, standardizePeter Eisentraut
terms, add some clarifications, fix some untranslatable attempts at dynamic message building.
2003-08-28Fix stupid typo (mine I suppose) in CopyGetData. Per report from Dave Cramer.Tom Lane
2003-08-13libpq failed to cope with COPY FROM STDIN if the command was issuedTom Lane
via extended query protocol, because it sends Sync right after Execute without realizing that the command to be executed is COPY. There seems to be no reasonable way for it to realize that, either, so the best fix seems to be to make the backend ignore Sync during copy-in mode. Bit of a wart on the protocol, but little alternative. Also, libpq must send another Sync after terminating the COPY, if the command was issued via Execute.
2003-08-08Another pgindent run with updated typedefs.Bruce Momjian
2003-08-04Update copyrights to 2003.Bruce Momjian
2003-08-04pgindent run.Bruce Momjian
2003-08-01Adjust 'permission denied' messages to be more useful and consistent.Tom Lane
2003-07-22Error message editing in backend/libpq, backend/postmaster, backend/tcop.Tom Lane
Along the way, fix some logic problems in pgstat_initstats, notably the bogus assumption that malloc returns zeroed memory.
2003-07-21Error message editing in backend/executor.Tom Lane
2003-07-20Another round of error message editing, covering backend/commands/.Tom Lane
2003-05-16Remove use of geteuid under Win32.Bruce Momjian
2003-05-09COPY BINARY uses the new binary I/O routines. Update a few more datatypesTom Lane
so that COPY BINARY regression test passes.
2003-05-09Implement new-protocol binary I/O support in DataRow, Bind, and FunctionCallTom Lane
messages. Binary I/O is now up and working, but only for a small set of datatypes (integers, text, bytea).
2003-05-08Update 3.0 protocol support to match recent agreements about how toTom Lane
handle multiple 'formats' for data I/O. Restructure CommandDest and DestReceiver stuff one more time (it's finally starting to look a bit clean though). Code now matches latest 3.0 protocol document as far as message formats go --- but there is no support for binary I/O yet.
2003-04-25COPY and pg_dump failed to cope with zero-column tables. Fix 'em.Tom Lane
2003-04-24Infrastructure for upgraded error reporting mechanism. elog.c isTom Lane
rewritten and the protocol is changed, but most elog calls are still elog calls. Also, we need to contemplate mechanisms for controlling all this functionality --- eg, how much stuff should appear in the postmaster log? And what API should libpq expose for it?
2003-04-22Another round of protocol changes. Backend-to-frontend messages now allTom Lane
have length words. COPY OUT reimplemented per new protocol: it doesn't need \. anymore, thank goodness. COPY BINARY to/from frontend works, at least as far as the backend is concerned --- libpq's PQgetline API is not up to snuff, and will have to be replaced with something that is null-safe. libpq uses message length words for performance improvement (no cycles wasted rescanning long messages), but not yet for error recovery.
2003-04-19Allow \r and \r\n termination for COPY files.Bruce Momjian
Output \r\n termination on Win32. Disallow literal carriage return as a data value, backslash-carriage-return and \r still allowed. Doc changes already committed.
2003-04-19Add pipe parameter to COPY function to allow proper line termination.Bruce Momjian
2003-04-19Second round of FE/BE protocol changes. Frontend->backend messages nowTom Lane
have length counts, and COPY IN data is packetized into messages.
2003-04-04Add Win32 path handling for / vs. \ and drive letters.Bruce Momjian
2003-03-27This patch implements holdable cursors, following the proposalBruce Momjian
(materialization into a tuple store) discussed on pgsql-hackers earlier. I've updated the documentation and the regression tests. Notes on the implementation: - I needed to change the tuple store API slightly -- it assumes that it won't be used to hold data across transaction boundaries, so the temp files that it uses for on-disk storage are automatically reclaimed at end-of-transaction. I added a flag to tuplestore_begin_heap() to control this behavior. Is changing the tuple store API in this fashion OK? - in order to store executor results in a tuple store, I added a new CommandDest. This works well for the most part, with one exception: the current DestFunction API doesn't provide enough information to allow the Executor to store results into an arbitrary tuple store (where the particular tuple store to use is chosen by the call site of ExecutorRun). To workaround this, I've temporarily hacked up a solution that works, but is not ideal: since the receiveTuple DestFunction is passed the portal name, we can use that to lookup the Portal data structure for the cursor and then use that to get at the tuple store the Portal is using. This unnecessarily ties the Portal code with the tupleReceiver code, but it works... The proper fix for this is probably to change the DestFunction API -- Tom suggested passing the full QueryDesc to the receiveTuple function. In that case, callers of ExecutorRun could "subclass" QueryDesc to add any additional fields that their particular CommandDest needed to get access to. This approach would work, but I'd like to think about it for a little bit longer before deciding which route to go. In the mean time, the code works fine, so I don't think a fix is urgent. - (semi-related) I added a NO SCROLL keyword to DECLARE CURSOR, and adjusted the behavior of SCROLL in accordance with the discussion on -hackers. - (unrelated) Cleaned up some SGML markup in sql.sgml, copy.sgml Neil Conway
2003-02-03Determine the set of constraints applied to a domain at executorTom Lane
startup, not in the parser; this allows ALTER DOMAIN to work correctly with domain constraint operations stored in rules. Rod Taylor; code review by Tom Lane.
2003-01-10Read-only transactions, as defined in SQL.Peter Eisentraut
2002-12-15Revise executor APIs so that all per-query state structure is built inTom Lane
a per-query memory context created by CreateExecutorState --- and destroyed by FreeExecutorState. This provides a final solution to the longstanding problem of memory leaked by various ExecEndNode calls.
2002-12-13Phase 3 of read-only-plans project: ExecInitExpr now builds expressionTom Lane
execution state trees, and ExecEvalExpr takes an expression state tree not an expression plan tree. The plan tree is now read-only as far as the executor is concerned. Next step is to begin actually exploiting this property.
2002-12-12Phase 2 of read-only-plans project: restructure expression-tree nodesTom Lane
so that all executable expression nodes inherit from a common supertype Expr. This is somewhat of an exercise in code purity rather than any real functional advance, but getting rid of the extra Oper or Func node formerly used in each operator or function call should provide at least a little space and speed improvement. initdb forced by changes in stored-rules representation.
2002-12-01Run COPY OUT in a temporary memory context that's reset once per row,Tom Lane
and eliminate its manual pfree() calls. This solves the encoding-conversion bug recently reported, and should be faster and more robust than the original coding anyway. For example, we are no longer at risk if datatype output routines leak memory or choose to return a constant string.
2002-11-26Use Params, rather than run-time-modified Const nodes, to handleTom Lane
sublink results and COPY's domain constraint checking. A Const that isn't really constant is just a Bad Idea(tm). Remove hacks in parse_coerce and other places that were needed because of the former klugery.
2002-11-25Remove unused constisset and constiscast fields of Const nodes. CleanTom Lane
up code and documentation associated with Param nodes.
2002-11-23This patch implements FOR EACH STATEMENT triggers, per my email toBruce Momjian
-hackers a couple days ago. Notes/caveats: - added regression tests for the new functionality, all regression tests pass on my machine - added pg_dump support - updated PL/PgSQL to support per-statement triggers; didn't look at the other procedural languages. - there's (even) more code duplication in trigger.c than there was previously. Any suggestions on how to refactor the ExecXXXTriggers() functions to reuse more code would be welcome -- I took a brief look at it, but couldn't see an easy way to do it (there are several subtly-different versions of the code in question) - updated the documentation. I also took the liberty of removing a big chunk of duplicated syntax documentation in the Programmer's Guide on triggers, and moving that information to the CREATE TRIGGER reference page. - I also included some spelling fixes and similar small cleanups I noticed while making the changes. If you'd like me to split those into a separate patch, let me know. Neil Conway
2002-11-13Add new palloc0 call as merge of palloc and MemSet(0).Bruce Momjian
2002-11-11Back out use of palloc0 in place if palloc/MemSet. Seems constant lenBruce Momjian
to MemSet is a performance boost.