Fix failure to check whether a rowtype's component types are sortable.

The existence of a btree opclass accepting composite types caused us to
assume that every composite type is sortable.  This isn't true of course;
we need to check if the column types are all sortable.  There was logic
for this for the case of array comparison (ie, check that the element
type is sortable), but we missed the point for rowtypes.  Per Teodor's
report of an ANALYZE failure for an unsortable composite type.

Rather than just add some more ad-hoc logic for this, I moved knowledge of
the issue into typcache.c.  The typcache will now only report out array_eq,
record_cmp, and friends as usable operators if the array or composite type
will work with those functions.

Unfortunately we don't have enough info to do this for anonymous RECORD
types; in that case, just assume it will work, and take the runtime failure
as before if it doesn't.

This patch might be a candidate for back-patching at some point, but
given the lack of complaints from the field, I'd rather just test it in
HEAD for now.

Note: most of the places touched in this patch will need further work
when we get around to supporting hashing of record types.

2 files changed, 17 insertions, 0 deletions

diff --git a/src/test/regress/expected/rowtypes.out b/src/test/regress/expected/rowtypes.out
index e5cd71421c6..9430bd9b486 100644
--- a/src/test/regress/expected/rowtypes.out
+++ b/src/test/regress/expected/rowtypes.out
@@ -286,6 +286,16 @@ select row(1,1.1) = any (array[ row(7,7.7), row(1,1.0), row(0,0.0) ]);
  f
 (1 row)
 
+-- Check behavior with a non-comparable rowtype
+create type cantcompare as (p point, r float8);
+create temp table cc (f1 cantcompare);
+insert into cc values('("(1,2)",3)');
+insert into cc values('("(4,5)",6)');
+select * from cc order by f1; -- fail, but should complain about cantcompare
+ERROR:  could not identify an ordering operator for type cantcompare
+LINE 1: select * from cc order by f1;
+                                  ^
+HINT:  Use an explicit ordering operator or modify the query.
 --
 -- Test case derived from bug #5716: check multiple uses of a rowtype result
 --
diff --git a/src/test/regress/sql/rowtypes.sql b/src/test/regress/sql/rowtypes.sql
index 9041df147fe..55e1ff9a9e9 100644
--- a/src/test/regress/sql/rowtypes.sql
+++ b/src/test/regress/sql/rowtypes.sql
@@ -118,6 +118,13 @@ select array[ row(1,2), row(3,4), row(5,6) ];
 select row(1,1.1) = any (array[ row(7,7.7), row(1,1.1), row(0,0.0) ]);
 select row(1,1.1) = any (array[ row(7,7.7), row(1,1.0), row(0,0.0) ]);
 
+-- Check behavior with a non-comparable rowtype
+create type cantcompare as (p point, r float8);
+create temp table cc (f1 cantcompare);
+insert into cc values('("(1,2)",3)');
+insert into cc values('("(4,5)",6)');
+select * from cc order by f1; -- fail, but should complain about cantcompare
+
 --
 -- Test case derived from bug #5716: check multiple uses of a rowtype result
 --