Merge with PostgreSQL 8.3.23.

Everything in PostgreSQL 8.3 is now in Greenplum. PostgreSQL 8.3 has been
end-of-lifed in upstream, so there will be no new upstream minor releases
of this series anymore either.

This includes a lot of new features from PostgreSQL 8.3, and a ton of bug
fixes. See upstream release notes for details. One big user-visible change
included is the removal of implicit cast from other datatypes to text. When
this was done in PostgreSQL, it caused a lot of sloppily written
application queries to break. Which is a good thing in the long run, but it
caused a lot of pain on upgrade.

A few features work slightly differently in GPDB:

* Lazy XIDs feature in upstream reduces XID consumption, by not assigning
  XIDs to read-only transactions. That optimization has not been
  implemented for GPDB distributed transactions, however, so practically
  all queries in GPDB still consume XIDs, whether they're read-only or not.

* temp_tablespaces GUC was added in upstream, but it has been disabled in
  GPDB, in favor of GPDB-specific system to decide which filespace to use
  for temporary files.

* B-tree indexes can now be used to answer "col IS NULL" queries. That
  support has not been implemented for bitmap indexes.

* Support was added for "top N" sorting, speeding up queries with ORDER BY
  and LIMIT. That support was not implemented for tuplesort_mk.c, however,
  so you only get that benefit with enable_mk_sort=off.

* Multi-worker autovacuum support was merged, but autovacuum as whole is
  still disabled in GPDB.

* Planner hook was added. Nothing special there, but we should consider
  refactoring the ORCA glue code to use it.

* Plan cache invalidation was added upstream, which made the
  gp_plpgsql_clear_cache_always option obsolete. We also backported the
  further invalidation improvements from 8.4. See below for more
  information.

In addition to those, there were some big refactorings that were not that
interesting from user point of view, but caused a lot of code churn:
InitPlans are now initialized separately at executor startup, the executor
range table was changed to be "flat", and code related to parse analysis of
DDL commands was moved around. The way UPDATE WHERE CURRENT OF was
implemented in GPDB was refactored to match the new upstream support for
the same (see below for more information).

Plan cache invalidation
-----------------------

One notable feature included in this merge is the support for plan cache
invalidation. GPDB contained a hack to invalidate all plans in master
between transactions, for the same purpose, but the proper plan cache
invalidation is a much better solution. However, because the GPDB hack also
handled dropped/recreated functions in some cases, if we just leave it out
and replace with the 8.3 plan cache invalidation, there will be a
 regression in those cases. To fix, this merge commit includes a backport
the rest of the plan cache invalidation support from PostgreSQL 8.4, which
 handles functions and operators too. The upstream commit for that was:

commit ee33b95
Author: Tom Lane <[email protected]>
Date:   Tue Sep 9 18:58:09 2008 +0000

    Improve the plan cache invalidation mechanism to make it invalidate plans
    when user-defined functions used in a plan are modified.  Also invalidate
    plans when schemas, operators, or operator classes are modified; but for these
    cases we just invalidate everything rather than tracking exact dependencies,
    since these types of objects seldom change in a production database.

    Tom Lane; loosely based on a patch by Martin Pihlak.

As a result of that, the gp_plpgsql_clear_cache_always GUC is removed, as
it's no longer needed. This also includes new test cases in the plpgsql
cache regression test, to demonstrate the improvements.

CURRENT OF code changes
-----------------------

This merge includes the upstream support for UPDATE WHERE CURRENT OF. GPDB
had support for that already, it's been refactored to use the upstream code
as much as possible (execCurrentOf()). Now that we have the upstream code
available, this also refactors the way current position of a cursor is
dispatched. Instead of modifying every CurrentOfExpr node in the plan tree
before dispatching it, store the current positions of each cursor mentioned
in a CurrentOfExpr in a separate CursorPosInfo node, and dispatch those
along with the query plan, in QueryDispatchDesc.

This was a joint effort between me (Heikki) and Daniel Gustafsson.

Author: hlinnaka

GNUmakefile.in

@@ -119,7 +119,7 @@ distdir	= postgresql-$(VERSION)
 dummy	= =install=
 garbage = =*  "#"*  ."#"*  *~*  *.orig  *.rej  core  postgresql-*
 
-dist: $(distdir).tar.gz 
+dist: $(distdir).tar.gz $(distdir).tar.bz2
 ifeq ($(split-dist), yes)
 dist: postgresql-base-$(VERSION).tar.gz postgresql-docs-$(VERSION).tar.gz postgresql-opt-$(VERSION).tar.gz postgresql-test-$(VERSION).tar.gz 
 endif
@@ -129,6 +129,8 @@ dist:
 $(distdir).tar: distdir
 	$(TAR) chf $@ $(distdir)
 
+.INTERMEDIATE: $(distdir).tar
+
 opt_files = \
 	src/tools src/tutorial \
 	$(addprefix src/pl/, plperl plpython tcl)
@@ -150,7 +152,7 @@ postgresql-test-$(VERSION).tar: distdir
 
 distdir:
 	-rm -rf $(distdir)* $(dummy)
-	for x in `cd $(top_srcdir) && find . -name CVS -prune -o -print`; do \
+	for x in `cd $(top_srcdir) && find . \( -name CVS -prune \) -o \( -name .git -prune \) -o -print`; do \
 	  file=`expr X$$x : 'X\./\(.*\)'`; \
 	  if test -d "$(top_srcdir)/$$file" ; then \
 	    mkdir "$(distdir)/$$file" && chmod 777 "$(distdir)/$$file";	\
@@ -165,12 +167,12 @@ distdir:
 	#cp $(distdir)/doc/src/sgml/INSTALL $(distdir)/
 	#cp $(distdir)/doc/src/sgml/regress_README $(distdir)/src/test/regress/README
 	$(MAKE) -C $(distdir) distclean
-	-rm -f $(distdir)/README.CVS
+	#rm -f $(distdir)/README.git
 
-distcheck: $(distdir).tar.gz
+distcheck: dist
 	-rm -rf $(dummy)
 	mkdir $(dummy)
-	$(GZIP) -d -c $< | $(TAR) xf -
+	$(GZIP) -d -c $(distdir).tar.gz | $(TAR) xf -
 	install_prefix=`cd $(dummy) && pwd`; \
 	cd $(distdir) \
 	&& ./configure --prefix="$$install_prefix"

config/c-library.m4

@@ -1,5 +1,5 @@
 # Macros that test various C library quirks
-# $PostgreSQL: pgsql/config/c-library.m4,v 1.33 2008/08/21 13:53:28 petere Exp $
+# config/c-library.m4
 
 
 # PGAC_VAR_INT_TIMEZONE
@@ -298,6 +298,7 @@ int main()
 AC_MSG_RESULT([$pgac_cv_printf_arg_control])
 ])# PGAC_FUNC_PRINTF_ARG_CONTROL
 
+
 # PGAC_TYPE_LOCALE_T
 # ------------------
 # Check for the locale_t type and find the right header file.  Mac OS

config/docbook.m4

@@ -1,4 +1,4 @@
-# config/docbook.m4
+# $PostgreSQL: pgsql/config/docbook.m4,v 1.8 2007/08/09 02:33:58 tgl Exp $
 
 # PGAC_PROG_JADE
 # --------------
@@ -55,7 +55,7 @@ AC_CACHE_VAL([pgac_cv_path_stylesheets],
 [if test -n "$DOCBOOKSTYLE"; then
   pgac_cv_path_stylesheets=$DOCBOOKSTYLE
 else
-  for pgac_prefix in /usr /usr/local /opt /sw; do
+  for pgac_prefix in /usr /usr/local /opt; do
     for pgac_infix in share lib; do
       for pgac_postfix in \
         sgml/stylesheets/nwalsh-modular \
@@ -64,8 +64,7 @@ else
         sgml/docbook-dsssl \
         sgml/docbook/dsssl/modular \
         sgml/docbook/stylesheet/dsssl/modular \
-        sgml/docbook/dsssl-stylesheets \
-        sgml/dsssl/docbook-dsssl-nwalsh
+        sgml/docbook/dsssl-stylesheets
       do
         pgac_candidate=$pgac_prefix/$pgac_infix/$pgac_postfix
         if test -r "$pgac_candidate/html/docbook.dsl" \
@@ -97,32 +96,3 @@ if test -n "$DOCBOOKSTYLE"; then
 else
   AC_PATH_PROGS(COLLATEINDEX, collateindex.pl)
 fi])# PGAC_PATH_COLLATEINDEX
-
-
-# PGAC_PATH_DOCBOOK2MAN
-# ---------------------
-# Find docbook2man program from the docbook2X package.  Upstream calls
-# this program docbook2man, but there is also a different docbook2man
-# out there from the docbook-utils package.  Thus, the program we want
-# is called docbook2x-man on Debian and db2x_docbook2man on Fedora.
-#
-# (Consider rewriting this macro using AC_PATH_PROGS_FEATURE_CHECK
-# when switching to Autoconf 2.62+.)
-AC_DEFUN([PGAC_PATH_DOCBOOK2MAN],
-[AC_CACHE_CHECK([for docbook2man], [ac_cv_path_DOCBOOK2MAN],
-[if test -z "$DOCBOOK2MAN"; then
-  _AS_PATH_WALK([],
-  [for ac_prog in docbook2x-man db2x_docbook2man docbook2man; do
-    ac_path="$as_dir/$ac_prog"
-    AS_EXECUTABLE_P(["$ac_path"]) || continue
-    if "$ac_path" --version 2>/dev/null | $GREP docbook2x >/dev/null 2>&1; then
-      ac_cv_path_DOCBOOK2MAN=$ac_path
-      break
-    fi
-  done])
-else
-  ac_cv_path_DOCBOOK2MAN=$DOCBOOK2MAN
-fi])
-DOCBOOK2MAN=$ac_cv_path_DOCBOOK2MAN
-AC_SUBST(DOCBOOK2MAN)
-])# PGAC_PATH_DOCBOOK2MAN

config/mkinstalldirs

@@ -105,13 +105,15 @@ esac
 
 for file
 do
-  set fnord `echo ":$file" | sed -ne 's/^:\//#/;s/^://;s/\// /g;s/^#/\//;p'`
+  # pathcomp starts with leading slash if present, else empty
+  pathcomp=`echo "$file" | sed -ne 's|^\(.\).*$|\1|;s|[^/]||;p'`
+  # swap spaces and slashes, then split on spaces
+  set fnord `echo "$file" | tr ' /' '/ '`
   shift
 
-  pathcomp=
   for d
   do
-    pathcomp="$pathcomp$d"
+    pathcomp="$pathcomp"`echo "$d" | tr '/' ' '`
     case $pathcomp in
       -*) pathcomp=./$pathcomp ;;
     esac

config/programs.m4

@@ -1,4 +1,4 @@
-# config/programs.m4
+# $PostgreSQL: pgsql/config/programs.m4,v 1.24 2008/08/29 13:02:32 petere Exp $
 
 
 # PGAC_PATH_BISON
@@ -10,7 +10,7 @@
 AC_DEFUN([PGAC_PATH_BISON],
 [# Let the user override the search
 if test -z "$BISON"; then
-  AC_PATH_PROGS(BISON, bison)
+  AC_CHECK_PROGS(BISON, bison)
 fi
 
 if test "$BISON"; then
@@ -19,15 +19,15 @@ if test "$BISON"; then
   if echo "$pgac_bison_version" | $AWK '{ if ([$]4 < 1.875) exit 0; else exit 1;}'
   then
     AC_MSG_WARN([
-*** The installed version of Bison, $BISON, is too old to use with PostgreSQL.
-*** Bison version 1.875 or later is required, but this is $pgac_bison_version.])
+*** The installed version of Bison is too old to use with PostgreSQL.
+*** Bison version 1.875 or later is required.])
     BISON=""
   fi
 fi
 
 if test -z "$BISON"; then
   AC_MSG_WARN([
-*** Without Bison you will not be able to build PostgreSQL from Git nor
+*** Without Bison you will not be able to build PostgreSQL from CVS nor
 *** change any of the parser definition files.  You can obtain Bison from
 *** a GNU mirror site.  (If you are using the official distribution of
 *** PostgreSQL then you do not need to worry about this, because the Bison
@@ -42,11 +42,8 @@ AC_SUBST(BISONFLAGS)
 # PGAC_PATH_FLEX
 # --------------
 # Look for Flex, set the output variable FLEX to its path if found.
-# Reject versions before 2.5.31, as we need a reasonably non-buggy reentrant
-# scanner.  (Note: the well-publicized security problem in 2.5.31 does not
-# affect Postgres, and there are still distros shipping patched 2.5.31,
-# so allow it.)  Also find Flex if its installed under `lex', but do not
-# accept other Lex programs.
+# Avoid the buggy version 2.5.3. Also find Flex if its installed
+# under `lex', but do not accept other Lex programs.
 
 AC_DEFUN([PGAC_PATH_FLEX],
 [AC_CACHE_CHECK([for flex], pgac_cv_path_flex,
@@ -78,6 +75,9 @@ else
 *** The installed version of Flex, $pgac_candidate, is too old to use with Greenplum DB.
 *** Flex version 2.5.4 or later is required, but this is $pgac_flex_version.])
           fi
+
+          pgac_cv_path_flex=$pgac_candidate
+          break 2
         fi
       fi
     done
@@ -88,8 +88,14 @@ fi
 ])[]dnl AC_CACHE_CHECK
 
 if test x"$pgac_cv_path_flex" = x"no"; then
+  if test -n "$pgac_broken_flex"; then
+    AC_MSG_WARN([
+*** The Flex version 2.5.3 you have at $pgac_broken_flex contains a bug. You
+*** should get version 2.5.4 or later.])
+  fi
+
   AC_MSG_WARN([
-*** Without Flex you will not be able to build PostgreSQL from Git nor
+*** Without Flex you will not be able to build PostgreSQL from CVS or
 *** change any of the scanner definition files.  You can obtain Flex from
 *** a GNU mirror site.  (If you are using the official distribution of
 *** PostgreSQL then you do not need to worry about this because the Flex
@@ -98,7 +104,7 @@ if test x"$pgac_cv_path_flex" = x"no"; then
   FLEX=
 else
   FLEX=$pgac_cv_path_flex
-  pgac_flex_version=`$FLEX --version 2>/dev/null`
+  pgac_flex_version=`$FLEX -V 2>/dev/null`
   AC_MSG_NOTICE([using $pgac_flex_version])
 fi
 
@@ -117,14 +123,15 @@ AC_SUBST(FLEXFLAGS)
 AC_DEFUN([PGAC_CHECK_READLINE],
 [AC_REQUIRE([AC_CANONICAL_HOST])
 
-AC_CACHE_CHECK([for library containing readline], [pgac_cv_check_readline],
+AC_CACHE_VAL([pgac_cv_check_readline],
 [pgac_cv_check_readline=no
 pgac_save_LIBS=$LIBS
 if test x"$with_libedit_preferred" != x"yes"
 then	READLINE_ORDER="-lreadline -ledit"
 else	READLINE_ORDER="-ledit -lreadline"
 fi
 for pgac_rllib in $READLINE_ORDER ; do
+  AC_MSG_CHECKING([for ${pgac_rllib}])
   for pgac_lib in "" " -ltermcap" " -lncurses" " -lcurses" ; do
     LIBS="${pgac_rllib}${pgac_lib} $pgac_save_LIBS"
     AC_TRY_LINK_FUNC([readline], [[
@@ -143,11 +150,14 @@ for pgac_rllib in $READLINE_ORDER ; do
     ]])
   done
   if test "$pgac_cv_check_readline" != no ; then
+    AC_MSG_RESULT([yes ($pgac_cv_check_readline)])
     break
+  else
+    AC_MSG_RESULT(no)
   fi
 done
 LIBS=$pgac_save_LIBS
-])[]dnl AC_CACHE_CHECK
+])[]dnl AC_CACHE_VAL
 
 if test "$pgac_cv_check_readline" != no ; then
   LIBS="$pgac_cv_check_readline $LIBS"
@@ -163,21 +173,19 @@ fi
 # Readline versions < 2.1 don't have rl_completion_append_character
 
 AC_DEFUN([PGAC_VAR_RL_COMPLETION_APPEND_CHARACTER],
-[AC_CACHE_CHECK([for rl_completion_append_character], pgac_cv_var_rl_completion_append_character,
-[AC_TRY_LINK([#include <stdio.h>
+[AC_MSG_CHECKING([for rl_completion_append_character])
+AC_TRY_LINK([#include <stdio.h>
 #ifdef HAVE_READLINE_READLINE_H
 # include <readline/readline.h>
 #elif defined(HAVE_READLINE_H)
 # include <readline.h>
 #endif
 ],
 [rl_completion_append_character = 'x';],
-[pgac_cv_var_rl_completion_append_character=yes],
-[pgac_cv_var_rl_completion_append_character=no])])
-if test x"$pgac_cv_var_rl_completion_append_character" = x"yes"; then
+[AC_MSG_RESULT(yes)
 AC_DEFINE(HAVE_RL_COMPLETION_APPEND_CHARACTER, 1,
-          [Define to 1 if you have the global variable 'rl_completion_append_character'.])
-fi])# PGAC_VAR_RL_COMPLETION_APPEND_CHARACTER
+          [Define to 1 if you have the global variable 'rl_completion_append_character'.])],
+[AC_MSG_RESULT(no)])])# PGAC_VAR_RL_COMPLETION_APPEND_CHARACTER
 
 
 

config/python.m4

@@ -85,13 +85,13 @@ AC_SUBST(python_libdir)[]dnl
 AC_SUBST(python_libspec)[]dnl
 AC_SUBST(python_additional_libs)[]dnl
 
-# threaded python is not supported on bsd's
+# threaded python is not supported on OpenBSD
 AC_MSG_CHECKING(whether Python is compiled with thread support)
 pythreads=`${PYTHON} -c "import sys; print(int('thread' in sys.builtin_module_names))"`
 if test "$pythreads" = "1"; then
   AC_MSG_RESULT(yes)
   case $host_os in
-  openbsd*|freebsd*)
+  openbsd*)
     AC_MSG_ERROR([threaded Python not supported on this platform])
     ;;
   esac