From: t-ishii@sra.co.jp

Hi, here are patches I promised (against 6.3.2):

* character_length(), position(), substring() are now aware of
          multi-byte characters
* add octet_length()
* add --with-mb option to configure
* new regression tests for EUC_KR
  (contributed by "Soonmyung. Hong" <hong@lunaris.hanmesoft.co.kr>)
* add some test cases to the EUC_JP regression test
* fix problem in regress/regress.sh in case of System V
* fix toupper(), tolower() to handle 8bit chars

note that:

o  patches for both configure.in and configure are
included. maybe the one for configure is not necessary.

o pg_proc.h was modified to add octet_length(). I used OIDs
(1374-1379) for that. Please let me know if these numbers are not
appropriate.

Author: scrappy

doc/README.mb

@@ -1,45 +1,52 @@
-postgresql 6.3 multi-byte(MB) patch PL2 README	  Mar 10 1998
+postgresql 6.3 multi-byte (MB) support README	  April 21 1998
 
 						Tatsuo Ishii
 						t-ishii@sra.co.jp
 		  http://www.sra.co.jp/people/t-ishii/PostgreSQL/
 
 Introduction
 
-MB patch is intended for allowing PostgreSQL to handle multi-byte
-charachter sets such as EUC(Extende Unix Code), Unicode and Mule
-internal code. With the MB patch you can use multi-byte character sets
-in regexp and LIKE. The encoding system chosen is determined at the
-compile time.
+The MB support is intended for allowing PostgreSQL to handle
+multi-byte character sets such as EUC(Extended Unix Code), Unicode and
+Mule internal code. With the MB enabled you can use multi-byte
+character sets in regexp ,LIKE and some functions. The encoding system
+chosen is determined at the compile time.
 
-The patch also fixes some problems concerning with 8-bit single byte
+MB also fixes some problems concerning with 8-bit single byte
 character sets including ISO8859. (I would not say all of problems
 have been fixed. I just confirmed that the regression test ran fine
 and a few French characters could be used with the patch. Please let
 me know if you find any problem while using 8-bit characters)
 
 How to use
 
-After applying the MB patch, create src/Makefile.custom with a line
-including:
+create src/Makefile.custom with a line including:
 
-MB=encoding_system
+	MB=encoding_system
+
+or run configure with the mb option:
+
+	% configure --with-mb=encoding_system
 
 where encoding_system is one of:
 
-EUC_JP			Japanese EUC
-EUC_CN			Chinese EUC
-EUC_KR			Korean EUC
-EUC_TW			Taiwan EUC
-UNICODE			Unicode(UTF-8)
-MULE_INTERNAL		Mule internal
+	EUC_JP			Japanese EUC
+	EUC_CN			Chinese EUC
+	EUC_KR			Korean EUC
+	EUC_TW			Taiwan EUC
+	UNICODE			Unicode(UTF-8)
+	MULE_INTERNAL		Mule internal
 
 Example:
 
-% cat Makefile.custom
-MB=EUC_JP
+	% cat Makefile.custom
+	MB=EUC_JP
+
+	or
 
-If MB is not defined, nothing is changed except better supporting for
+	% configure --with-mb=EUC_JP
+
+If MB is disabled, nothing is changed except better supporting for
 8-bit single byte character sets.
 
 References
@@ -59,6 +66,19 @@ Unicode: http://www.unicode.org/
 
 History
 
+April 21, 1998 some enhancements/fixes
+	* character_length(), position(), substring() are now aware of 
+	  multi-byte characters
+	* add octet_length()
+	* add --with-mb option to configure
+	* new regression tests for EUC_KR
+  	  (contributed by "Soonmyung. Hong" <hong@lunaris.hanmesoft.co.kr>)
+	* add some test cases to the EUC_JP regression test
+	* fix problem in regress/regress.sh in case of System V
+	* fix toupper(), tolower() to handle 8bit chars
+
+Mar 25, 1998 MB PL2 is incorporated into PostgreSQL 6.3.1
+
 Mar 10, 1998 PL2 released
 	* add regression test for EUC_JP, EUC_CN and MULE_INTERNAL
 	* add an English document (this file)

doc/README.mb.jp

@@ -1,14 +1,12 @@
-postgresql 6.3 multi-byte (MB) patch PL2 README	       1998/3/10 $B:n@.(B
+postgresql 6.3.2 multi-byte (MB) support README	       1998/4/21 $B:n@.(B
 
 							$B@P0fC#IW(B
 						t-ishii@sra.co.jp
 		  http://www.sra.co.jp/people/t-ishii/PostgreSQL/
 
 $B$O$8$a$K!'(B
-  $B$3$N%Q%C%A$O!"%U%j!<$J(B RDBMS(Relational Database Management System)$B$N(B
-  PostgreSQL (http://www.postgresql.org/)$B$N:G?7HG(B 6.3 $B$GF|K\8l(B EUC
-  $B$J$I!"%^%k%A%P%$%HJ8;z$r07$&$3$H$r2DG=$K$9$k$?$a$N$b$N$G$9!#$3$N%Q%C(B
-  $B%A$r$"$F$k$3$H$K$h$j!"0J2<$N$3$H$,2DG=$K$J$j$^$9!#(B
+
+  PostgreSQL $B$K$*$1$k%^%k%A%P%$%H%5%]!<%H$O0J2<$N$h$&$JFCD'$r;}$C$F$$$^$9!#(B
 
     1.$B%^%k%A%P%$%HJ8;z$H$7$F!"F|K\8l!"Cf9q8l$J$I$N3F9q$N(B EUC$B!"(BUnicode$B!"(B
       mule internal code $B$,%3%s%Q%$%k;~$KA*Br2DG=!#%G!<%?%Y!<%9$K$O(B
@@ -19,45 +17,24 @@ postgresql 6.3 multi-byte (MB) patch PL2 README	       1998/3/10 $B:n@.(B
     4.$B%G!<%?$=$N$b$N$K$b%^%k%A%P%$%HJ8;z$,;HMQ2DG=(B
     5.$B%^%k%A%P%$%HJ8;z$N@55,I=8=8!:w$,;HMQ2DG=(B
     6.$B%^%k%A%P%$%HJ8;z$N(B LIKE $B8!:w$,;HMQ2DG=(B
+    7.character_length(), position(), substring() $B$G$N%^%k%A%P%$%H(B
+      $B%5%]!<%H(B
 
-    ($B$?$@$7!"(B2,3,4 $B$K$D$$$F$O%Q%C%A$r$"$F$J$/$F$b2DG=$G$9!#(B)
-
-postgresql-6.3 $B$NF~<jJ}K!!'(B
-  postgresql-6.3.tar.gz $B$O(B postgresql $B$NF|K\$G$N8x<0%_%i!<%5%$%H$G(B
-  $B$"$k(B ftp://ftp.jaist.ac.jp/pub/dbms/PostgreSQL/ $B$+$iF~<j$G$-$^$9!#(B
-  $B2?$i$+$NM}M3$G$3$3$+$iF~<j$G$-$J$$>l9g$O!"(B
-  ftp://ftp.sra.co.jp/pub/cmd/postgres/6.3/ $B$bMxMQ$G$-$^$9!#(B
-  $B$J$*!"(Bpostgresql $B$N%*%j%8%J%k(B ftp $B%5%$%H$O(B ftp://ftp.postgresql.org
-  $B$G$9!#(B
-
-$B$3$N%Q%C%A$NF~<jJ}K!!'(B
-
-  ftp://ftp.sra.co.jp/pub/cmd/postgres/6.3/patches/6.3mbPL2.patch.gz 
-  $B$rF~<j$7$F2<$5$$!#(B
-
-$B%Q%C%A$N$"$F$+$?!'(B
-  $BF~<j$7$?%Q%C%A%U%!%$%k$rE83+$7$^$9!#(B
-
-	% gunzip 6.3mbPL2.patch.gz
-
-  postgresql-6.3 $B$N%=!<%9$rE83+$7$^$9!#(B
-
-	% gtar xfz postgresql-6.3.tar.gz
+$B%$%s%9%H!<%k!'(B
+  $B%G%U%)%k%H$G$O(B PostgreSQL $B$O%^%k%A%P%$%H$r%5%]!<%H$7$F$$$^$;$s!#(B
+  $B%^%k%A%P%$%H%5%]!<%H$rM-8z$K$9$kJ}K!$r@bL@$7$^$9!#(B
 
-  $B$9$k$H!"(Bpostgresql-6.3 $B$H$$$&%G%#%l%/%H%j$,$G$-$k$N$G!"$=$3$K(B
-  cd $B$7$^$9!#(B
-
-	% cd postgresql-6.3
-
-  $B%Q%C%A$rEv$F$^$9!#(B
-
-	% patch -p1 < 6.3mbPL2.patch 
-
-  $B$H$7$F$"$F$F$/$@$5$$!#<!$K!"(Bsrc/Makefile.custom $B$H$$$&%U%!%$%k$r:n$j!"(B
+  src/Makefile.custom $B$H$$$&%U%!%$%k$r:n$j!"(B
 
 	MB=EUC_JP
 
-  $B$N(B 1 $B9T$rDI2C$7$^$9!#(BEUC_JP $B$r4^$a!"0J2<$N%3!<%I$,;XDj$G$-$^$9!#(B
+  $B$N(B 1 $B9T$rDI2C$7$^$9!#$"$k$$$O!"(Bconfigure $B5/F0;~$K0J2<$N$h$&$K;XDj$7$^$9!#(B
+
+  % configure --with-mb=EUC_JP
+
+  $BJ8;z%3!<%I$H$7$F$O(B EUC_JP $B$r4^$a!"0J2<$N%3!<%I$,;XDj$G$-$^$9!#(B
+  ($B8=:_$N<BAu$G$O!"J8;z%3!<%I$O%3%s%Q%$%k;~$K7hDj$5$l!"<B9T;~$K(B
+   $BF0E*$KJQ99$9$k$3$H$O$G$-$^$;$s(B)
 
 	EUC_JP		$BF|K\8l(B EUC
 	EUC_CN		GB $B$r%Y!<%9$K$7$?CfJ8(BEUC$B!#(Bcode set 2 $B$O(B
@@ -93,6 +70,22 @@ postgresql-6.3 $B$NF~<jJ}K!!'(B
 
 $B2~DjMzNr!'(B
 
+  1998/4/21 $B5!G=DI2C!?%P%0=$@5(B
+	* character_length(), position(), substring() $B$N%^%k%A%P%$%H(B
+	  $BBP1~(B
+	* octet_length() $BDI2C(B $B"*(B initdb $B$N$d$jD>$7I,MW(B
+	* configure $B$N%*%W%7%g%s$K(B MB $B%5%]!<%HDI2C(B
+	  (ex. configure --with-mb=EUC_JP)
+	* EUC_KR $B$N(B regression test $BDI2C(B
+	  ("Soonmyung. Hong" <hong@lunaris.hanmesoft.co.kr> $B$5$sDs6!(B)
+	* EUC_JP $B$N(B regression test $B$K(B character_length(), position(),
+	  substring(), octet_length() $BDI2C(B
+	* regress.sh $B$N(B SystemV $B$K$*$1$kHs8_49@-=$@5(B
+	* toupper(), tolower() $B$K(B 8bit $BJ8;z$,EO$k$HMn$A$k$3$H$,(B
+	  $B$"$k$N$r=$@5(B
+
+  1998/3/25 PostgreSQL 6.3.1 $B%j%j!<%9!"(BMB PL2 $B$,<h$j9~$^$l$k(B
+
   1998/3/10 PL2 $B$r%j%j!<%9(B
 	* EUC_JP, EUC_CN, MULE_INTERNAL $B$N(B regression test $B$rDI2C(B
 	  (EUC_CN $B$N%G!<%?$O(B he@sra.co.jp $B$5$sDs6!(B)

src/Makefile.global.in

@@ -7,7 +7,7 @@
 #
 #
 # IDENTIFICATION
-#    $Header: /cvsroot/pgsql/src/Makefile.global.in,v 1.40 1998/04/27 14:54:05 scrappy Exp $
+#    $Header: /cvsroot/pgsql/src/Makefile.global.in,v 1.41 1998/04/27 17:07:22 scrappy Exp $
 #
 # NOTES
 #    Essentially all Postgres make files include this file and use the 
@@ -147,6 +147,11 @@ X_CFLAGS= @X_CFLAGS@
 X_LIBS= @X_LIBS@
 X11_LIBS= -lX11 @X_EXTRA_LIBS@
 
+#
+# enable multi-byte support
+# choose one of:
+# EUC_JP,EHC_CN,EUC_KR,EUC_TW,UNICODE,MULE_INTERNAL
+MB=@MB@
 
 ##############################################################################
 #

src/backend/regex/utils.c

@@ -1,7 +1,7 @@
 /*
  * misc conversion functions between pg_wchar and other encodings.
  * Tatsuo Ishii
- * $Id: utils.c,v 1.1 1998/03/15 07:38:39 scrappy Exp $
+ * $Id: utils.c,v 1.2 1998/04/27 17:07:53 scrappy Exp $
  */
 #include <regex/pg_wchar.h>
 /*
@@ -324,25 +324,151 @@ static void pg_mule2wchar_with_len(const unsigned char *from, pg_wchar *to, int
   *to = 0;
 }
 
+static int pg_euc_mblen(const unsigned char *s)
+{
+  int len;
+
+  if (*s == SS2) {
+    len = 2;
+  } else if (*s == SS3) {
+    len = 3;
+  } else if (*s & 0x80) {
+    len = 2;
+  } else {
+    len = 1;
+  }
+  return(len);
+}
+
+static int pg_eucjp_mblen(const unsigned char *s)
+{
+  return(pg_euc_mblen(s));
+}
+
+static int pg_euckr_mblen(const unsigned char *s)
+{
+  return(pg_euc_mblen(s));
+}
+
+static int pg_eucch_mblen(const unsigned char *s)
+{
+  int len;
+
+  if (*s == SS2) {
+    len = 3;
+  } else if (*s == SS3) {
+    len = 3;
+  } else if (*s & 0x80) {
+    len = 2;
+  } else {
+    len = 1;
+  }
+  return(len);
+}
+
+static int pg_euccn_mblen(const unsigned char *s)
+{
+  int len;
+
+  if (*s == SS2) {
+    len = 4;
+  } else if (*s == SS3) {
+    len = 3;
+  } else if (*s & 0x80) {
+    len = 2;
+  } else {
+    len = 1;
+  }
+  return(len);
+}
+
+static int pg_utf_mblen(const unsigned char *s)
+{
+  int len = 1;
+
+  if ((*s & 0x80) == 0) {
+    len = 1;
+  } else if ((*s & 0xe0) == 0xc0) {
+    len = 2;
+  } else if ((*s & 0xe0) == 0xe0) {
+    len = 3;
+  }
+  return(len);
+}
+
+static int pg_mule_mblen(const unsigned char *s)
+{
+  int len;
+
+  if (IS_LC1(*s)) {
+    len = 2;
+  } else if (IS_LCPRV1(*s)) {
+    len = 3;
+  } else if (IS_LC2(*s)) {
+    len = 3;
+  } else if (IS_LCPRV2(*s)) {
+    len = 4;
+  } else {	/* assume ASCII */
+    len = 1;
+  }
+  return(len);
+}
+
 typedef struct {
-  void	(*mb2wchar)();
-  void	(*mb2wchar_with_len)();
+  void	(*mb2wchar)();		/* convert a multi-byte string to a wchar */
+  void	(*mb2wchar_with_len)();	/* convert a multi-byte string to a wchar 
+				   with a limited length */
+  int	(*mblen)();		/* returns the length of a multi-byte word */
 } pg_wchar_tbl;
 
 static pg_wchar_tbl pg_wchar_table[] = {
-  {pg_eucjp2wchar, pg_eucjp2wchar_with_len},
-  {pg_eucch2wchar, pg_eucch2wchar_with_len},
-  {pg_euckr2wchar, pg_euckr2wchar_with_len},
-  {pg_euccn2wchar, pg_euccn2wchar_with_len},
-  {pg_utf2wchar, pg_utf2wchar_with_len},
-  {pg_mule2wchar, pg_mule2wchar_with_len}};
+  {pg_eucjp2wchar, pg_eucjp2wchar_with_len, pg_eucjp_mblen},
+  {pg_eucch2wchar, pg_eucch2wchar_with_len, pg_eucch_mblen},
+  {pg_euckr2wchar, pg_euckr2wchar_with_len, pg_euckr_mblen},
+  {pg_euccn2wchar, pg_euccn2wchar_with_len, pg_euccn_mblen},
+  {pg_utf2wchar, pg_utf2wchar_with_len, pg_utf_mblen},
+  {pg_mule2wchar, pg_mule2wchar_with_len, pg_mule_mblen}};
 
+/* convert a multi-byte string to a wchar */
 void pg_mb2wchar(const unsigned char *from, pg_wchar *to)
 {
   (*pg_wchar_table[MB].mb2wchar)(from,to);
 }
 
+/* convert a multi-byte string to a wchar with a limited length */
 void pg_mb2wchar_with_len(const unsigned char *from, pg_wchar *to, int len)
 {
   (*pg_wchar_table[MB].mb2wchar_with_len)(from,to,len);
 }
+
+/* returns the byte length of a multi-byte word */
+int pg_mblen(const unsigned char *mbstr)
+{
+  return((*pg_wchar_table[MB].mblen)(mbstr));
+}
+
+/* returns the length (counted as a wchar) of a multi-byte string */
+int pg_mbstrlen(const unsigned char *mbstr)
+{
+  int len = 0;
+  while (*mbstr) {
+    mbstr += pg_mblen(mbstr);
+    len++;
+  }
+  return(len);
+}
+
+/* returns the length (counted as a wchar) of a multi-byte string 
+   (not necessarily  NULL terminated) */
+int pg_mbstrlen_with_len(const unsigned char *mbstr, int limit)
+{
+  int len = 0;
+  int l;
+  while (*mbstr && limit > 0) {
+    l = pg_mblen(mbstr);
+    limit -= l;
+    mbstr += l;
+    len++;
+  }
+  return(len);
+}