The fti.pl supplied with the fulltextindex module generate ALL possible substrings of two characters or greater, and is case-sensitive.
The fti.pl supplied with the fulltextindex module generate ALL possible
substrings of two characters or greater, and is case-sensitive.

This patch makes it work correctly.  It generates only the suffixes of each
word, plus lowercases them - as specified by the README file.

This brings it into line with the fti.c function, makes it case-insensitive
properly, removes the problem with duplicate rows being returned from an fti
search and greatly reduces the size of the generated index table.

It was written by my co-worker, Brett Toolin.

Christopher Kings-Lynne

greenplum-db/gpdb,arcivanov/postgres-xl,ashwinstar/gpdb,royc1/gpdb,zeroae/postgres-xl,CraigHarris/gpdb,chrishajas/gpdb,cjcjameson/gpdb,xinzweb/gpdb,janebeckman/gpdb,edespino/gpdb,0x0FFF/gpdb,ashwinstar/gpdb,Postgres-XL/Postgres-XL,postmind-net/postgres-xl,lpetrov-pivotal/gpdb,50wu/gpdb,lpetrov-pivotal/gpdb,adam8157/gpdb,edespino/gpdb,tangp3/gpdb,janebeckman/gpdb,randomtask1155/gpdb,arcivanov/postgres-xl,kaknikhil/gpdb,ashwinstar/gpdb,yuanzhao/gpdb,rubikloud/gpdb,Quikling/gpdb,lisakowen/gpdb,rvs/gpdb,CraigHarris/gpdb,oberstet/postgres-xl,cjcjameson/gpdb,50wu/gpdb,atris/gpdb,lisakowen/gpdb,royc1/gpdb,atris/gpdb,rvs/gpdb,chrishajas/gpdb,tangp3/gpdb,cjcjameson/gpdb,xinzweb/gpdb,tangp3/gpdb,atris/gpdb,zeroae/postgres-xl,0x0FFF/gpdb,xuegang/gpdb,zaksoup/gpdb,rvs/gpdb,atris/gpdb,0x0FFF/gpdb,jmcatamney/gpdb,foyzur/gpdb,janebeckman/gpdb,jmcatamney/gpdb,postmind-net/postgres-xl,greenplum-db/gpdb,ovr/postgres-xl,postmind-net/postgres-xl,kaknikhil/gpdb,randomtask1155/gpdb,tpostgres-projects/tPostgres,foyzur/gpdb,0x0FFF/gpdb,cjcjameson/gpdb,Quikling/gpdb,lintzc/gpdb,Chibin/gpdb,lisakowen/gpdb,Chibin/gpdb,Quikling/gpdb,kaknikhil/gpdb,kaknikhil/gpdb,tpostgres-projects/tPostgres,xinzweb/gpdb,janebeckman/gpdb,adam8157/gpdb,jmcatamney/gpdb,ahachete/gpdb,lisakowen/gpdb,ashwinstar/gpdb,lpetrov-pivotal/gpdb,pavanvd/postgres-xl,rvs/gpdb,royc1/gpdb,ovr/postgres-xl,tangp3/gpdb,chrishajas/gpdb,kaknikhil/gpdb,Quikling/gpdb,rubikloud/gpdb,Quikling/gpdb,zeroae/postgres-xl,Chibin/gpdb,zaksoup/gpdb,tpostgres-projects/tPostgres,xuegang/gpdb,Quikling/gpdb,zaksoup/gpdb,50wu/gpdb,snaga/postgres-xl,snaga/postgres-xl,ashwinstar/gpdb,Postgres-XL/Postgres-XL,rubikloud/gpdb,pavanvd/postgres-xl,arcivanov/postgres-xl,xuegang/gpdb,Chibin/gpdb,edespino/gpdb,xinzweb/gpdb,pavanvd/postgres-xl,ahachete/gpdb,techdragon/Postgres-XL,yazun/postgres-xl,ahachete/gpdb,techdragon/Postgres-XL,ashwinstar/gpdb,zeroae/postgres-xl,cjcjameson/gpdb,janebeckman/gpdb,rvs/gpdb,lisakowen/gpdb,adam8157/gpdb,greenplum-db/gpdb,atris/gpdb,tangp3/gpdb,janebeckman/gpdb,Postgres-XL/Postgres-XL,CraigHarris/gpdb,yuanzhao/gpdb,kaknikhil/gpdb,kaknikhil/gpdb,CraigHarris/gpdb,rubikloud/gpdb,chrishajas/gpdb,yazun/postgres-xl,techdragon/Postgres-XL,jmcatamney/gpdb,tpostgres-projects/tPostgres,xuegang/gpdb,foyzur/gpdb,0x0FFF/gpdb,rubikloud/gpdb,50wu/gpdb,jmcatamney/gpdb,royc1/gpdb,yuanzhao/gpdb,adam8157/gpdb,atris/gpdb,lisakowen/gpdb,lintzc/gpdb,xuegang/gpdb,edespino/gpdb,CraigHarris/gpdb,foyzur/gpdb,50wu/gpdb,arcivanov/postgres-xl,chrishajas/gpdb,foyzur/gpdb,jmcatamney/gpdb,CraigHarris/gpdb,xuegang/gpdb,50wu/gpdb,pavanvd/postgres-xl,cjcjameson/gpdb,rubikloud/gpdb,yuanzhao/gpdb,zaksoup/gpdb,arcivanov/postgres-xl,randomtask1155/gpdb,xinzweb/gpdb,rvs/gpdb,Postgres-XL/Postgres-XL,edespino/gpdb,0x0FFF/gpdb,techdragon/Postgres-XL,oberstet/postgres-xl,oberstet/postgres-xl,greenplum-db/gpdb,ovr/postgres-xl,rvs/gpdb,lpetrov-pivotal/gpdb,yazun/postgres-xl,kmjungersen/PostgresXL,yuanzhao/gpdb,adam8157/gpdb,greenplum-db/gpdb,edespino/gpdb,zaksoup/gpdb,jmcatamney/gpdb,Quikling/gpdb,lpetrov-pivotal/gpdb,Chibin/gpdb,Quikling/gpdb,CraigHarris/gpdb,tangp3/gpdb,lintzc/gpdb,techdragon/Postgres-XL,cjcjameson/gpdb,kmjungersen/PostgresXL,cjcjameson/gpdb,lpetrov-pivotal/gpdb,xinzweb/gpdb,edespino/gpdb,greenplum-db/gpdb,zaksoup/gpdb,zaksoup/gpdb,postmind-net/postgres-xl,0x0FFF/gpdb,0x0FFF/gpdb,snaga/postgres-xl,lpetrov-pivotal/gpdb,chrishajas/gpdb,Chibin/gpdb,yuanzhao/gpdb,foyzur/gpdb,randomtask1155/gpdb,yazun/postgres-xl,lpetrov-pivotal/gpdb,Quikling/gpdb,ovr/postgres-xl,janebeckman/gpdb,xinzweb/gpdb,rvs/gpdb,lintzc/gpdb,kaknikhil/gpdb,atris/gpdb,kmjungersen/PostgresXL,randomtask1155/gpdb,rvs/gpdb,rubikloud/gpdb,snaga/postgres-xl,edespino/gpdb,lintzc/gpdb,CraigHarris/gpdb,atris/gpdb,ahachete/gpdb,chrishajas/gpdb,foyzur/gpdb,tangp3/gpdb,50wu/gpdb,Chibin/gpdb,snaga/postgres-xl,lisakowen/gpdb,jmcatamney/gpdb,randomtask1155/gpdb,xinzweb/gpdb,zaksoup/gpdb,yuanzhao/gpdb,janebeckman/gpdb,xuegang/gpdb,cjcjameson/gpdb,Chibin/gpdb,ahachete/gpdb,Quikling/gpdb,zeroae/postgres-xl,Postgres-XL/Postgres-XL,yuanzhao/gpdb,oberstet/postgres-xl,postmind-net/postgres-xl,adam8157/gpdb,CraigHarris/gpdb,cjcjameson/gpdb,lisakowen/gpdb,kmjungersen/PostgresXL,yazun/postgres-xl,Chibin/gpdb,oberstet/postgres-xl,xuegang/gpdb,randomtask1155/gpdb,pavanvd/postgres-xl,ahachete/gpdb,tangp3/gpdb,kaknikhil/gpdb,rvs/gpdb,ahachete/gpdb,xuegang/gpdb,royc1/gpdb,ashwinstar/gpdb,lintzc/gpdb,yuanzhao/gpdb,rubikloud/gpdb,50wu/gpdb,arcivanov/postgres-xl,royc1/gpdb,tpostgres-projects/tPostgres,lintzc/gpdb,edespino/gpdb,yuanzhao/gpdb,adam8157/gpdb,adam8157/gpdb,chrishajas/gpdb,lintzc/gpdb,lintzc/gpdb,kaknikhil/gpdb,randomtask1155/gpdb,greenplum-db/gpdb,royc1/gpdb,royc1/gpdb,ahachete/gpdb,janebeckman/gpdb,edespino/gpdb,ovr/postgres-xl,ashwinstar/gpdb,foyzur/gpdb,Chibin/gpdb,greenplum-db/gpdb,kmjungersen/PostgresXL,janebeckman/gpdb