I found the libpq function PGunescapeBytea a little slow. It was taking a minute and a half to decode a 500Kb on a fairly fast machine. I think the culprit is sscanf.
I found the libpq function PGunescapeBytea a little slow. It was taking a
minute and a half to decode a 500Kb on a fairly fast machine. I think the
culprit is sscanf.

I attach a patch that replaces the function with one used to perform the same
task in pyPgSQL (a Python interface to PostgreSQL). This code was written by
Billy Allie, author of pyPgSQL. I've changed a few variable names to match
those in the original code and removed a bit of Pythonness.

Billy has kindly looked at the code and points out that it is slightly
stricter than the original implementation and if it encounters an invalid
bytea such as '\12C' it drops the unescape '\' and outputs '12C'.

The code is licensed by the author under a BSD license.

I've performed limited testing of the function by putting JPEGs into
PostgreSQL, extracting them using them using the new function and diffing
against the original files.

The new function is significantly faster on my machine with the JPEGs being
decoded in less than a second. I attach a modified libpq example program that
I used for my testing.

Ben Lamb.

CraigHarris/gpdb,Chibin/gpdb,adam8157/gpdb,tpostgres-projects/tPostgres,rvs/gpdb,Postgres-XL/Postgres-XL,rvs/gpdb,randomtask1155/gpdb,50wu/gpdb,oberstet/postgres-xl,adam8157/gpdb,cjcjameson/gpdb,lintzc/gpdb,randomtask1155/gpdb,snaga/postgres-xl,techdragon/Postgres-XL,tpostgres-projects/tPostgres,CraigHarris/gpdb,atris/gpdb,Chibin/gpdb,50wu/gpdb,ashwinstar/gpdb,randomtask1155/gpdb,ahachete/gpdb,chrishajas/gpdb,xuegang/gpdb,royc1/gpdb,ashwinstar/gpdb,cjcjameson/gpdb,oberstet/postgres-xl,50wu/gpdb,yazun/postgres-xl,arcivanov/postgres-xl,edespino/gpdb,50wu/gpdb,jmcatamney/gpdb,edespino/gpdb,chrishajas/gpdb,Chibin/gpdb,CraigHarris/gpdb,atris/gpdb,lisakowen/gpdb,atris/gpdb,randomtask1155/gpdb,xuegang/gpdb,kaknikhil/gpdb,CraigHarris/gpdb,rvs/gpdb,janebeckman/gpdb,arcivanov/postgres-xl,janebeckman/gpdb,tpostgres-projects/tPostgres,arcivanov/postgres-xl,adam8157/gpdb,edespino/gpdb,Chibin/gpdb,foyzur/gpdb,postmind-net/postgres-xl,xuegang/gpdb,Quikling/gpdb,oberstet/postgres-xl,foyzur/gpdb,ahachete/gpdb,royc1/gpdb,edespino/gpdb,rvs/gpdb,yuanzhao/gpdb,tangp3/gpdb,rubikloud/gpdb,cjcjameson/gpdb,edespino/gpdb,yazun/postgres-xl,adam8157/gpdb,kaknikhil/gpdb,postmind-net/postgres-xl,yuanzhao/gpdb,rubikloud/gpdb,xuegang/gpdb,kaknikhil/gpdb,janebeckman/gpdb,ashwinstar/gpdb,kaknikhil/gpdb,jmcatamney/gpdb,ovr/postgres-xl,0x0FFF/gpdb,lpetrov-pivotal/gpdb,kmjungersen/PostgresXL,snaga/postgres-xl,yazun/postgres-xl,chrishajas/gpdb,lintzc/gpdb,cjcjameson/gpdb,xinzweb/gpdb,atris/gpdb,janebeckman/gpdb,adam8157/gpdb,snaga/postgres-xl,0x0FFF/gpdb,rvs/gpdb,xuegang/gpdb,lpetrov-pivotal/gpdb,edespino/gpdb,Quikling/gpdb,postmind-net/postgres-xl,greenplum-db/gpdb,snaga/postgres-xl,lintzc/gpdb,janebeckman/gpdb,tangp3/gpdb,janebeckman/gpdb,0x0FFF/gpdb,cjcjameson/gpdb,lisakowen/gpdb,rubikloud/gpdb,greenplum-db/gpdb,adam8157/gpdb,randomtask1155/gpdb,janebeckman/gpdb,rubikloud/gpdb,edespino/gpdb,xinzweb/gpdb,lintzc/gpdb,Quikling/gpdb,jmcatamney/gpdb,lisakowen/gpdb,yuanzhao/gpdb,adam8157/gpdb,atris/gpdb,techdragon/Postgres-XL,Chibin/gpdb,kaknikhil/gpdb,greenplum-db/gpdb,zaksoup/gpdb,lintzc/gpdb,royc1/gpdb,greenplum-db/gpdb,ahachete/gpdb,xuegang/gpdb,rvs/gpdb,cjcjameson/gpdb,CraigHarris/gpdb,pavanvd/postgres-xl,0x0FFF/gpdb,Quikling/gpdb,foyzur/gpdb,foyzur/gpdb,royc1/gpdb,techdragon/Postgres-XL,50wu/gpdb,cjcjameson/gpdb,lpetrov-pivotal/gpdb,rvs/gpdb,greenplum-db/gpdb,CraigHarris/gpdb,cjcjameson/gpdb,Quikling/gpdb,lisakowen/gpdb,janebeckman/gpdb,0x0FFF/gpdb,kaknikhil/gpdb,tangp3/gpdb,ahachete/gpdb,lisakowen/gpdb,foyzur/gpdb,zeroae/postgres-xl,xinzweb/gpdb,kmjungersen/PostgresXL,zaksoup/gpdb,CraigHarris/gpdb,kaknikhil/gpdb,xuegang/gpdb,Chibin/gpdb,pavanvd/postgres-xl,royc1/gpdb,Postgres-XL/Postgres-XL,kaknikhil/gpdb,chrishajas/gpdb,lisakowen/gpdb,Quikling/gpdb,lintzc/gpdb,janebeckman/gpdb,CraigHarris/gpdb,yuanzhao/gpdb,ashwinstar/gpdb,lisakowen/gpdb,pavanvd/postgres-xl,ashwinstar/gpdb,zeroae/postgres-xl,xuegang/gpdb,yuanzhao/gpdb,0x0FFF/gpdb,edespino/gpdb,Quikling/gpdb,zaksoup/gpdb,Chibin/gpdb,janebeckman/gpdb,ahachete/gpdb,zaksoup/gpdb,xinzweb/gpdb,edespino/gpdb,chrishajas/gpdb,pavanvd/postgres-xl,jmcatamney/gpdb,tangp3/gpdb,rubikloud/gpdb,techdragon/Postgres-XL,50wu/gpdb,techdragon/Postgres-XL,jmcatamney/gpdb,rubikloud/gpdb,lisakowen/gpdb,50wu/gpdb,chrishajas/gpdb,xinzweb/gpdb,tangp3/gpdb,ovr/postgres-xl,yuanzhao/gpdb,ahachete/gpdb,yuanzhao/gpdb,arcivanov/postgres-xl,jmcatamney/gpdb,atris/gpdb,yuanzhao/gpdb,chrishajas/gpdb,lpetrov-pivotal/gpdb,yazun/postgres-xl,ovr/postgres-xl,ovr/postgres-xl,0x0FFF/gpdb,tpostgres-projects/tPostgres,randomtask1155/gpdb,zaksoup/gpdb,ovr/postgres-xl,xuegang/gpdb,rvs/gpdb,rvs/gpdb,lpetrov-pivotal/gpdb,yuanzhao/gpdb,postmind-net/postgres-xl,snaga/postgres-xl,tangp3/gpdb,lintzc/gpdb,yazun/postgres-xl,chrishajas/gpdb,xinzweb/gpdb,Postgres-XL/Postgres-XL,randomtask1155/gpdb,CraigHarris/gpdb,Quikling/gpdb,zeroae/postgres-xl,ashwinstar/gpdb,lpetrov-pivotal/gpdb,lintzc/gpdb,oberstet/postgres-xl,zaksoup/gpdb,kmjungersen/PostgresXL,royc1/gpdb,kaknikhil/gpdb,tpostgres-projects/tPostgres,jmcatamney/gpdb,greenplum-db/gpdb,zeroae/postgres-xl,ashwinstar/gpdb,Quikling/gpdb,zaksoup/gpdb,tangp3/gpdb,rubikloud/gpdb,cjcjameson/gpdb,kmjungersen/PostgresXL,randomtask1155/gpdb,royc1/gpdb,atris/gpdb,arcivanov/postgres-xl,Postgres-XL/Postgres-XL,oberstet/postgres-xl,zaksoup/gpdb,cjcjameson/gpdb,kaknikhil/gpdb,foyzur/gpdb,royc1/gpdb,tangp3/gpdb,greenplum-db/gpdb,0x0FFF/gpdb,yuanzhao/gpdb,ashwinstar/gpdb,rubikloud/gpdb,jmcatamney/gpdb,pavanvd/postgres-xl,xinzweb/gpdb,edespino/gpdb,ahachete/gpdb,zeroae/postgres-xl,atris/gpdb,Chibin/gpdb,arcivanov/postgres-xl,foyzur/gpdb,kmjungersen/PostgresXL,Postgres-XL/Postgres-XL,50wu/gpdb,postmind-net/postgres-xl,adam8157/gpdb,lpetrov-pivotal/gpdb,Chibin/gpdb,rvs/gpdb,foyzur/gpdb,lintzc/gpdb,ahachete/gpdb,lpetrov-pivotal/gpdb,xinzweb/gpdb,Chibin/gpdb,Quikling/gpdb,greenplum-db/gpdb