develooper Front page | perl.perl5.porters | Postings from September 2001

RE: Wrong values passed to filter_store_key in DB_File

Thread Next
Paul Marquess
September 30, 2001 13:05
RE: Wrong values passed to filter_store_key in DB_File
Message ID:
Hi Robin,

I'm cc'ing this to p5p to see if anyone else can help.

I do think there is a problem with either DB_File or Perl or both. The
problem isn't that the DBM filter sub's are being called too often. The
issue is what filter_store_key is being called with. When the tied hash
method, NEXTKEY, gets called it should be passed the value of the previous
key that was fetched via FIRSTKEY or NEXTKEY. What is happening here is that
although it is being passed the previous value of the key, it is being given
value it was *before* filter_fetch_key modified it. In your case it is being
passed DB_ABC and DB_XYZ, when it should be passed MY_ABC and MY_XYZ.

This is even stranger, given that the perl script does print the filtered
values correctly.

I think the reason this hasn't been spotted before is because the value of
the key parameter that gets passed to NEXTKEY isn't used by Berkeley DB to
determine the next key.

GDBM_File & SDBM_File exhibit the same problem and I imagine NDBM_File &
ODBM_File will as well. I suspect the problem is the ckFilter macro that all
the *DB_File modules use.

#define ckFilter(arg,type,name)
	if (db->type) {
	    SV * save_defsv ;
          /* printf("filtering %s\n", name) ;*/
	    if (db->filtering)
	        croak("recursion detected in %s", name) ;
	    db->filtering = TRUE ;
	    save_defsv = newSVsv(DEFSV) ;
	    sv_setsv(DEFSV, arg) ;
	    PUSHMARK(sp) ;
	    (void) perl_call_sv(db->type, G_DISCARD|G_NOARGS);
	    sv_setsv(arg, DEFSV) ;
	    sv_setsv(DEFSV, save_defsv) ;
	    SvREFCNT_dec(save_defsv) ;
	    db->filtering = FALSE ;
	    /*printf("end of filtering %s\n", name) ;*/

From the tests I've carried out so far the problem only occurs if you
sequence through a *DB_File hash using "each". Both "keys" and "values"
works as expected.

I can make this problem go away for DB_File, and probably the other
*DB_File's, by changing the NEXTKEY code to ignore the key value that gets
passed into it, but that is avoiding the underlying problem.

Anyone got any idea what the problem is?


> -----Original Message-----
> From: Robin Barker []
> Sent: 28 September 2001 14:54
> To:
> Subject: Wrong values passed to filter_store_key in DB_File
> Paul
> Any thoughts on this bug.  I posted it to perlbug in April with
> no response.
> The following script:
> ----------------
> use strict;
> use DB_File 1.78;
> my $db = tie our %test, 'DB_File' or die "Failed opening DB_File: $!\n";
> $db -> filter_fetch_key ( sub { s/^DB_/MY_/ or warn "Bad fetch
> key $_\n"; } );
> $db -> filter_store_key ( sub { s/^MY_/DB_/ or warn "Bad store
> key $_\n"; } );
> $test{MY_ABC}= 'abc';
> $test{MY_XYZ}= 'xyz';
> while( my($k,$v) = each %test ) { printf "%-8s => %-8s\n", $k, $v;  }
> ----------------
> Produces
> 	MY_XYZ   => xyz
> 	Bad store key DB_XYZ
> 	MY_ABC   => abc
> 	Bad store key DB_ABC
> Values which have been already filtered by filter_store_key are being
> resubmitted to filter_store_key.  I can't work out where this is
> happening, or even if it is bug in DB_File.xs, mg.c or db itself.
> Robin
> --
> Robin Barker                        | Email:
> CMSC, Building 10,                  | Phone: +44 (0) 20 8943 7090
> National Physical Laboratory,       | Fax:   +44 (0) 20 8977 7091
> Teddington, Middlesex, UK. TW11 OLW | WWW:

Do You Yahoo!?
Get your free address at

Thread Next Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at | Group listing | About