[perl #24206] SDBM_File sometimes fails with "sdbm store() returned -1"

# New Ticket Created by  Steve Hay 
# Please include the string:  [perl #24206]
# in the subject line of all future correspondence about this issue. 
# <URL: http://rt.perl.org/rt2/Ticket/Display.html?id=24206 >


This is a bug report for perl from steve.hay@uk.radan.com,
generated with the help of perlbug 1.34 running under perl v5.8.1.


Hashes tied to SDBM_File don't seem to be able to cope with certain kinds of
data. I've experienced breakage with "real" data involving many small, 
similar
keys and large (but not too large!) values; the test program below 
reproduces
the problem using data like that:

WARNING! This program will create a *very* large file, so make sure you
don't run out of space in your filesystem!

=====
use strict;
use warnings;

use Fcntl;
use SDBM_File;

tie my %h, 'SDBM_File', 'testsdbm', O_RDWR | O_CREAT | O_TRUNC, 0666
    or die "Couldn't tie SDBM file 'testsdbm': $!\n";

for my $i (0 .. 255) {
    for my $j (0 .. 255) {
        printf "%03d - %03d\n", $i, $j;
        for my $k (0 .. 5) {
            my $key = chr($i) . chr($j) . chr($k);
            my $val = ' ' x 1000;
            $h{$key} = $val;
        }
    }
}

untie %h;
=====

The combined length of the key + value here is always 1003, which is
within the 1008 limit imposed by sdbm, so there should be no problem
there.

However, the above program always crashes with this error:

    sdbm store returned -1, errno 22, key "..."

at the point where i==1 and j==0.

I've hacked the ext/SDBM_File/sdbm/sdbm.c file in the Perl source to
include some fprintf(stderr, ...) statements in pertinent places, and I
have found that it is the last lseek() within the main while { ... }
loop in makroom() that fails. The offset that is being passed to that
lseek() is -130153472 (when formatted with %ld, which should be correct
since the offset is a "long").

At the point where it failed, the .pag file was 1.87GB (to be exact, it
was 2,017,331,200 bytes), so I wondered if this might be a "large file"
problem with a 2GB limit since my Perl doesn't have large file support?

However, I retried the same program using ActivePerl Build 806 (which
has large file support enabled) and got the same error at the same
point.

The same program completes without error if I just change SDBM_File to
DB_File. The (single) file which it creates is only about 0.5GB when the
program is finished!

---
Flags:
    category=library
    severity=medium
---
Site configuration information for perl v5.8.1:

Configured by steveh at Thu Sep 25 17:01:07 2003.

Summary of my perl5 (revision 5 version 8 subversion 1) configuration:
  Platform:
    osname=MSWin32, osvers=4.0, archname=MSWin32-x86-perlio
    uname=''
    config_args='undef'
    hint=recommended, useposix=true, d_sigaction=undef
    usethreads=undef use5005threads=undef useithreads=undef 
usemultiplicity=undef
    useperlio=define d_sfio=undef uselargefiles=undef usesocks=undef
    use64bitint=undef use64bitall=undef uselongdouble=undef
    usemymalloc=y, bincompat5005=undef
  Compiler:
    cc='cl', ccflags ='-nologo -Gf -W3 -MD -DNDEBUG -O1 -DWIN32 
-D_CONSOLE -DNO_STRICT -DHAVE_DES_FCRYPT  -DPERL_DEBUGGING_MSTATS 
-DUSE_PERLIO -DPERL_MSVCRT_READFIX',
    optimize='-MD -DNDEBUG -O1',
    cppflags='-DWIN32'
    ccversion='', gccversion='', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
    d_longlong=undef, longlongsize=8, d_longdbl=define, longdblsize=10
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', 
lseeksize=4
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='link', ldflags ='-nologo -nodefaultlib -release  
-libpath:"C:\perl5\lib\CORE"  -machine:x86'
    libpth=C:\PROGRA~1\MICROS~2\VC98\lib
    libs=  oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib  
comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib  
netapi32.lib uuid.lib wsock32.lib mpr.lib winmm.lib  version.lib 
odbc32.lib odbccp32.lib msvcrt.lib
    perllibs=  oldnames.lib kernel32.lib user32.lib gdi32.lib 
winspool.lib  comdlg32.lib advapi32.lib shell32.lib ole32.lib 
oleaut32.lib  netapi32.lib uuid.lib wsock32.lib mpr.lib winmm.lib  
version.lib odbc32.lib odbccp32.lib msvcrt.lib
    libc=msvcrt.lib, so=dll, useshrplib=yes, libperl=perl58.lib
    gnulibc_version='undef'
  Dynamic Linking:
    dlsrc=dl_win32.xs, dlext=dll, d_dlsymun=undef, ccdlflags=' '
    cccdlflags=' ', lddlflags='-dll -nologo -nodefaultlib -release  
-libpath:"C:\perl5\lib\CORE"  -machine:x86'

Locally applied patches:
   

---
@INC for perl v5.8.1:
    C:/perl5/lib
    C:/perl5/site/lib
    .

---
Environment for perl v5.8.1:
    HOME (unset)
    LANG (unset)
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    
PATH=C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\perl5\bin
    PERL5LIB (unset)
    PERL_BADLANG (unset)
    SHELL (unset)

• [perl #24206] SDBM_File sometimes fails with "sdbm store() returned -1" by Steve Hay