All I want for Christmas is: Destructive unpack()

The pack() function is great for really easily building strings, or
outputting to files, binary structured data:

  sub strpack_elem
  {
    my $self = shift;
    my ( $key, $value ) = @_;
    $self->{str} .= pack( "N/a* N/a*", $key, $value );
  }

  sub fpack_elem
  {
    my $self = shift;
    my ( $key, $value ) = @_;
    $self->print( pack( "N/a* N/a*", $key, $value ) );
  }

This works great because the string-contact operator and $fh->print
operation both append new data to the output, and advances the position
so the next append operation will work correctly.

The same however, sadly cannot be said of unpack(). All that unpack()
can do is return values by looking at the string of input you give it.
That string MUST be long enough, or it fails in non-obvious ways [1].
Further, unpack gives no hint on how many bytes of input it wanted to
consume, requiring any of several additional hacks to get that length
somehow. About the neatest I could write the converse operations to
theabove is

  sub strunpack_elem
  {
    my $self = shift;
    length $self->{str} >= 4 or die "Need more";
    my ( $keylen ) = unpack( "N", $self->{str} );
    length $self->{str} >= 8 + $keylen or die "Need more";
    my ( $vallen ) = unpack( "N", substr $self->{str}, 4 + $keylen, 4 );
    length $self->{str} >= 8 + $keylen + $vallen or die "Need more";

    # Now committed
    substr( $self->{str}, 0, 4, "" ); # keylen
    my $key = substr( $self->{str}, 0, $keylen, "" );
    substr( $self->{str}, 0, 4, "" ); # vallen
    my $val = substr( $self->{str}, 0, $vallen, "" );

    return ( $key, $val );
  }

  sub funpack_elem
  {
    my $self = shift;
    my $buf = $self->read( 4 );
    my ( $keylen ) = unpack( "N", $buf );
    my $key = $self->read( $keylen );

    $buf = $self->read( 4 );
    my ( $vallen ) = unpack( "N", $buf );
    my $val = $self->read( $vallen );

    return ( $key, $val );
  }

I think we can all agree that the latter functions are much more
horrible and less readable than the pack versions - these make the
logic much harder to see, for all the machinery noise in the middle. 

I wrote a C library called "libpack", somewhat based on Perl's
pack/unpack functions [2]. This library has a bunch of destructive
unpack functions which consume bytes of the input string or filehandle
as they go, meaning the equivalents of the above are much neater to
write. It would be nice to have some of the neatness of that, in Perl
as well.

-----

In summary:

  I'd like Perl to provide destructive unpack()-like functions to work
  on a string buffer or a filehandle, atomically reading and consuming
  input data as they go, to either return the full result or take no
  action.

-----

[1]: consider the following:

  eval: [ unpack "N", "abcd" ]
  [ '1633837924' ]                   # OK

  eval: [ unpack "N", "abc" ]
  [  ]                               # Fair enough

  eval: [ unpack "N n", "abc" ]
  [ '24930' ]                        # Uhhmm....

  eval: [ unpack "N a2", "abc" ]
  [ 'ab' ]                           # Go home unpack(), you are drunk


[2]: http://www.leonerd.org.uk/code/libpack/doc.html


-- 
Paul "LeoNerd" Evans

leonerd@leonerd.org.uk
http://www.leonerd.org.uk/  |  https://metacpan.org/author/PEVANS

• All I want for Christmas is: Destructive unpack() by Paul "LeoNerd" Evans
• Re: All I want for Christmas is: Destructive unpack() by H.Merijn Brand