Re: All I want for Christmas is: Destructive unpack()

On Thu, 25 Dec 2014 15:03:15 +0000, "Paul \"LeoNerd\" Evans"
<leonerd@leonerd.org.uk> wrote:

> The pack() function is great for really easily building strings, or
> outputting to files, binary structured data:
> 
>   sub strpack_elem
>   {
>     my $self = shift;
>     my ( $key, $value ) = @_;
>     $self->{str} .= pack( "N/a* N/a*", $key, $value );
>   }
> 
>   sub fpack_elem
>   {
>     my $self = shift;
>     my ( $key, $value ) = @_;
>     $self->print( pack( "N/a* N/a*", $key, $value ) );
>   }
> 
> This works great because the string-contact operator and $fh->print
> operation both append new data to the output, and advances the position
> so the next append operation will work correctly.
> 
> The same however, sadly cannot be said of unpack(). All that unpack()
> can do is return values by looking at the string of input you give it.
> That string MUST be long enough, or it fails in non-obvious ways [1].
> Further, unpack gives no hint on how many bytes of input it wanted to
> consume, requiring any of several additional hacks to get that length
> somehow. About the neatest I could write the converse operations to
> theabove is

I want unpack on streams!

See for a great example http://www.perlmonks.org/?node_id=1104462

ikegami's answer:
--8<---
my $template = "iifff";
my $rec_size = template_len ($template); # not shown
while (1) {
   my $rv = read ($fh, my $rec, $rec_size);
   defined $rv     or  die "$!\n";
   $rv             or  last;
   $rv < $rec_size and die "Premature EOF\n";

   my @fields = unpack $template, $rec;
   ...
   }
-->8---

could be shortened to

while (my @fields = unpack "iifff" => $fh) {
   ...
   }

>   sub strunpack_elem
>   {
>     my $self = shift;
>     length $self->{str} >= 4 or die "Need more";
>     my ( $keylen ) = unpack( "N", $self->{str} );
>     length $self->{str} >= 8 + $keylen or die "Need more";
>     my ( $vallen ) = unpack( "N", substr $self->{str}, 4 + $keylen, 4 );
>     length $self->{str} >= 8 + $keylen + $vallen or die "Need more";
> 
>     # Now committed
>     substr( $self->{str}, 0, 4, "" ); # keylen
>     my $key = substr( $self->{str}, 0, $keylen, "" );
>     substr( $self->{str}, 0, 4, "" ); # vallen
>     my $val = substr( $self->{str}, 0, $vallen, "" );
> 
>     return ( $key, $val );
>   }
> 
>   sub funpack_elem
>   {
>     my $self = shift;
>     my $buf = $self->read( 4 );
>     my ( $keylen ) = unpack( "N", $buf );
>     my $key = $self->read( $keylen );
> 
>     $buf = $self->read( 4 );
>     my ( $vallen ) = unpack( "N", $buf );
>     my $val = $self->read( $vallen );
> 
>     return ( $key, $val );
>   }
> 
> I think we can all agree that the latter functions are much more
> horrible and less readable than the pack versions - these make the
> logic much harder to see, for all the machinery noise in the middle. 
> 
> I wrote a C library called "libpack", somewhat based on Perl's
> pack/unpack functions [2]. This library has a bunch of destructive
> unpack functions which consume bytes of the input string or filehandle
> as they go, meaning the equivalents of the above are much neater to
> write. It would be nice to have some of the neatness of that, in Perl
> as well.
> 
> -----
> 
> In summary:
> 
>   I'd like Perl to provide destructive unpack()-like functions to work
>   on a string buffer or a filehandle, atomically reading and consuming
>   input data as they go, to either return the full result or take no
>   action.
> 
> -----
> 
> [1]: consider the following:
> 
>   eval: [ unpack "N", "abcd" ]
>   [ '1633837924' ]                   # OK
> 
>   eval: [ unpack "N", "abc" ]
>   [  ]                               # Fair enough
> 
>   eval: [ unpack "N n", "abc" ]
>   [ '24930' ]                        # Uhhmm....
> 
>   eval: [ unpack "N a2", "abc" ]
>   [ 'ab' ]                           # Go home unpack(), you are drunk
> 
> 
> [2]: http://www.leonerd.org.uk/code/libpack/doc.html
> 
> 


-- 
H.Merijn Brand  http://tux.nl   Perl Monger  http://amsterdam.pm.org/
using perl5.00307 .. 5.21   porting perl5 on HP-UX, AIX, and openSUSE
http://mirrors.develooper.com/hpux/        http://www.test-smoke.org/
http://qa.perl.org   http://www.goldmark.org/jeff/stupid-disclaimers/

• All I want for Christmas is: Destructive unpack() by Paul "LeoNerd" Evans
• Re: All I want for Christmas is: Destructive unpack() by H.Merijn Brand