develooper Front page | perl.perl5.porters | Postings from March 2019

[perl #133949] aragraph input mode ($/="") prevents readline fromreturning latest paragraph from pipe

From:
Robert Navarro
Date:
March 24, 2019 03:15
Subject:
[perl #133949] aragraph input mode ($/="") prevents readline fromreturning latest paragraph from pipe
Message ID:
rt-4.0.24-22246-1553294898-738.133949-75-0@perl.org
# New Ticket Created by  Robert Navarro 
# Please include the string:  [perl #133949]
# in the subject line of all future correspondence about this issue. 
# <URL: https://rt.perl.org/Ticket/Display.html?id=133949 >


Subject: paragraph input mode ($/="") prevents readline from returning
latest paragraph from pipe
Message-Id: <5.10.1_6992_1553291784@centos67S>
Reply-To: rob.navarro@appdynamics.com
To: perlbug@perl.org


This is a bug report for perl from rob.navarro@appdynamics.com,
generated with the help of perlbug 1.39 running under perl 5.10.1.


-----------------------------------------------------------------
Noticed whilst trying to filter live and long-running "iostat -tmzx 3" in
time sensitive application.

The very last "paragraph" written into a pipe is not found by readline
until the next block of pipe inputs are emitted or EOF is sent.
There are no issues when reading from a file.
This is only a problem with readline in paragraph mode reading from a pipe
i.e. perl -000 ... or perl -e '$/=""; ...'

[terminal output follows]
# Firstly to prove I can read line buffered rows on right hand side of pipe
i.e. reading when they are sent without delay:

$ perl -MPOSIX -e 'select((select( STDOUT ), $|=1)[0]);for (1..3) { print
"## ".strftime( "%FT%T", localtime )." $_\n\n" for (1..3); sleep 5 }'| {
while read -r L; do echo "$(date +%FT%T)>> $L"; done; }
2019-03-22T15:22:27>> ## 2019-03-22T15:22:27 1
2019-03-22T15:22:27>>
2019-03-22T15:22:27>> ## 2019-03-22T15:22:27 2
2019-03-22T15:22:27>>
2019-03-22T15:22:27>> ## 2019-03-22T15:22:27 3
2019-03-22T15:22:27>>
2019-03-22T15:22:32>> ## 2019-03-22T15:22:32 1
2019-03-22T15:22:32>>
2019-03-22T15:22:32>> ## 2019-03-22T15:22:32 2
2019-03-22T15:22:32>>
2019-03-22T15:22:32>> ## 2019-03-22T15:22:32 3
2019-03-22T15:22:32>>
2019-03-22T15:22:37>> ## 2019-03-22T15:22:37 1
2019-03-22T15:22:37>>
2019-03-22T15:22:37>> ## 2019-03-22T15:22:37 2
2019-03-22T15:22:37>>
2019-03-22T15:22:37>> ## 2019-03-22T15:22:37 3
2019-03-22T15:22:37>>

# Notice that in above the pipe reader's timestamp matches the timestamp
sent through the pipe for each of the three "paragraphs".
# Now with Perl reading from pipe with -000 or equally problematic with
$/=""

$ perl -MPOSIX -e 'select((select( STDOUT ), $|=1)[0]);for (1..3) { print
"## ".strftime( "%FT%T", localtime )." $_\n\n" for (1..3); sleep 5 }'  |
perl -MPOSIX -000 -e 'while (<>) { print "## ".strftime( "%FT%T", localtime
)." ## >$_<"; }'
## 2019-03-22T15:22:44 ## >## 2019-03-22T15:22:44 1

<## 2019-03-22T15:22:44 ## >## 2019-03-22T15:22:44 2

<## 2019-03-22T15:22:50 ## >## 2019-03-22T15:22:44 3

<## 2019-03-22T15:22:50 ## >## 2019-03-22T15:22:50 1

<## 2019-03-22T15:22:50 ## >## 2019-03-22T15:22:50 2

<## 2019-03-22T15:22:55 ## >## 2019-03-22T15:22:50 3

<## 2019-03-22T15:22:55 ## >## 2019-03-22T15:22:55 1

<## 2019-03-22T15:22:55 ## >## 2019-03-22T15:22:55 2

<## 2019-03-22T15:23:00 ## >## 2019-03-22T15:22:55 3

# Notice that only two of the three paragraphs are output before 5 second
sleep. Pipe reader timestamp is always 5 seconds behind for third
"paragraph".

[end of terminal output]

This issue seems to occur regardless of how big or small each paragraph is
and seems to be just related to the last paragraph not being found by
readline.
There is no problem with $/="\n" or sysread from a pipe, only a problem
when paragraph mode for readline() has been enabled i.e. -000 or $/="";

Also a problem on v5.18.2



[Please do not change anything below this line]
-----------------------------------------------------------------
---
Flags:
    category=core
    severity=medium
---
This perlbug was built using Perl 5.10.1 in the Fedora build system.
It is being executed now by Perl 5.10.1 - Wed Mar 22 11:01:16 UTC 2017.

Site configuration information for perl 5.10.1:

Configured by Red Hat, Inc. at Wed Mar 22 11:01:16 UTC 2017.

Summary of my perl5 (revision 5 version 10 subversion 1) configuration:

  Platform:
    osname=linux, osvers=3.10.0-514.10.2.el7.x86_64,
archname=x86_64-linux-thread-multi
    uname='linux c1bm.rdu2.centos.org 3.10.0-514.10.2.el7.x86_64 #1 smp fri
mar 3 00:04:05 utc 2017 x86_64 x86_64 x86_64 gnulinux '
    config_args='-des -Doptimize=-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2
-fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64
-mtune=generic -DDEBUGGING=-g -Dversion=5.10.1 -Dmyhostname=localhost
-Dperladmin=root@localhost -Dcc=gcc -Dcf_by=Red Hat, Inc. -Dprefix=/usr
-Dvendorprefix=/usr -Dsiteprefix=/usr/local
-Dsitelib=/usr/local/share/perl5 -Dsitearch=/usr/local/lib64/perl5
-Dprivlib=/usr/share/perl5 -Darchlib=/usr/lib64/perl5
-Dvendorlib=/usr/share/perl5/vendor_perl
-Dvendorarch=/usr/lib64/perl5/vendor_perl -Dinc_version_list=5.10.0
-Darchname=x86_64-linux-thread-multi -Dlibpth=/usr/local/lib64 /lib64
/usr/lib64 -Duseshrplib -Dusethreads -Duseithreads -Duselargefiles
-Dd_dosuid -Dd_semctl_semun -Di_db -Ui_ndbm -Di_gdbm -Di_shadow -Di_syslog
-Dman3ext=3pm -Duseperlio -Dinstallusrbinperl=n -Ubincompat5005
-Uversiononly -Dpager=/usr/bin/less -isr -Dd_gethostent_r_proto
-Ud_endhostent_r_proto -Ud_sethostent_r_proto -Ud_endprotoent_r_proto
-Ud_setprotoent_r_proto -Ud_endservent_r_proto -Ud_setservent_r_proto
-Dscriptdir=/usr/bin -Dusesitecustomize'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=define, usemultiplicity=define
    useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
    use64bitint=define, use64bitall=define, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='gcc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing
-pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE
-D_FILE_OFFSET_BITS=64',
    optimize='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions
-fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe
-fstack-protector -I/usr/local/include'
    ccversion='', gccversion='4.4.7 20120313 (Red Hat 4.4.7-18)',
gccosandvers=''
    intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
    ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='gcc', ldflags =' -fstack-protector'
    libpth=/usr/local/lib64 /lib64 /usr/lib64
    libs=-lresolv -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lpthread -lc
    perllibs=-lresolv -lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
    libc=, so=so, useshrplib=true, libperl=libperl.so
    gnulibc_version='2.12'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E
-Wl,-rpath,/usr/lib64/perl5/CORE'
    cccdlflags='-fPIC', lddlflags='-shared -O2 -g -pipe -Wall
-Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector
--param=ssp-buffer-size=4 -m64 -mtune=generic'

Locally applied patches:


---
@INC for perl 5.10.1:
    /usr/local/lib64/perl5
    /usr/local/share/perl5
    /usr/lib64/perl5/vendor_perl
    /usr/share/perl5/vendor_perl
    /usr/lib64/perl5
    /usr/share/perl5
    .

---
Environment for perl 5.10.1:
    HOME=/home/appdyn
    LANG=en_US.UTF-8
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)

PATH=/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/appdyn/bin
    PERL_BADLANG (unset)
    SHELL=/bin/bash


-- 
*Rob Navarro* | Principal Software Engineer
Office +1 415-442-3004

*AppDynamics*
The Application Intelligence Company
Watch our video
<https://player.vimeo.com/external/137403850.hd.mp4?s=14c2030b6f8bc436aebba39b1b89ef03&profile_id=119>
  |  Try our FREE Trial
<https://portal.appdynamics.com/account/signup/signupForm>  |  Twitter
<https://twitter.com/appdynamics>  |  LinkedIn
<https://www.linkedin.com/company/appdynamics>  |  appdynamics.com
<https://www.appdynamics.com>

<http://www.appdynamics.com/appsphere>AppDynamics, Inc. is a Delaware
corporation with its principal place of business located at 303 Second
Street, North Tower, 8th Floor, San Francisco, CA 94107.
Tel: +1 415.442.8400 website: appdynamics.com <https://www.appdynamics.com>




nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About