From: Moritz Lenz TSa wrote:
> HaloO,
>
> Larry Wall wrote:
>> The operator iterates each function until the function fails to
>> produce any list elements....
>
> What is the list type these days? It used to be List for writeable
> lists and Seq for readonly ones.

Array = mutable
List = immutable, lazy
Seq = immutable, eager

Moritz
--
Moritz Lenz
http://perlgeek.de/ | http://perl-6.de/ | http://sudokugarden.de/

From: TSa HaloO,

Larry Wall wrote:
> The operator iterates each function until the function fails to
> produce any list elements....

What is the list type these days? It used to be List for writeable
lists and Seq for readonly ones. But isn't the latter superseded by
Capture? I would think it handy to have a type like Range that allows
to introspect the list e.g. to get the routine that calculates the
next item when given a previous one or the routine that calculates
the nth item in a random access fashion. Random access for a numeric
Range is particularly easy:

multi method postcircumfix:<[ ]> ( Range[Num] $range, Int $index )
{
my $ret = $range.from + $index * $range.by;
return $ret <= $range.to ?? $ret !! undef;
}

How would the corresponding method look like for List?

multi method postcircumfix:<[ ]> ( List $list, Int $index )
{
my &nth = $list.nth; # get nth item generator

if &nth
{
return nth($index);
}
else
{
# use iterator interface
}
}

Note that every function that takes an Int as first parameter could then
be wrapped up as a list with random access.

> Note you can go the other way too: the
> function can actually return multiple list elements at once, so you
> can interleave two (or more) independent lists:
>
> 0,1 ... { $^even + 2, $^odd + 2 }
>
> Interestingly, since the returned list is a capture, such a list
> in slice context would turn into [0,1],[2,3],[4,5]..., In list
> context it's just 0,1,2,3,4,5
>
> I think that's just wicked cool.

Very subtle it is. And I wonder how exactly this is implemented.
I mean the pairs returned from the generator are captured into
what, protoarrays that auto-flatten into a list and become real
arrays in a slice?


Regards, TSa.
--

"The unavoidable price of reliability is simplicity" -- C.A.R. Hoare
"Simplicity does not precede complexity, but follows it." -- A.J. Perlis
1 + 2 + 3 + 4 + ... = -1/12 -- Srinivasa Ramanujan

From: larry Author: larry
Date: Mon Oct 6 18:15:17 2008
New Revision: 14588

Modified:
doc/trunk/design/syn/S05.pod

Log:
Added ~ twiddle macro to make it easier to write bracketing constructs.


Modified: doc/trunk/design/syn/S05.pod
==============================================================================
--- doc/trunk/design/syn/S05.pod (original)
+++ doc/trunk/design/syn/S05.pod Mon Oct 6 18:15:17 2008
@@ -14,9 +14,9 @@
Maintainer: Patrick Michaud <pmichaud@pobox.com> and
Larry Wall <larry@wall.org>
Date: 24 Jun 2002
- Last Modified: 7 Jul 2008
+ Last Modified: 6 Oct 2008
Number: 5
- Version: 83
+ Version: 84

This document summarizes Apocalypse 5, which is about the new regex
syntax. We now try to call them I<regex> rather than "regular
@@ -685,6 +685,59 @@

[ <ident> !~~ 'moose' ] || 'squirrel'

+=item *
+
+The C<~> operator is a helper for matching nested subrules with a
+specific terminator as the goal. It appears to be placed between the
+opening and closing bracket, like so:
+
+ '(' ~ ')' <expression>
+
+However, it mostly ignores the left argument, and operates on the next
+two atoms (which may be quantified). Its operation on those next
+two atoms is to "twiddle" them so that they are actually matched in
+reverse order. Hence the expression above, at first blush, is merely
+shortand for:
+
+ '(' <expression> ')'
+
+But beyond that, when it rewrites the atoms it also inserts the
+apparatus that will set up the inner expression to recognize the
+terminator, and to produce an appropriate error message if the
+inner expression does not terminate on the required closing atom.
+So it really does pay attention to the left bracket as well, and it
+actually rewrites our example to something more like:
+
+ $<OPEN> = '(' <SETGOAL: ')'> <expression> [ $GOAL || <FAILGOAL> ]
+
+Note that you can use this construct to set up expectations for
+a closing construct even when there's no opening bracket:
+
+ <null> ~ ')' \d+
+
+By default the error message uses the name of the current rule as an
+indicator of the abstract goal of the parser at that point. However,
+often this is not terribly informative, especially when rules are named
+according to an internal scheme that will not make sense to the user.
+The C<:dba> ("doing business as") adverb may be used to set up a more informative name for
+what the following code is trying to parse:
+
+ token postfix:sym<[ ]> {
+ :dba<array subscript>
+ '[' ~ ']' <expression>
+ }
+
+Then instead of getting a message like:
+
+ Unable to parse expression in postfix:sym<[ ]>; couldn't find final ']'
+
+you'll get a message like:
+
+ Unable to parse expression in array subscript; couldn't find final ']'
+
+(The C<:dba> adverb may also be used to give names to alternations
+and alternatives, which helps the lexer give better error messages.)
+
=back

=head1 Bracket rationalization

From: Larry Wall On Mon, Oct 06, 2008 at 09:09:55AM -0700, Jon Lang wrote:
: Larry Wall wrote:
: > On Sun, Oct 05, 2008 at 08:19:42PM -0700, Jon Lang wrote:
: > : <larry@cvs.perl.org> wrote:
: > : > Log:
: > : > Add missing series operator, mostly for readability.
: > :
: > : Is there a way for the continuing function to access its index as well
: > : as, or instead of, the values of one or more preceding terms? And/or
: > : to access elements by counting forward from the start rather than
: > : backward from the end?
: >
: > That's what the other message was about. @_ represents the entire list
: > generated so far, so you can look at its length or index it from the
: > beginning. Not guaranteed to be as efficient though.
:
: If I understand you correctly, an "All even numbers" list could be written as:
:
: my @even = () ... { 2 * +@_ }
:
: And the Fibonacci series could be written as:
:
: my @fib = () ... { (pow((1 + sqrt(5))/2, +@_) - pow((1 - sqrt(5))/2,
: +@_)) / sqrt(5)) }
:
: Mind you, these are bulkier than the versions described in the patch.
: And as presented, they don't have any advantage to offset their
: bulkiness, because you still have to determine every intervening
: element in sequential order. If I could somehow replace '+@_' in the
: above code with an integer that identifies the element that's being
: asked for, it would be possible to skip over the unnecessary elements,
: leaving them undefined until they're directly requested. So:
:
: say @fib[4];
:
: would be able to calculate the fifth fibonacci number without first
: calculating the prior four.
:
: It's possible that the '...' series operator might not be the right
: way to provide random access to elements. Perhaps there should be two
: series operators, one for sequential access (i.e., 'infix:<...>') and
: one for random access (e.g., 'infix:<...[]>'). This might clean
: things up a lot: the sequential access series operator would feed the
: last several elements into the generator:
:
: 0, 1 ... -> $a, $b { $a + $b }
:
: while the random access series operator would feed the requested index
: into the generator:
:
: () ...[] -> $n { (pow((1 + sqrt(5))/2, $n) - pow((1 - sqrt(5))/2,
: $n)) / sqrt(5)) }
:
: I'd suggest that both feed the existing array into @_.

That's what the ++(state $) example was about in the other message.
The only downside is that it's assuming only one element is feed in
on the left. Otherwise you need to initialize the anonymous state
variable to some larger number. Maybe it's an idiom where you just
always initialize $n to the number of elements on the left:

0,1,2 ... { state $n = 3; $n++ }

I don't see much need for additional relief, especially since
it's normally trivial to write such lists other ways:

map { func($^n) } 0..*
func($_) for 0..*
for 0..* { .func }

: On an additional note: the above patch introduces some ambiguity into
: the documentation. Specifically, compare the following three lines:
:
: X List infix Z minmax X X~X X*X XeqvX ...
: R List prefix : print push say die map substr ... [+] [*] any $ @
:
: N Terminator ; <==, ==>, <<==, ==>>, {...}, unless, extra ), ], }
:
: On the first line, '...' is the name of an operator; on the second and
: third lines, '...' is documentation intended to mean "...and so on"
: and "yadda-yadda", respectively. However, it is not immediately
: apparent that this is so: a casual reader will be inclined to read the
: first line as "...and so on" rather than 'infix:<...>', and will not
: realize his error until he gets down to the point where the series
: operator is defined.

No, the second one not metasyntactic--it's the prefix:<...>, also known
as yada. It's essentially a synonym for fail, and so takes a list
argument. The third one is not yada--it's metasyntactic to represent
any "expect infix" block that stops the current expression such as in

if $x { say "yup" }

Admittedly the table is rather terse.

: __________________________
: Another question: what would the following do?
:
: 0 ... { $_ + 2 } ... &infix:<+> ... *
:
: If I'm reading it right, this would be the same as:
:
: infix:<...> (0; { $_ + 2 }; &infix:<+>; *)
:
: ...but beyond that, I'm lost.

It would never get to the + or the * because $_ + 2 is never ().
The operator iterates each function until the function fails to
produce any list elements. Note you can go the other way too: the
function can actually return multiple list elements at once, so you
can interleave two (or more) independent lists:

0,1 ... { $^even + 2, $^odd + 2 }

Interestingly, since the returned list is a capture, such a list
in slice context would turn into [0,1],[2,3],[4,5]..., In list
context it's just 0,1,2,3,4,5...

I think that's just wicked cool.

Larry

From: Jon Lang Larry Wall wrote:
> On Sun, Oct 05, 2008 at 08:19:42PM -0700, Jon Lang wrote:
> : <larry@cvs.perl.org> wrote:
> : > Log:
> : > Add missing series operator, mostly for readability.
> :
> : Is there a way for the continuing function to access its index as well
> : as, or instead of, the values of one or more preceding terms? And/or
> : to access elements by counting forward from the start rather than
> : backward from the end?
>
> That's what the other message was about. @_ represents the entire list
> generated so far, so you can look at its length or index it from the
> beginning. Not guaranteed to be as efficient though.

If I understand you correctly, an "All even numbers" list could be written as:

my @even = () ... { 2 * +@_ }

And the Fibonacci series could be written as:

my @fib = () ... { (pow((1 + sqrt(5))/2, +@_) - pow((1 - sqrt(5))/2,
+@_)) / sqrt(5)) }

Mind you, these are bulkier than the versions described in the patch.
And as presented, they don't have any advantage to offset their
bulkiness, because you still have to determine every intervening
element in sequential order. If I could somehow replace '+@_' in the
above code with an integer that identifies the element that's being
asked for, it would be possible to skip over the unnecessary elements,
leaving them undefined until they're directly requested. So:

say @fib[4];

would be able to calculate the fifth fibonacci number without first
calculating the prior four.

It's possible that the '...' series operator might not be the right
way to provide random access to elements. Perhaps there should be two
series operators, one for sequential access (i.e., 'infix:<...>') and
one for random access (e.g., 'infix:<...[]>'). This might clean
things up a lot: the sequential access series operator would feed the
last several elements into the generator:

0, 1 ... -> $a, $b { $a + $b }

while the random access series operator would feed the requested index
into the generator:

() ...[] -> $n { (pow((1 + sqrt(5))/2, $n) - pow((1 - sqrt(5))/2,
$n)) / sqrt(5)) }

I'd suggest that both feed the existing array into @_.
__________________________
> : It would be nice if the programmer
> : were given the tools to do this sort of thing explicitly instead of
> : having to rely on the optimizer to know how to do this implicitly.
>
> Um, I don't understand what you're asking for. Explicit solutions
> are always available...

This was a reaction to something I (mis)read in the patch, concerning
what to do when the series operator is followed by a 'whatever'.
Please ignore.
__________________________
On an additional note: the above patch introduces some ambiguity into
the documentation. Specifically, compare the following three lines:

X List infix Z minmax X X~X X*X XeqvX ...
R List prefix : print push say die map substr ... [+] [*] any $ @

N Terminator ; <==, ==>, <<==, ==>>, {...}, unless, extra ), ], }

On the first line, '...' is the name of an operator; on the second and
third lines, '...' is documentation intended to mean "...and so on"
and "yadda-yadda", respectively. However, it is not immediately
apparent that this is so: a casual reader will be inclined to read the
first line as "...and so on" rather than 'infix:<...>', and will not
realize his error until he gets down to the point where the series
operator is defined.
__________________________
Another question: what would the following do?

0 ... { $_ + 2 } ... &infix:<+> ... *

If I'm reading it right, this would be the same as:

infix:<...> (0; { $_ + 2 }; &infix:<+>; *)

...but beyond that, I'm lost.
__________________________
Jonathan "Dataweaver" Lang

From: Michael G Schwern So, the concrete use-case I'm thinking of here is currency.


Darren Duncan wrote:
>> [2] "Num" should have an optional limit on the number of decimal places
>> it remembers, like NUMERIC in SQL, but that's a simple truncation.
>
> I disagree.
>
> For starters, any "limit" built into a type definition should be defined
> not as stated above but rather with a simple subtype declaration, eg
> "subtype of Rat where ..." that tests for example that the Rat is an
> exact multiple of 1/1000.
>
> Second, any truncation should be done at the operator level not at the
> type level; for example, the rational division operator could have an
> optional extra argument that says the result must be rounded to be an
> exact multiple of 1/1000; without the extra argument, the division
> doesn't truncate anything.
>
> Any numeric operations that would return an irrational number in the
> general case, such as sqrt() and sin(), and the user desires the result
> to be truncated to an exact rational number rather than as a symbolic
> number, then those operators should have an extra argument that
> specifies rounding, eg to an exact multiple of 1/1000.

That seems like scattering a lot of redundant extra arguments around. The
nice thing about doing it as part of the type is you just specify it once.

But instead of truncating data in the type, maybe what I want is to leave the
full accuracy inside and instead override string/numification to display only
2 decimal places.

But sometimes I want to hang onto fractional pennies and use them in
calculation, just hide them from view. And sometimes I don't.


> Note, a generic numeric rounding operator would also take the "exact
> multiple of" argument rather than a "number of digits" argument, except
> when that operator is simply rounding to an integer, in which case no
> such argument is applicable.
>
> Note, for extra determinism and flexibility, any operation
> rounding/truncating to a rational would also take an optional argument
> specifying the rounding method, eg so users can choose between the likes
> of half-up, to-even, to-zero, etc. Then Perl can easily copy any
> semantics a user desires, including when code is ported from other
> languages and wants to maintain exact semantics.

Yes, this is very important for currency operations.


> Now, as I see it, if "Num" has any purpose apart from "Rat", it would be
> like a "whatever" numeric type or effectively a union of the
> Int|Rat|that-symbolic-number-type|etc types, for people that just want
> to accept numbers from somewhere and don't care about the exact
> semantics. The actual underlying type used in any given situation would
> determine the exact semantics. So Int and Rat would be exact and
> unlimited precision, and maybe Symbolic or IRat or something would be
> the symbolic number type, also with exact precision components.

That sounds right. It's the "whatever can conceivably be called a number" type.



--
44. I am not the atheist chaplain.
-- The 213 Things Skippy Is No Longer Allowed To Do In The U.S. Army
http://skippyslist.com/list/

From: Darren Duncan Nicholas Clark wrote:
> If one has floating point in the mix [and however much one uses rationals,
> and has the parser store all decimal string constants as rationals, floating
> point enters the mix as soon as someone wants to use transcendental functions
> such as sin(), exp() or sqrt()], I can't see how any implementation that wants
> to preserve "infinite" precision for as long as possible is going to work,
> apart from
>
> storing every value as a thunk that holds the sequence of operations that
> were used to compute the value, and defer calculation for as long as
> possible. (And possibly as a sop to efficiency, cache the floating point
> outcome of evaluating the thunk if it gets called)

Floating point has no effect on this. A float is just a syntax for an exact
rational. What you're talking about mostly is how an exact irrational would be
represented, symbolically. The result of an exact numeric sin() etc would be a
symbolic number, and it is only when you separately explicitly convert it to a
rational does any rounding and loss occur, and it would be the rounding
operation itself that has extra parameters to control the rounding.
Alternately, an inexact numeric sin() that results in an exact rational is just
a wrapper over the last 2 operations combined, and it takes rounding-control
parameters. I see no problem here. -- Darren Duncan

From: Geoffrey Broadwell On Sun, 2008-10-05 at 17:05 -0700, larry@cvs.perl.org wrote:
> +C<< infix:<...> >>, the series operator.

Lovely, just lovely.

> + 1, 3, 5 ... * # odd numbers
> + 1. 2. 4 ... * # powers of 2

Did you mean to use commas on that second line?


-'f

From: Tom Christiansen In-Reply-To: Message from Nicholas Clark <nick@ccl4.org>
of "Sun, 05 Oct 2008 22:13:14 BST." <20081005211314.GA6160@plum.flirble.org>

> Studiously ignoring that request to nail down promotion and demotion, I'm
> going to jump straight to implementation, and ask:

> If one has floating point in the mix [and however much one uses rationals,
> and has the parser store all decimal string constants as rationals, floating
> point enters the mix as soon as someone wants to use transcendental functions
> such as sin(), exp() or sqrt()], I can't see how any implementation that wants
> to preserve "infinite" precision for as long as possible is going to work,
> apart from

> storing every value as a thunk that holds the sequence of operations that
> were used to compute the value, and defer calculation for as long as
> possible. (And possibly as a sop to efficiency, cache the floating point
> outcome of evaluating the thunk if it gets called)

>Nicholas Clark

My dear Nicholas,

You mentioned sin(), exp(), and sqrt() as being "transcendental" functions,
but this is not true! Perhaps you meant something more in the way of their
being--um, "irrational". but far be it from me to risk using so loaded a
word in reference to anyone's hypothetical intentions or rationale! :-)

While all transcendentals are indeed irrationals, the opposite relationship
does *not* apply. It's all really rather simple, provided you look at it
as a brief, binary decision-tree.

=> All reals are also one of either rational or irrational:

+ Rational numbers are those expressible as the RATIO of I/J,
where I is any integer and J any non-zero integer.

- Irrationals are all other reals *EXCEPT* the rationals.

=> All irrationals are also one of either algebraic or transcendental:

+ Algebraic numbers are solutions to polynomial equations of a single
variable and integer coefficients. When you solve for x in the
polynomial equation 3*x**2 - 15 == 0, you get an algebraic number.

- Transcendentals are all other irrationals *EXCEPT* the algebraics.

Thinking of the sine function and its inverse, I notice that
sin(pi/2) == 1 and asin(1) is pi/2. Pi is *the* most famous
of transcendental numbers, and sin() is a transcendental function.

Thinking of the exponential function and its inverse, I notice that
exp(1) == e and log(e) == 1. And e, "Euler's number", is likely the
#2 most famous transcendental, and exp() is a transcendental function.

However, we come now to a problem.

If you solved the simple equation I presented above as one whose solution
was by definition *not* a transcendental but rather an algebraic number,
you may have noticed that solution is 5**(1/2), better known as sqrt(5).
So that makes sqrt(5) an algebraic number, and sqrt() is an algebraic
function, which means therefore that it is *not* a transcendental one.

Q.E.D. :-)

Ok, I was teasing a little. But I'd now like to politely and sincerely
inquire into your assertion that floating point need inevitably enter the
picture just to determine sin(x), exp(x), or sqrt(x).

Your last one, sqrt(), isn't hard at all. Though I no longer recall the
algorithm, there exists one for solving square roots by hand that is only a
little more complicated than that of solving "long division" by hand. Like
division, it is an iterative process, somewhat tedious but quite well-defined
easily implemented even on pen and paper. Perhaps that has something to do
with sqrt() being an algebraic function. :-) j/k

As for the two transcendental functions, this does ask for more work. But
it's not as though we do not understand them, nor how to derive them at
need from first principles! They aren't magic black-ball functions with
secret look-up tables that when poked with a given input, return some
arbitrary answer.

We *know* how to *do* these!

Sure, many and probably most solutions, at least for the transendentals, do
involve power series, and usually Taylor Series. But this only means that
you get to joyfully sum up an infinite sequence of figures receding into
infinity ("And beyond!" quoth Buzz), but where each figure in said series
tends to be a reasonably simple and straightforward computation.

For example, each term in the Taylor Series for exp(x) is simply x**N / N!,
and the final answer the sum of all suchterms for N going from 0 to infinity.
Its series is therefore

x**0 / 0! # er: that's just 1, of course :-)
+ x**1 / 1!
+ x**2 / 2!
+ x**3 / 3!
+ x**4 / 4!
+
+ + + + + + + + + + ad infinitum.

For sin(x), it's a bit harder, but not much: the series is a convergent one
of alternating sign, running N from 0 to infinity and producing a series
that looks like this:

(x**1 / 1!) # er: that's just x, of course :-)
- (x**3 / 3!)
+ (x**5 / 5!)
- (x**7 / 7!)
+ (x**9 / 9!)
-
+ - + - + - + - + ad infinitum.

Each term in the sin(x) series is still a comparitively easy one,
reading much better on paper than on the computer with lazy ASCII
equations, for I assume you'd be subkeen on eqn(1) descriptions. :)

( (-1)**N * (x ** (2*N - 1)) )
/
(2*N + 1)!

where that's a factorial bang, not an exclamatory one. :-)

Now you're very nearly done: all you have left to do to get that perfect
answer is add up infinitely many individual terms, for the limit of that
summed-up infinite series is none other than the *exact* answer to those
transcendental functions. They don't have algebraic solutions, but they
do have algorithmic ones, and this is not rocket science. On the other
hand, you probably can't even *do* rocket science without such nifty
mathematical devices. :-)

Nick, you and I both know the secret of infinity, although your average
pedestrian--call him Zeno--will surely falter in its pursuit. And that
secret is that infinity means nothing fancier than "as many as I jolly well
please, thank you very much". In other words, you just keep going (and
going (and going)), until finally you arrive at however many digits of
precision you've been asked to produce.

Now, just where in those equations I've presented above do you see some
need for and need to resort to lossy floating-point numbers? I don't see
any. Sure, I do see plenty of places for Memoize-style caching of these
numbers of infinite (read: arbitrarily great) precision.

But where is the need floating-point that you seem to see, and why?

Beyong the rockety equations I just presented above, I also have to offer
you not just one, but *two* existence-proofs that you've no need to resort
to machine-native floating-point variables or instructions.

The first is bc(1). It reports that the square root of 5 played
out to to 300 digits of precision is:

$ (echo scale=300; echo 'sqrt(5)') | bc -l
2.2360679774997896964091736687312762354406183596115257242708972454105\
209256378048994144144083787822749695081761507737835042532677244470738\
635863601215334527088667781731918791658112766453226398565805357613504\
175337850034233924140644420864325390972525926272288762995174024406816\
11775908909498492371390729

Here are 50 digits of precision for sin(1):

$ (echo scale=50; echo 's(1)') | bc -l
.84147098480789650665250232163029899962256306079837

And here are 500 digits for exp(1)

$ (echo scale=500; echo 'e(1)') | bc -l
2.7182818284590452353602874713526624977572470936999595749669676277240\
766303535475945713821785251664274274663919320030599218174135966290435\
729003342952605956307381323286279434907632338298807531952510190115738\
341879307021540891499348841675092447614606680822648001684774118537423\
454424371075390777449920695517027618386062613313845830007520449338265\
602976067371132007093287091274437470472306969772093101416928368190255\
151086574637721112523897844250569536967707854499699679468644549059879\
3163688923009879312

So, that's one existence proof.

And if you look at Math::BigFloat, you'll fine the other.
Or bignum, if you prefer.


% perl -Mbignum=p,-305 -E 'say sqrt(5)'
2.23606797749978969640917366873127623544061835961152572427089724541052092563780489941441440837878227496950817615077378350425326772444707386358636012153345270886677817319187916581127664532263985658053576135041753378500342339241406444208643253909725259262722887629951740244068161177590890949849237139072972890

or formatted the way bc does it:

2.2360679774997896964091736687312762354406183596115257242708972454105\
209256378048994144144083787822749695081761507737835042532677244470738\
635863601215334527088667781731918791658112766453226398565805357613504\
175337850034233924140644420864325390972525926272288762995174024406816\
1177590890949849237139072972890

And sin(1) to a ways out:

% perl -Mbignum=p,-55 -E 'say sin(1)'
0.8414709848078965066525023216302989996225630607983710657

Followed by a good 500+ digits of e, with formatting:

% perl -Mbignum=p,-505 -E 'say exp(1)'

2.7182818284590452353602874713526624977572470936999595749669676277240\
766303535475945713821785251664274274663919320030599218174135966290435\
729003342952605956307381323286279434907632338298807531952510190115738\
341879307021540891499348841675092447614606680822648001684774118537423\
454424371075390777449920695517027618386062613313845830007520449338265\
602976067371132007093287091274437470472306969772093101416928368190255\
151086574637721112523897844250569536967707854499699679468644549059879\
316368892300987931277362

So, um, what's that floating-point thing about again, eh? :-)

--tom

From: larry Author: larry
Date: Sun Oct 5 17:05:41 2008
New Revision: 14586

Modified:
doc/trunk/design/syn/S03.pod

Log:
Add missing series operator, mostly for readability.


Modified: doc/trunk/design/syn/S03.pod
==============================================================================
--- doc/trunk/design/syn/S03.pod (original)
+++ doc/trunk/design/syn/S03.pod Sun Oct 5 17:05:41 2008
@@ -12,9 +12,9 @@

Maintainer: Larry Wall <larry@wall.org>
Date: 8 Mar 2004
- Last Modified: 1 Oct 2008
+ Last Modified: 5 Oct 2008
Number: 3
- Version: 140
+ Version: 141

=head1 Overview

@@ -49,7 +49,7 @@
R Item assignment = := ::= => += -= **= xx= .=
L Loose unary true not
X Comma operator , p5=>
- X List infix Z minmax X X~X X*X XeqvX
+ X List infix Z minmax X X~X X*X XeqvX ...
R List prefix : print push say die map substr ... [+] [*] any $ @
X Loose and and andthen
X Loose or or xor orelse
@@ -1584,6 +1584,75 @@

See L</Cross operators>.

+=item *
+
+C<< infix:<...> >>, the series operator.
+
+This operator takes a concrete list on its left and a function to be
+iterated on its right when the list must be extended.
+
+The value of the operator is the lazy list formed of the
+concrete list followed by the result of applying the function to the
+tail of the list as needed. The function indicates by its arity
+how many of the preceding values to pay attention to (and which the
+operator must track internally). Demonstration of this falls to the
+lot of the venerable Fibonacci sequence:
+
+ 1, 1 ... { $^y + $^z } # 1,1,2,3,5,8...
+ 1, 1 ... &infix:<+> # 1,1,2,3,5,8...
+
+More typically the function is unary, in which case any extra values
+in the list may be construed as human-readable documentation:
+
+ 0,2,4 ... { $_ + 2 } # same as 1..*:by(2)
+ <a b c> ... { .succ } # same as 'a'..*
+
+The function need not be monotonic, of course:
+
+ 1 ... { -$_ } # 1, -1, 1, -1, 1, -1...
+ False ... &prefix:<!> # False, True, False...
+
+The function can be 0-ary as well:
+
+ () ... &rand # list of random numbers
+
+If the right operand is C<*> (Whatever) and the sequence is obviously
+arithmetic or geometric, the appropriate function is deduced:
+
+ 1, 3, 5 ... * # odd numbers
+ 1. 2. 4 ... * # powers of 2
+
+Conjecture: other such patterns may be recognized in the future,
+depending on which unrealistic benchmark we want to run faster. C<:)>
+
+Note: the yada operator is recognized only where a term is expected.
+This operator may only be used where an infix is expected. If you
+put a comma before the C<...> it will be taken as a yada list operator
+expressing the desire to fail when the list reaches that point:
+
+ 1..20, ... "I only know up to 20 so far mister"
+
+If the yada operator finds a closure for its argument at compile time,
+it should probably whine about the fact that it's difficult to turn
+a closure into an error message. Alternately, we could treat
+an elipsis as special when it follows a comma to better support
+traditional math notation.
+
+The function may choose to terminate its list by returning ().
+Since this operator is list associative, an inner function may be
+followed by a C<...> and another function to continue the list,
+and so on. Hence,
+
+ 1 ... { $_ + 1 if $_ < 10 }
+ ... { $_ + 10 if $_ < 100 }
+ ... { $_ + 100 if $_ < 1000 }
+
+produces
+
+ 1,2,3,4,5,6,7,8,9,
+ 10,20,30,40,50,60,70,80,90,
+ 100,200,300,400,500,600,700,800,900
+
=back

Many of these operators return a list of Captures, which depending on

From: Nicholas Clark On Sat, Oct 04, 2008 at 09:37:29PM -0700, Mark Biggar wrote:

> trivial and vice versa. But promotion (or demotion) between IEEE floats
> and rationals is really hard and I don't know of a language that even
> tries. The major problem is that the demotion from rational to IEEE
> float is very lossy. In general, there are many possible distinct
> rationals that convert into the same IEEE value and converting IEEE
> float to the simplest rational out of that set is a very expensive
> operation.. Once you're in rationals you probably never want to demote

And I can't see why it's anything other than a heuristic. For example,
0.2 is an infinite binary fraction. So store that as an IEEE float (heck,
in any form of float with a binary mantissa of fixed precision) and 1/5 isn't
the *only* rational that could have got you there. Something like

0b00011001100110011001100110011001100110011001100110011 /
0b10000000000000000000000000000000000000000000000000000

(900719925474099 / 4503599627370496)

would also result in the same bit pattern in the mantissa.

How does the implementation tell which rational the floating point value
actually came from

> back to IEEE (except possibly at the very end). Every language that
> supports rationals, that I know of, leaves it up to the programmer to
> decide whether they will be doing computations in IEEE float or
> rationals and do not try to automatically convert back and forth. I
> looked at thos and basically gave up when I was writing the perl 5
> Bigint and Bigrat packages.
>
> Before you discuss implementations, you should define exactly what rules
> you are going to use for promotion and demotion between the various types.

Studiously ignoring that request to nail down promotion and demotion, I'm
going to jump straight to implementation, and ask:

If one has floating point in the mix [and however much one uses rationals,
and has the parser store all decimal string constants as rationals, floating
point enters the mix as soon as someone wants to use transcendental functions
such as sin(), exp() or sqrt()], I can't see how any implementation that wants
to preserve "infinite" precision for as long as possible is going to work,
apart from

storing every value as a thunk that holds the sequence of operations that
were used to compute the value, and defer calculation for as long as
possible. (And possibly as a sop to efficiency, cache the floating point
outcome of evaluating the thunk if it gets called)


Nicholas Clark

From: Tom Christiansen In-Reply-To: Message from Michael G Schwern <schwern@pobox.com>
of "Sat, 04 Oct 2008 02:06:18 EDT." <48E707DA.3070706@pobox.com>

> Larry Wall wrote:
>> The status of numeric upgrades in Perl 6 is fine. It's rakudo that
>> doesn't do so well. :)
>>
>> As another datapoint:
>>
>> $ pugs -e 'say 2**40'
>> 1099511627776
>> $ pugs -e 'say 2**50'
>> 1125899906842624
>> $ pugs -e 'say 2**1100'
>> 1358298529049385849277351428359266778603493846931744549748519669727813<SNIP>

> That's good [1] to hear, thanks.

>> I don't think of Int as a type that automatically upgrades. I think
>> of it as an arbitrarily large integer that the implementation can in
>> some cases choose to optimize to a smaller or faster representation,

> Oh don't worry, I do. I just got so flustered when I saw Rakudo do the
> same thing that Perl 5 does I was worried this got lost somewhere along
> the line.

> [1] We need a polite way to say "less bad".

!
ah
fab
yay
good
cool
ayup
d'oh!
helps
tasty
yummy
smooth
cheers
better
yippee!
soothes
pleases
niftier
relieves
mediates
inspires
mollifies
mitigates
clarifies
my mistake
oh, right!
grrrrreat!
delightful
not to worry
oh be joyful!
'tain't so bad
calms my qualms
less sub-optimal
cheers my spirit
soothes my nerves
dispells my doubts
heartens my resolve
cushions the cudgel
dismisses my dismay
drives out the dread
comforts me to learn
inspires me with hope
restores my confidence
gladdens my good humor
brightens my rainy day
alleviates my concerns
mollifies my misgivings
alleviates my confusion
puts down the false alarm
perks/plucks up my courage
pacifies my preoccupations
banishes my paranoia-demons
felicitates my facilitation
facilitates my felicitation
assuages my misapprehensions
shows I was worrying too much
encourages me; is encouraging
warms the cockles of my heart
offers hope for a better world
trounces my tetchy trepidations
dispells my misplaced anxieties
sure puts a spiffier shine on it
makes molehills out of mountains
eases up on my nerves a fair bit
not nearly so gnarly as I'd feared
'tis not too late to seek a newer world
way better than I'd half-begun to suspect
patches the potholes in my crumbling wetware
softens the imagined blow that wasn't even there to start with
serenades such sweet sonnets as to nullify nervous nellies' natterings

From: Michael G Schwern Larry Wall wrote:
> On Fri, Oct 03, 2008 at 11:57:30PM -0400, Michael G Schwern wrote:
> : What's the status of numeric upgrades in Perl 6? Is see the docs say "Perl 6
> : intrinsically supports big integers and rationals through its system of type
> : declarations. Int automatically supports promotion to arbitrary precision" but
> : it looks like it's doing the same thing as Perl 5.
> :
> : $ ./perl6 -e 'say 2**40'
> : 1099511627776
> :
> : $ ./perl6 -e 'say 2**50'
> : 1.12589990684262e+15
> :
> : $ ./perl6 -e 'say 2**1100'
> : inf
>
> The status of numeric upgrades in Perl 6 is fine. It's rakudo that
> doesn't do so well. :)
>
> As another datapoint:
>
> $ pugs -e 'say 2**40'
> 1099511627776
> $ pugs -e 'say 2**50'
> 1125899906842624
> $ pugs -e 'say 2**1100'
> 13582985290493858492773514283592667786034938469317445497485196697278130927542418487205392083207560592298578262953847383475038725543234929971155548342800628721885763499406390331782864144164680730766837160526223176512798435772129956553355286032203080380775759732320198985094884004069116123084147875437183658467465148948790552744165376

That's good [1] to hear, thanks.


> I don't think of Int as a type that automatically upgrades. I think
> of it as an arbritrarily large integer that the implementation can in
> some cases choose to optimize to a smaller or faster representation,

Oh don't worry, I do. I just got so flustered when I saw Rakudo do the same
thing that Perl 5 does I was worried this got lost somewhere along the line.


[1] We need a polite way to say "less bad".


--
The mind is a terrible thing,
and it must be stopped.

From: Larry Wall On Fri, Oct 03, 2008 at 11:57:30PM -0400, Michael G Schwern wrote:
: What's the status of numeric upgrades in Perl 6? Is see the docs say "Perl 6
: intrinsically supports big integers and rationals through its system of type
: declarations. Int automatically supports promotion to arbitrary precision" but
: it looks like it's doing the same thing as Perl 5.
:
: $ ./perl6 -e 'say 2**40'
: 1099511627776
:
: $ ./perl6 -e 'say 2**50'
: 1.12589990684262e+15
:
: $ ./perl6 -e 'say 2**1100'
: inf

The status of numeric upgrades in Perl 6 is fine. It's rakudo that
doesn't do so well. :)

As another datapoint:

$ pugs -e 'say 2**40'
1099511627776
$ pugs -e 'say 2**50'
1125899906842624
$ pugs -e 'say 2**1100'
13582985290493858492773514283592667786034938469317445497485196697278130927542418487205392083207560592298578262953847383475038725543234929971155548342800628721885763499406390331782864144164680730766837160526223176512798435772129956553355286032203080380775759732320198985094884004069116123084147875437183658467465148948790552744165376

I don't think of Int as a type that automatically upgrades. I think
of it as an arbritrarily large integer that the implementation can in
some cases choose to optimize to a smaller or faster representation,
but only if that can be transparent to the user (apart from the size
and speed effects). Certainly this should be the case for anything
declared "Int". A native "int" is another matter, but that's just
a storage constraint, and probably shouldn't influence intermediate
computations, which are done in Int. So it shouldn't matter whether 2
and 1100 are stored in Int or int form; the result should be the same
(subject, perhaps to pragmatic control of temp types).

Larry

From: Michael G Schwern What's the status of numeric upgrades in Perl 6? Is see the docs say "Perl 6
intrinsically supports big integers and rationals through its system of type
declarations. Int automatically supports promotion to arbitrary precision" but
it looks like it's doing the same thing as Perl 5.

$ ./perl6 -e 'say 2**40'
1099511627776

$ ./perl6 -e 'say 2**50'
1.12589990684262e+15

$ ./perl6 -e 'say 2**1100'
inf

And...

$ ./perl6 -e 'my Int $foo = 2**32; say $foo'
Type check failed
current instr.: 'parrot;Perl6Object;infix:=' pc 60 (src/gen_builtins.pir:52)
called from Sub '_block11' pc 98 (EVAL_12:41)
called from Sub 'parrot;PCT::HLLCompiler;eval' pc 806
(src/PCT/HLLCompiler.pir:480)
called from Sub 'parrot;PCT::HLLCompiler;command_line' pc 1298
(src/PCT/HLLCompiler.pir:707)
called from Sub 'parrot;Perl6::Compiler;main' pc 17029 (perl6.pir:172)
perl6(4325) malloc: *** Deallocation of a pointer not malloced: 0x4157170;
This could be a double free(), or free() called with the middle of an
allocated block; Try setting environment variable MallocHelp to see tools to
help debug
perl6(4325) malloc: *** Deallocation of a pointer not malloced: 0xda71b; This
could be a double free(), or free() called with the middle of an allocated
block; Try setting environment variable MallocHelp to see tools to help debug
Bus error (core dumped)

though hopefully that's transient.


--
"Clutter and overload are not an attribute of information,
they are failures of design"
-- Edward Tufte

From: Jon Lang Timothy S. Nelson wrote:
>> <TimToady> note to treematching folks: it is envisaged that signatures in
>> a rule will match nodes in a tree
>>
>> My question is, how is this expected to work? Can someone give an
>> example?
>
> I'm assuming that this relates to Jon Lang's comment about using
> rules to match non-strings.

Pretty much - although there are some patterns that one might want to
use that can't adequately be expressed in this way - at least, not
without relaxing some of the constraints on signature definition.
Some examples:

A signature always anchors its "positional parameters" pattern to the
first and last positional parameters (analogous to having implicit '^'
and '$' markup at the start and end of a textual pattern), and does
not provide any sort of "zero or more"/"one or more" qualifiers, other
than a single tail-end "slurpy list" option. Its "zero or one"
qualifier is likewise constrained in that once you use an optional
positional, you're limited to optionals and slurpies from that point
on. This makes it difficult to set up a pattern that matches, e.g.,
"any instance within the list of a string followed immediately by a
number".

The other issue that signatures-as-patterns doesn't handle very well
is that of capturing and returning matches. I suppose that this could
be handled, to a limited extent, by breaking the signature up into
several signatures joined together by <,>, and then indicating which
"sub-signatures" are to be returned; but that doesn't work too well
once hierarchal arrangements are introduced.

Perhaps an approach more compatible with normal rules syntax might be
to introduce a series of xml-like tags:

<[> ... <]> lets you denote a nested list of patterns - analogous to
what [ ... ] does outside of rules. Within its reach, '^' and '$'
refer to "just before the first element" and "just after the last
element", respectively. Otherwise, this works just like the "list of
objects and/or strings" patterns currently described in S05.

<{> ... <}> does likewise with a nested hash of values, with standard
pair notation being used within in order to link key patterns to value
patterns. Since hashes are not ordered, '^' and '$' would be
meaningless within this context. Heck, order in general is
meaningless within this context.

<item> replaces <elem> as the object-based equivalent of '.' ('elem'
is too list-oriented of a term). I'd recommend doing this even if you
don't take either of the suggestions above.

You might even do a <[[> ... <]]> pairing to denote a list that is
nested perhaps more than one layer down. Or perhaps that could be
handled by using '<[>+' or the like.

> But how would it be if I wanted to search a tree for all nodes
> whose "readonly" attribute were true, and return an array of
> those nodes?

This can already be done, for the most part:

/ (<.does(ro)>) /

Mind you, this only searches a list; to make it search a tree, you'd
need a drill-down subrule such as I outline above:

/ <[>* (<.does(ro)>) <]>* /

--
Jonathan "Dataweaver" Lang

From: Timothy S. Nelson On Fri, 3 Oct 2008, Timothy S. Nelson wrote:

> On Fri, 3 Oct 2008, Timothy S. Nelson wrote:
>
>> On Thu, 2 Oct 2008, Timothy S. Nelson wrote:
>>
>>> Now that Perl6 is in the mix, though, I think that the best way to do
>>> it is to make roles that model eg. Nodes, Plexes (Documents), Elements,
>>> and the like, and then have operators on them do all the work (like my
>>> idea of using a slash for a combined feed and call code operator). I
>>> could be wrong, but it seems to me that we could get something that's
>>> somewhat like XPath this way, without having to worry about defining an
>>> XPath grammar or anything.
>>
>> I'm talking to myself here :). The guys on IRC convinced me that the
>> way to go might be something like a grammar, but that does trees and tree
>> transformations instead of a text input stream. See the IRC log for
>> details :).
>
> Talking to myself again. I'm not as convinced as I was. I'll write
> up a long post about that if necessary, but want to get something else
> figured out first. First, a paste from the IRC log:
>
> <TimToady> note to treematching folks: it is envisaged that signatures in a
> rule will match nodes in a tree
>
> My question is, how is this expected to work? Can someone give an
> example?

I'm assuming that this relates to Jon Lang's comment about using rules
to match non-strings.

I'm starting to see how *matching* would work now. But how would it
be if I wanted to search a tree for all nodes whose "readonly" attribute were
true, and return an array of those nodes? Hmm. Or something like the
following XPath...

/html/body//p/a[@name = "#SampleName"]

(my XPath isn't that great, but I'm assuming this will find all a tags
in the html body that are the direct descendant of a paragraph, and have their
@name attribute set to "#SampleName"). My guess is something like this:

$htmlobject = HTML->new();

$htmlobject.children()
==> grep { /html/ } ==> map { .children() }
==> grep { /body/ } ==> map { .children() }
==> recursivegrep { /p/ } ==> map { .children() }
==> grep { /a/ and .name eq "#Samplename" } ==> $anchors

I'm guessing that might do it (although the tree role and
recursivegrep would also require some work). But the syntax is dreadful :).
You can see why I was talking about having a feed operator that did a grep
and got children as well as doing the feed. But no doubt I'm missing
something here. And I'm still thinking like a Perl5 programmer :).

:)


---------------------------------------------------------------------
| Name: Tim Nelson | Because the Creator is, |
| E-mail: wayland@wayland.id.au | I am |
---------------------------------------------------------------------

----BEGIN GEEK CODE BLOCK----
Version 3.12
GCS d+++ s+: a- C++$ U+++$ P+++$ L+++ E- W+ N+ w--- V-
PE(+) Y+>++ PGP->+++ R(+) !tv b++ DI++++ D G+ e++>++++ h! y-
-----END GEEK CODE BLOCK-----