This site is a static rendering of the Trac instance that was used by R7RS-WG1 for its work on R7RS-small (PDF), which was ratified in 2013. For more information, see Home.
Source for wiki GeneratorsGauche version 2
author
cowan
comment
ipnr
127.11.51.1
name
GeneratorsGauche
readonly
0
text
{{{
#!html
<h2 class="section">Generators</h2>
<p>A generator is merely a procedure with no arguments that works
as a source of a series of values. Every time it is called,
it yields a value. Returning an eof object indicates the generator is exhausted.
For example, <code>read-char</code> can be seen as a generator that
generates characters from the current input port.
</p>
<p>It is common practice to abstract the source of values in such a way,
so it is useful to define utility procedures that work on the
generators. This module provides them.
</p>
<p>A generator is very lightweight, and handy to implement simple
on-demand calculations. However, keep in mind that it is
side-effecting construct; you can't safely backtrack, for example..
</p>
<p>You can construct, combine, and consume various generators with this library.
</p>
<h3 class="subsection">Generator constructors</h3>
<p>A generator isn't a special datatype but just an ordinary procedure,
so you can make a generator with lambdas. This module provides
some common generator constructors for convenience.
</p>
<p>If you want to use your procedure as a generator, note that a
generator can be invoked many times even after it returns eof once.
You have to code it so that once it returns eof, it keeps returning
eof for the subsequent calls.
</p>
<p>The result of generator constructors is merely a procedure,
and printing it doesn't show much. In the examples in this section
we use <code>generator->list</code> to convert the generator to the list.
See <a href="#Generator-operations">Generator operations</a> for the description of <code>generator->list</code>.
</p>
<p>These procedures have names beginning with <code>make-</code>
and ending with <code>-generator</code>.
<dl>
<dt><a name="index-null_002dgenerator"></a>'<code>make-element-generator</code><i> arg …</i></dt>
<dd><p>The simplest generator. Returns the given arguments followed by an eof object when called.
</p></dd></dl>
<dl>
<dt><a name="index-circular_002dgenerator"></a>'<code>make-circular-generator</code><i> arg …</i></dt>
<dd><p>Returns an infinite generator that repeats the given arguments forever.
</p>
<table><tr><td> </td><td><pre class="example">(generator->list (circular-generator 1 2 3) 10)
⇒ (1 2 3 1 2 3 1 2 3 1)
</pre></td></tr></table>
<p>Note that the above example limits the length of
the converted list by 10; otherwise
<code>generator->list</code> won't return.
</p></dd></dl>
<dl>
<dt><a name="index-giota"></a>'<code>make-iota-generator</code><i> [ count [ start [ step ] ] ]</i></dt>
<dd><p>Creates a generator
of a series of <var>count</var> numbers (by default, an infinite number), starting from <var>start</var> (0 by default)
and increased by <var>step</var> (1 by default).
</p>
<table><tr><td> </td><td><pre class="example">(generator->list (make-iota-generator 10 3 2))
⇒ (3 5 7 9 11 13 15 17 19 21)
</pre></td></tr></table>
<p>If both <var>start</var> and <var>step</var> are exact, the generator
yields exact numbers; otherwise it yields inexact numbers.
</p>
<table><tr><td> </td><td><pre class="example">(generator->list (giota +inf.0 1/2 1/3) 6)
⇒ (1/2 5/6 7/6 3/2 11/6 13/6)
(generator->list (giota +inf.0 1.0 2.0) 5)
⇒ (1.0 3.0 5.0 7.0 9.0)
</pre></td></tr></table>
</dd></dl>
<dl>
<dt><a name="index-grange"></a>'<code>make-range-generator</code><i> start end [ step ]</i></dt>
<dd><p>Creates a generator of a series of
numbers. The series begins with <var>start</var>, increases by <var>step</var> (default 1),
and continues while the number is below <var>end</var>.
</p>
<table><tr><td> </td><td><pre class="example">(generator->list (make-range-generator 3 8))
⇒ (3 4 5 6 7)
</pre></td></tr></table>
</dd></dl>
<dl>
<dt><a name="index-generate"></a>'<code>make-coroutine-generator</code><i> proc</i></dt>
<dd><p>Creates a generator from a coroutine.
</p>
<p>The <var>proc</var> argument is a procedure that takes one argument,
<var>yield</var>. When called, <code>generate</code> immediately returns
a generator <var>g</var>. When <var>g</var> is called, <var>proc</var> runs
until it calls <var>yield</var>. Calling <var>yield</var> causes
the execution of <var>proc</var> to be suspended, and <var>g</var> returns the value passed
to <var>yield</var>.
</p>
<p>Once <var>proc</var> returns, it is the end of the series — <var>g</var> returns an
eof object from then on. The return value of <var>proc</var> is ignored.
</p>
<p>The following code creates a generator that produces a series
0, 1, and 2 (effectively the same as <code>(make-iota-generator 3)</code> and binds
it to <code>g</code>.
</p>
<table><tr><td> </td><td><pre class="example">(define g
(generate
(lambda (yield) (let loop ((i 0))
(when (< i 3) (yield i) (loop (+ i 1)))))))
(generator->list g) ⇒ (0 1 2)
</pre></td></tr></table>
</dd></dl>
<dl>
<dt><a name="index-list_002d_003egenerator"></a>'<code>make-list-generator</code><i> lis :optional start end</i></dt>
<dt><a name="index-vector_002d_003egenerator"></a>'<code>make-vector-generator</code><i> vec :optional start end</i></dt>
<dt><a name="index-reverse_002dvector_002d_003egenerator"></a>'<code>make-reverse-vector-generator</code><i> vec [ start [ end ] ]</i></dt>
<dt><a name="index-string_002d_003egenerator"></a>'<code>make-string-generator</code><i> str :optioanl start end</i></dt>
<dd><p>These procedures return generator that yields each item in the given argument.
A generator returned from <code>make-vector-reverse-generator</code> procedures runs in
reverse order.
</p>
<table><tr><td> </td><td><pre class="example">(make-generator-list (list->generator '(1 2 3 4 5)))
⇒ (1 2 3 4 5)
(generator->list (make-vector-generator '#(1 2 3 4 5)))
⇒ (1 2 3 4 5)
(generator->list (make-reverse-vector-generator '#(1 2 3 4 5)))
⇒ (5 4 3 2 1)
(generator->list (make-string-generator "abcde"))
⇒ (#\a #\b #\c #\d #\e)
</pre></td></tr></table>
<p>By default the generator is exhausted once all items are retrieved;
the optional <var>start</var> and <var>end</var> arguments can limit the range
the generator walks across.
</p>
<p>For forward generators, the first value the generator yields
is <var>start</var>-th element, and it ends right before <var>end</var>-th element.
For reverse generators, the first value is the item right before
to the <var>end</var>-th element, and the last value is the <var>start</var>-th
element.
</p>
<table><tr><td> </td><td><pre class="example">(generator->list (vector->generator '#(a b c d e) 2))
⇒ (c d e)
(generator->list (make-vector-generator '#(a b c d e) 2 4))
⇒ (c d)
(generator->list (make-reverse-vector-generator '#(a b c d e) 2))
⇒ (e d c b)
(generator->list (make-reverse-vector-generator '#(a b c d e) 2 4))
⇒ (d c)
(generator->list (make-reverse-vector-generator '#(a b c d e) #f 2))
⇒ (b a)
</pre></td></tr></table>
</dd></dl>
<dl>
<dt><a name="index-bits_002d_003egenerator"></a>'<code>make-bits-generator</code><i> n</i></dt>
<dd><p>This procedure takes an exact integer and treats it as a sequence of
boolean values (0 for false and 1 for true), taking bits from
the least significant bit.
</p>
<table><tr><td> </td><td><pre class="example">(generator->list (bits->generator #b10110))
⇒ (#f #t #t #f #t)
</pre></td></tr></table>
</dd></dl>
<dl>
<dt><a name="index-port_002d_003esexp_002dgenerator"></a>'<code>make-port-sexp-generator</code><i> input-port</i></dt>
<dt><a name="index-port_002d_003eline_002dgenerator"></a>'<code>make-port-line-generator</code><i> input-port</i></dt>
<dt><a name="index-port_002d_003echar_002dgenerator"></a>'<code>make-port-char-generator</code><i> input-port</i></dt>
<dt><a name="index-port_002d_003ebyte_002dgenerator"></a>'<code>make-port-byte-generator</code><i> input-port</i></dt>
<dd><p>Returns generators that read characters or bytes from the given
port, respectively.
</p></dd></dl>
<dl>
<dt><a name="index-x_002d_003egenerator"></a><code>make-for-each-generator</code><i> obj for-each</i></dt>
<dd><p>A generic version to convert any collection <var>obj</var> to a generator
that walks across the <var>obj</var> using the <var>for-each</var> procedure
appropriate for <var>obj</var>.
</p></dd></dl>
<dt><a name="index-gunfold"></a>'<code>make-gunfold-generator</code><i> p f g seed</i></dt>
<dd><p>A generator constructor similar to <var>unfold</var>.
</p>
<p><var>P</var> is a predicate that takes a seed value and determines
where to stop. <var>F</var> is a procedure that calculates a value
from a seed value. <var>G</var> is a procedure that calculates the
next seed value from the current seed value.
</p>
<p>For each call of the resulting generator, <var>p</var> is called with
the current seed value. If it returns true, then we see we've
done, and we return an eof object. Otherwise,
we apply <var>f</var> on the current seed value to get the value to
return, and use <var>g</var> to update the seed value.
</p>
<table><tr><td> </td><td><pre class="example">(generator->list (gunfold (^s (> s 5)) (^s (* s 2)) (^s (+ s 1)) 0))
⇒ '(0 2 4 6 8 10)
</pre></td></tr></table>
</dd></dl>
<h3 class="subsection">Generator operations</h3>
<p>The following procedures take generators (noted as <var>gen</var> and
<var>gen2</var>) and return a generator. For the convenience, they
also accept any collection to <var>gen</var> and <var>gen2</var> parameters;
if a collection is passed where a generator is expected,
it is implicitly coerced into a generator.
</p>
<p>The names of these procedures begin with <code>g</code>.
</p>
<dl>
<dt><a name="index-gcons_002a"></a>'<code>gcons*</code><i> item … gen</i></dt>
<dd><p>Returns a generator that adds <var>item</var>s in front of <var>gen</var>.
</p>
<table><tr><td> </td><td><pre class="example">(generator->list (gcons* 'a 'b (giota 2)))
⇒ (a b 0 1)
</pre></td></tr></table>
</dd></dl>
<dl>
<dt><a name="index-gappend"></a>'<code>gappend</code><i> gen …</i></dt>
<dd><p>Returns a generator that yields the items from the first given
generator, and once it is exhausted, use the second generator, and so on.
</p>
<table><tr><td> </td><td><pre class="example">(generator->list (gappend (giota 3) (giota 2)))
⇒ (0 1 2 0 1)
(generator->list (gappend))
⇒ ()
</pre></td></tr></table>
</dd></dl>
<dl>
<dt><a name="index-gconcatenate"></a>'<code>gconcatenate</code><i> gen</i></dt>
<dd><p>The <var>gen</var> argument should generate generators and/or sequences.
Returns a generator that yields elements from the first generator/sequence,
then the second one, then the third, etc.
</p>
<p>It is similar to <code>(apply gappend (generator->list gen))</code>, except
that <code>gconcatenate</code> can work even <var>gen</var> generates infinite
number of generators.
</p>
<table><tr><td> </td><td><pre class="example">($ generator->list $ gconcatenate
$ list->generator `(,(giota 3) ,(giota 2)))
⇒ (0 1 2 0 1)
</pre></td></tr></table>
</dd></dl>
<dl>
<dt><a name="index-gmerge"></a>'<code>gmerge</code><i> comparator gen gen2 …</i></dt>
<dt><a name="index-gunion"></a>'<code>gunion</code><i> comparator gen gen2 …</i></dt>
<dt><a name="index-gintersection"></a>'<code>gintersection</code><i> comparator gen gen2 …</i></dt>
<dd><p>Creates a generator that yields elements out of input generators,
with the order determined by a SRFI 114 comparator.
<code>Gmerge</code> returns all the elements;
<code>Gunion</code> returns all the elements that are distinct in the sense of the comparator;
<code>Gmerge</code> returns all the elements that appear in all the generators.
</p>
<p>Each input generator must yield appropriately ordered elements by itself;
otherwise the result won't be ordered.
</p>
<p>If only one generator is given, it is just returned.
In that case, <var>constructor</var> is ignored.
</p>
<table><tr><td> </td><td><pre class="example">(generator->list (gmerge < '(1 3 8) '(5) '(2 4)))
⇒ '(1 2 3 4 5 8)
</pre></td></tr></table>
</dd></dl>
<dl>
<dt><a name="index-gmap"></a>'<code>gmap</code><i> proc gen gen2 …</i></dt>
<dd><p>Returns a generator that yields a value returned by <var>proc</var>
applied on the values from given generators. The returned generator
terminates when any of the given generator is exhausted.
</p>
<p>Note: This differs from <code>generator-collect</code>,
which consumes all
values at once and returns the results as a list, while <code>gmap</code>
returns a generator immediately without consuming input.
</p></dd></dl>
<dl>
<dt><a name="index-gmap_002daccum"></a>'<code>gmap-accum</code><i> proc seed gen gen2 …</i></dt>
<dd><p>A generator version of <code>map-accum</code>,
mapping with states.
</p>
<p>The <var>proc</var> argument is a procedure that takes as many arguments
as the input generators plus one. It is called as
<code>(proc v v2 … seed)</code> where <code>v</code>, <code>v2</code>, … are
the values yielded from the input generators, and <var>seed</var> is the
current seed value. It must return two values, the yielding value
and the next seed.
</p></dd></dl>
<dl>
<dt><a name="index-gfilter"></a>'<code>gfilter</code><i> pred gen</i></dt>
<dt><a name="index-gremove"></a>'<code>gfilter</code><i> pred gen</i></dt>
<dd><p>Returns a generator that yield the items from the source generator,
except those on which <var>pred</var> answers false or true respectively.
</p></dd></dl>
<dl>
<dt><a name="index-gfilter_002dmap"></a>'<code>gfilter-map</code><i> proc gen gen2 …</i></dt>
<dd><p>Works the same as <code>(gfilter values (gmap proc gen gen2 …))</code>,
only slightly more efficiently.
</p></dd></dl>
<dl>
<dt><a name="index-gstate_002dfilter"></a>'<code>gstate-filter</code><i> proc seed gen</i></dt>
<dd><p>This allows stateful filtering of a series. The <var>proc</var> argument
must be a procedure that takes a value <var>V</var> from the source generator and
a seed value. It should return two values, a boolean flag and the next
seed value. If it returns true for the boolean flag, the generator
yields <var>V</var>. Otherwise, the generator keeps calling <var>proc</var>,
with updating the seed value, until it sees the true flag value
or the source generator is exhausted.
</p>
<p>The following example takes a generator of oscillating values
and yields only values that are greater than their previous value.
</p>
<table><tr><td> </td><td><pre class="example">(generator->list
(gstate-filter (^[v s] (values (< s v) v)) 0
(list->generator '(1 2 3 2 1 0 1 2 3 2 1 0 1 2 3))))
⇒ (1 2 3 1 2 3 1 2 3)
</pre></td></tr></table>
</dd></dl>
<dl>
<dt><a name="index-gbuffer_002dfilter"></a>'<code>gbuffer-filter</code><i> proc seed gen :optional tail-gen</i></dt>
<dd><p>This procedure allows n-to-m mapping between elements in input and output—
that is, you can take a look at serveral input elements to generate
one or more output elements.
</p>
<p>The procedure <var>proc</var> receives the next input element and accumulated
seed value. It returns two values: A list of output values, and the next
seed value. If you need to look at more input to generate
output, you can return an empty list as the first value.
</p>
<p>If the input reaches the end, <var>tail-gen</var> is called with the
current seed value; it should return a list of last output values.
If omitted, the output ends when the output of the last call to <var>proc</var>
is exhausted (the last seed value is discarded).
</p>
<p>Suppose you have a text file. Each line contains a command,
but if the line ends with backslash, next line is treated as a
continuation of the current line. The following code creates
a generator that returns <em>logical</em> lines, that is,
the lines after such line continuations are taken care of.
</p>
<table><tr><td> </td><td><pre class="example">(gbuffer-filter (^[v s]
(if-let1 m (#/\\$/ v)
(values '() (cons (m 'before) s))
(values `(,(string-concatenate-reverse (cons v s))) '())))
'()
(file->line-generator "input-file.txt")
(^[s] `(,(string-concatenate-reverse s))))
</pre></td></tr></table>
</dd></dl>
<dl>
<dt><a name="index-gtake"></a>'<code>gtake</code><i> gen k [ padding ]</i></dt>
<dt><a name="index-gdrop"></a>'<code>gdrop</code><i> gen k</i></dt>
<dd><p>The generator version of
<code>take</code> and <code>drop</code>.
<code>Gtake</code> returns a generator that yields (at most) the first <var>k</var> items
of the source generator, while <code>gdrop</code> returns a generator
that skips the first <var>k</var> items of the source generator.
</p>
<p>These won't complain if the source generator is exhausted before generating
<var>k</var> items. By default, the generator returned by <code>gtake</code>
terminates as the source ends, but if you provide the <var>padding</var> argument,
then the returned generator will yield <var>k</var> items, using the
<var>padding</var> value to fill the rest.
</p>
</dd></dl>
<dl>
<dt><a name="index-gtake_002dwhile"></a>'<code>gtake-while</code><i> pred gen</i></dt>
<dt><a name="index-gdrop_002dwhile"></a>'<code>gdrop-while</code><i> pred gen</i></dt>
<dd><p>The generator version of <code>take-while</code> and <code>drop-while</code>. The generator returned
from <code>gtake-while</code> yields items from the source generator
as long as <var>pred</var> returns true for each. The generator returned
from <code>gdrop-while</code> first reads items from the source generator
while <var>pred</var> returns true for them, then starts yielding items.
</p></dd></dl>
<dl>
<dt><a name="index-gslices"></a>'<code>gslices</code><i> gen k [ padding ])</i></dt>
<dd><p>
This returns a generator, that yields a list of <var>k</var> items from
the input generator <var>gen</var> at a time.
</p>
<table><tr><td> </td><td><pre class="example">(generator->list (gslices (giota 7) 3))
⇒ ((0 1 2) (3 4 5) (6))
</pre></td></tr></table>
<p>The <var>padding</var> argument controls how the end
of input is handled, just like <code>gtake</code>. When <var>padding</var> is
not provided, the last item from the output generator may not
have <var>k</var> items if the input is too short to fill them, as shown
in the above example. If <var>padding</var> is present and the input is
too short to complete <var>k</var> items, <var>padding</var> is used
to fill the rest.
</p>
<table><tr><td> </td><td><pre class="example">(generator->list (gslices (giota 6) 3 #t 'x))
⇒ ((0 1 2) (3 4 5))
(generator->list (gslices (giota 7) 3 #t 'x))
⇒ ((0 1 2) (3 4 5) (6 x x))
</pre></td></tr></table>
</dd></dl>
<h3 class="subsection">Folding generated values</h3>
<p>These procedures consume all the values of the generator passed to them. They
have names beginning with <code>generator-</code>.
</p>
<dl>
<dt><a name="index-generator_002d_003elist"></a>'<code>generator->list</code><i> generator :optional k</i></dt>
<dd><p>Reads items from <var>generator</var> and returns a list of them.
By default, this reads until the generator is exhausted. If
an optional argument <var>k</var> is given, it must be a nonnegative
integer, and the list ends either <var>k</var> items are read,
or the generator is exhausted.
</p>
<p>Be careful not to pass an infinite generator to this without
specifying <var>k</var>, or this procedure will not return.
</p></dd></dl>
<dl>
<dt><a name="index-generator_002dfold"></a>'<code>generator-fold</code><i> proc seed gen gen2 …</i></dt>
<dd><p>Works like <code>fold</code> on the generated values by generator
procedures <var>gen</var> <var>gen2</var> ….
</p>
<p>When one generator is given, for each value <var>v</var> generated by <var>gen</var>,
<var>proc</var> is called as <code>(<var>proc</var> <var>v</var> <var>r</var>)</code>, where
<var>r</var> is the current accumulated result; the initial value of the
accumulated result is <var>seed</var>,
and the return value from <var>proc</var> becomes the next accumulated result.
When <var>gen</var> returns an eof object, the accumulated result at that time is returned
from <code>generator-fold</code>.
</p>
<p>When more than one generator is given, <var>proc</var> is
called as <code>(<var>proc</var> <var>v1</var> <var>v2</var> … <var>r</var>)</code>,
where <var>v1</var>, <var>v2</var> … are the values yielded from
<var>gen</var>, <var>gen2</var>, …, respectively, and <var>r</var> is
the current accumulated result. The iteration terminates when
any one of the generators returns eof.
</p>
<table><tr><td> </td><td><pre class="example">(with-input-from-string "a b c d e"
(cut generator-fold cons 'z read))
⇒ (e d c b a . z)
</pre></td></tr></table>
</dd></dl>
<dl>
<dt><a name="index-generator_002dfor_002deach"></a>'<code>generator-for-each</code><i> proc gen gen2 …</i></dt>
<dd><p>A generator version of <code>for-each</code>. Repeatedly applies <var>proc</var> on
the values yielded by <var>gen</var>, <var>gen2</var> … until any one of
the generators yields eof. The values returned from <var>proc</var> are discarded.
</p>
<p>This procedure consumes generated values using side effects.
</p></dd></dl>
<dl>
<dt><a name="index-generator_002dmap"></a>'<code>generator-collect</code><i> proc gen gen2 …</i></dt>
<dd><p>A generator analogue of <code>map</code>. Repeatedly applies <var>proc</var> on
the values yielded by <var>gen</var>, <var>gen2</var> … until any one of
the generators yields eof. The values returned from <var>proc</var>
are collected into a list and returned.
</p>
<table><tr><td> </td><td><pre class="example">(with-input-from-string "a b c d e"
(cut generator-map symbol->string read))
⇒ ("a" "b" "c" "d" "e")
</pre></td></tr></table>
<p>The same effects can be achieved by combining <code>generator->list</code>
and <code>gmap</code>.
</p>
<table><tr><td> </td><td><pre class="example">(generator->list (gmap proc gen gen2 …))
</pre></td></tr></table>
</dd></dl>
<dl>
<dt><a name="index-generator_002dfind"></a>'<code>generator-find</code><i> pred gen</i></dt>
<dd><p>Returns the first item from the generator <var>gen</var> that satisfies
the predicate <var>pred</var>.
</p>
<h3 class="subsection">Syntax</h3>
<dl>
<dt><a name="index-glet_002a"></a><code>glet*</code><i> (binding …) body body2 …</i></dt>
<dd><p>This captures a monadic pattern frequently appears in the generator
code. It is in a similar spirit of <code>and-let*</code>, but returns
as soon as the evaluating expression returns eof, instead of <code>#f</code>
as <code>and-let*</code> does.
</p>
<p>The <var>binding</var> part can be either <code>(var expr)</code> or <code>( expr )</code>.
The actual definition will explain this syntax clearly.
</p>
<table><tr><td> </td><td><pre class="example">(define-syntax glet*
(syntax-rules ()
[(_ () body body2 ...) (begin body body2 ...)]
[(_ ([var gen-expr] more-bindings ...) . body)
(let1 var gen-expr
(if (eof-object? var)
var
(glet* (more-bindings ...) . body)))]
[(_ ([ gen-expr ] more-bindings ...) . body)
(let1 var gen-expr
(if (eof-object? var)
var
(glet* (more-bindings ...) . body)))]))
</pre></td></tr></table>
</dd></dl>
<dl>
<dt><a name="index-do_002dgenerator"></a><code>do-generator</code><i> (var gexpr) body …</i></dt>
<dd><p>This is a generator version of <code>dolist</code> and <code>dotimes</code>.
</p>
<p><var>Gexpr</var> is an expression that yields a generator. It is
evaluated once. The resulting generator is called repeatedly
until it returns eof. Every time the generator is called,
<var>body</var> … are evaluated in the scope
where <var>var</var> is bound to the value yielded from the generator.
</p>
<p>Like <code>dolist</code> and <code>dotimes</code>, this macro exists for
side-effects. You can write the same thing with <code>for-each</code> families,
but sometimes this macro makes the imperative code more readable:
</p>
<table><tr><td> </td><td><pre class="example">(do-generator [line (file->line-generator "filename")]
;; do some side-effecting stuff with line
)
</pre></td></tr></table>
</dd></dl>
}}}
time
2014-09-25 11:24:48
version
2