racket syntax pattern with multiple ...s

Question

I'm working on a syntax for racket using pipes similar to unix, something like this:

> ("FOO" > string-replace "O" "E" > string-append "x" > string-downcase)
"feex"

Here is a brute force solution, which supports procedures with 2, 1, and 0 (extra) arguments:

(require (prefix-in racket/base/ racket/base) syntax/parse/define)
(define-syntax-parser #%app
  [(_ data (~literal >) proc a b (~literal >) rest ...) #'(#%app (proc data a b) > rest ...)]
  [(_ data (~literal >) proc a (~literal >) rest ...) #'(#%app (proc data a) > rest ...)]
  [(_ data (~literal >) proc (~literal >) rest ...) #'(#%app (proc data) > rest ...)]
  [(_ data (~literal >) proc rest ...) #'(#%app proc data rest ...)]
  [(_ rest ...) #'(racket/base/#%app rest ...)])

The problem is finding the next pipe, because the syntax pattern does not allow multiple ... patterns. The macro needs to know where the next pipe is to close the form for the first one. Unless there is a way to build partial syntax objects with unmatched parens?

I can nest the ellipses, but then I have to use extra parens:

(define-syntax-parser #%app
  [(_ data (~literal >) (proc params ...) > rest ...) #'(#%app (proc data params ...) > rest ...)]
  [(_ data (~literal >) proc rest ...) #'(#%app proc data rest ...)]
  [(_ rest ...) #'(racket/base/#%app rest ...)])

> ("FOO" > (string-replace "O" "E") > (string-append "x") > string-downcase)
"feex"

Is there any way to do this without the extra parens?

I'm aware of clojure's threading macros, but they are difficult to follow if you have to nest them.

EDIT: the solution to this problem is now available as a racket package and on github

Answer 1

You can use the ~seq pattern combined with elipses to match without parens. For example:

(define-syntax-parser split
  [(_ (~seq a b) ...)
   #'(list a ... b ...))

Will require that split is given an even number of arguments, which will then be re-arranged and built into a list:

(split 1 2 3 4 5 6)
; =>
(list 1 3 5 2 4 6)

Of course, be mindful ~seq isn't magic and has limited support for backtracking when parsing. But in principle you should be able to do something like:

(data (~seq (~literal >) proc args ...) ...)

Answer 2

@Leif Andersen's suggestion works best if the args pattern doesn't allow >, so adding a (~not (~literal >)) on the args might help:

(_ data (~seq (~literal >) proc (~and args (~not (~literal >))) ...) ...)

Answer 3

Combining the answers of @Lief Andersen and @Alex Knauth I came up with this solution. As far as I can tell, there is no way to avoid recursion, but maybe there is some magic in syntax/parse I don't know about.

(require (prefix-in base/ racket/base) syntax/parse/define)
(define-syntax-parser #%app
  [(_ data (~seq (~literal >) proc (~and args (~not (~literal >))) ...)) #'(base/#%app proc data args ...)]
  [(_ data (~seq (~literal >) proc (~and args (~not (~literal >))) ...) rest ...) #'(#%app (proc data args ...) rest ...)]
  [(_ rest ...) #'(base/#%app rest ...)])

example usage (note the first > is the repl prompt, not code):

 > ("FOO" > string-downcase > string-replace "o" "e" > string-append "abc")
 "feeabc"

It is quite simple to add support for piping the data to the last argument of the procedure too:

(require (prefix-in base/ racket/base) syntax/parse/define)
(define-syntax-parser #%app
  [(_ data (~seq (~literal >) proc (~and args (~not (~literal >)) (~not (~literal >>))) ...)) #'(base/#%app proc data args ...)]
  [(_ data (~seq (~literal >>) proc (~and args (~not (~literal >)) (~not (~literal >>))) ...)) #'(base/#%app proc args ... data)]
  [(_ data (~seq (~literal >) proc (~and args (~not (~literal >)) (~not (~literal >>))) ...) rest ...) #'(#%app (proc data args ...) rest ...)]
  [(_ data (~seq (~literal >>) proc (~and args (~not (~literal >)) (~not (~literal >>))) ...) rest ...) #'(#%app (proc args ... data) rest ...)]
  [(_ rest ...) #'(base/#%app rest ...)])

example usage:

> ('(1 2 3) > first > * 3 >> list-ref '(a b c d e f))
'd

I'd be interesting in comments on how this macro affects performance. Maybe it would be better to do the transformation in code rather than patterns.