`apply` or `funcall` for macros instead of functions

Question

In Lisp, a function's arguments are evaluated first before entering the function body. Macro arguments stay not evaluated.

But sometimes, one wants to inject code pieces stored in variables into a macro. This means evaluating the argument for the macro first, and then apply the macro-of-choice on this evaluated result.

One has to resort to

(eval `(macro ,arg))

To achieve this - but eval does not behave correctly in different environments.

The best thing would be, if one could do:

(apply macro (list arg))

or

(funcall macro arg)

But since the macro is not a function this doesn't work.

Is it possible to achieve something like this? - To circumvent that problem oder to make the macro available in the functions namespace?

Or am I missing some other ways to solve such problems?

I came to this question while trying to answer How to produce HTML from a list. but also in Generate TYPECASE with macro in common lisp, Evaluate arguments passed to a macro that generates functions in lisp, and How to convert a list to code/lambda in scheme?. But I always thought while answering them it would be good to have an apply or funcall-like function which can take macros.

Answer 1

It is not clear what you are trying to do, although it is almost certain that you are confused about something. In particular if you are calling eval inside macroexpansions then in almost all cases you are doing something both seriously wrong and seriously dangerous. I can't ever think of a case where I've wanted macros which expand to things including eval and I have written Lisp for a very very long time.

---

That being said, here is how you call the function associated with a macro, and why it is very seldom what you want to do.

Macros are simply functions whose domain and range is source code: they are compilers from a language to another language. It is perfectly possible to call the function associated with a macro, but what that function will return is source code, and what you will then need to do with that source code is evaluate it. If you want a function which deals with run-time data which is not source code, then you need that function, and you can't turn a macro into that function by some magic trick which seems to be what you want to do: that magic trick does not, and can not, exist.

So for instance if I have a macro

(defmacro with-x (&body forms)
  `(let ((x 1))
     ,@forms))

Then I can call its macro function on a bit of source code:

> (funcall (macro-function 'with-x)
                     '(with-x (print "foo")) nil)
(let ((x 1)) (print "foo"))

But the result of this is another bit of source code: I need to compile or evaluate it, and nothing I can do will get around this.

Indeed in (almost?) all cases this is just the same as macroexpand-1):

> (macroexpand-1 '(with-x (print "foo")))
(let ((x 1)) (print "foo"))
t

And you can probably write macroexpand-1 in terms of macro-function:

(defun macroexpand-1/equivalent (form &optional (env nil))
  (if (and (consp form)
           (symbolp (first form))
           (macro-function (first form)))
      (values (funcall (macro-function (first form)) form env)
              t)
    (values form nil)))

So, if the result of calling a macro is source code, what do you do with that source code to get a result which is not source code? Well, you must evaluate it. And then, well, since the evaluator expands macros for you anyway, you might as well just write something like

(defun evaluate-with-x (code)
  (funcall (compile nil `(lambda ()
                           (with-x ,@code)))))

So you didn't need to call the macro's function in any case. And this is not the magic trick which turns macros into functions dealing with data which is not source code: it is a terrible horror which is entirely made of exploding parts.

A concrete example: CL-WHO

It looks like this question might have its origins in this one and the underlying problem there is that that's not what CL-WHO does. In particular it is a confusion to think that something like CL-WHO is a tool for taking some kind of list and turning it into HTML. It's not: it's a tool for taking the source code of a language which is built on CL but includes a way of expressing HTML output mingled with CL code, and compiles it into CL code which will do the same thing. It happens to be the case that CL source code is expressed as lists & symbols, but CL-WHO isn't really about that: it's a compiler from, if you like, 'the CL-WHO language' to CL.

So, let's try the trick we tried above and see why it's a disaster:

(defun form->html/insane (form)
  (funcall 
   (compile nil `(lambda () 
                   (with-html-output-to-string (,(make-symbol "O"))
                     ,@form)))))

And you might, if you did not look at this too closely, think that this function does in fact do the magic trick:

> (form->html/insane '(:p ((:a :href "foo") "the foo")))
"<p></p><a href='foo'>the foo</a>"

But it doesn't. What happens if we call form->html/insane on this perfectly innocuous list:

(:p (uiop/run-program:run-program "rm -rf $HOME" :output t))

Hint: don't call form->html/insane on this list if you don't have very good backups.

CL-WHO is an implementation of a programming language which is a strict superset of CL: if you try to turn it into a function to turn lists into HTML you end up with something involving the same nuclear weapon you tinker with every time you call eval, except that nuclear weapon is hidden inside a locked cupboard where you can't see it. But it doesn't care about that: if you set it off it will still reduce everything within a few miles to radioactive ash and rubble.

So if you want a tool which will turn lists – lists which aren't source code – into HTML then write that tool. CL-WHO might have the guts of such a tool in its implemenentation, but you can't use it as it is.

---

And this is the same problem you face whenever you are trying to abuse macros this way: the result of calling a macro's function is Lisp source code, and to evaluate that source code you need eval or an equivalent of eval. And eval is not only not a terrible solution to almost any problem: it's also a nuclear weapon. There are, perhaps problems for which nuclear weapons are good solutions, but they are few and far between.