I'd like to make some closures to speed up operations that are specified at run time, as it is done for regex compilation. So let's take a *very* simple exmaple:
So that's ok. Now let's suppose we would like the invert the test depending on a predicate value:
CL-USER> (defun foo (value predicate) #'(lambda (x) (if predicate (eq x value) (not (eq x value))))) FOO
Or, in order to avoid a test at runtime:
CL-USER> (defun foo (value predicate) (if predicate #'(lambda (x) (eq x value)) #'(lambda (x) (not (eq x value))))) FOO
It looks like a macro would allow to avoid code duplication here
CL-USER> (defmacro maybe (predicate statement) (if predicate `(,@statement) `(not ,statement))) MAYBE CL-USER> (macroexpand-1 '(maybe T (eq x value))) (EQ X VALUE) T CL-USER> (macroexpand-1 '(maybe NIL (eq x value))) (NOT (EQ X VALUE)) T
Now redefine foo: CL-USER> (defun foo (value predicate) #'(lambda (x) (maybe predicate (eq x value)))) ;[...] ; caught STYLE-WARNING: ; The variable PREDICATE is defined but never used. ;[...] STYLE-WARNING: redefining FOO in DEFUN FOO
Calling the function shows that the variable PREDICATE is indeed not taken into account. Now (finally) the questions:
1. Why ? 2. How to use macros for generating the lambdas ?
> I'd like to make some closures to speed up operations that are > specified at run time, as it is done for regex compilation. So let's > take a *very* simple exmaple:
On 2 Nov., 17:06, Nicolas Edel <nicolas.e...@gmail.com> wrote:
> Dear group
> I'd like to make some closures to speed up operations that are > specified at run time, as it is done for regex compilation. So let's > take a *very* simple exmaple:
(if predicate ...) - you are checking a macro expansion time for the value of predicate.
Not a good idea.
You think you now a macro expansion time the value of predicate, but you don't. All you know is the code, but not its value.
Try: `(if ,predicate ... )
> MAYBE > CL-USER> (macroexpand-1 '(maybe T (eq x value))) > (EQ X VALUE) > T > CL-USER> (macroexpand-1 '(maybe NIL (eq x value))) > (NOT (EQ X VALUE)) > T
> Now redefine foo: > CL-USER> (defun foo (value predicate) > #'(lambda (x) > (maybe predicate (eq x value))))
The macro maybe is used.
It has two parameters: predicate and statement.
The arguments are predicate and (eq x value)
predicate = predicate
statement = (eq x value)
Remember above
(if predicate ... ) ?
predicate evaluates to the symbol predicate, which is always true.
On Nov 2, 7:24 pm, "Captain Obvious" <udode...@users.sourceforge.net> wrote:
> NE> 1. Why ?
> Because macros work at macroexpansion time, NOT > runtime.
> NE> 2. How to use macros for generating the lambdas ?
> You can use macro to generate a code which generates lambda, > for example.
Ok, so I use macro to generate code and then invoke the compiler at runtime. What is best practice for this: eval or compile ? Do they have similar meaning in this context ?
Nicolas Edel wrote: > On Nov 2, 7:24 pm, "Captain Obvious" <udode...@users.sourceforge.net> > wrote: >> NE> 1. Why ?
>> Because macros work at macroexpansion time, NOT >> runtime.
>> NE> 2. How to use macros for generating the lambdas ?
>> You can use macro to generate a code which generates lambda, >> for example.
> Ok, so I use macro to generate code and then invoke the compiler at > runtime. > What is best practice for this: eval or compile ? Do they have similar > meaning in this context ?
The semantics are the same. Which one is better depends on context. If you use these functions only one or two times, or so, it's not worthwhile to compile them, because compilation creates a considerable overhead by itself. If you use them more often, then compilation is worthwhile (although I recently heard that in, say, ECL or GCL, compilation is almost always prohibitive).
You can also say (coerce '(lambda ...) 'function), and let the CL implementation itself decide.
> > Ok, so I use macro to generate code and then invoke the compiler at > > runtime. > > What is best practice for this: eval or compile ? Do they have similar > > meaning in this context ?
> The semantics are the same. Which one is better depends on context. If > you use these functions only one or two times, or so, it's not > worthwhile to compile them, because compilation creates a considerable > overhead by itself. If you use them more often, then compilation is > worthwhile (although I recently heard that in, say, ECL or GCL, > compilation is almost always prohibitive).
At that time, I don't know if they will be used often. But if they are to be used in a context such as packet filtering, there will be a slow startup time but this will be runtime efficient. I wanted to learn how to make this now to avoid problems at experimentation time.
> You can also say (coerce '(lambda ...) 'function), and let the CL > implementation itself decide.
Nice, indeed. But before I let the implementation decide by itself, I would like to know if there are fundamental differences when calling eval and compile in this context.
> On Nov 3, 9:31 am, Pascal Costanza <p...@p-cos.net> wrote: > > > Ok, so I use macro to generate code and then invoke the compiler at > > > runtime. > > > What is best practice for this: eval or compile ? Do they have similar > > > meaning in this context ?
> > The semantics are the same. Which one is better depends on context. If > > you use these functions only one or two times, or so, it's not > > worthwhile to compile them, because compilation creates a considerable > > overhead by itself. If you use them more often, then compilation is > > worthwhile (although I recently heard that in, say, ECL or GCL, > > compilation is almost always prohibitive).
> At that time, I don't know if they will be used often. But if they are > to be used in a context such as packet filtering, there will be a slow > startup time but this will be runtime efficient. I wanted to learn how > to make this now to avoid problems at experimentation time.
> > You can also say (coerce '(lambda ...) 'function), and let the CL > > implementation itself decide.
> Nice, indeed. But before I let the implementation decide by itself, I > would like to know if there are fundamental differences when calling > eval and compile in this context.
No semantic differences, just performance differences. Compile will create a compiled function, eval will create an interpreted function in some implementations.
-- Barry Margolin, bar...@alum.mit.edu Arlington, MA *** PLEASE post questions in newsgroups, not directly to me *** *** PLEASE don't copy me on replies, I'll read them in the group ***
Barry Margolin <bar...@alum.mit.edu> writes: > Nicolas Edel <nicolas.e...@gmail.com> wrote:
> > Nice, indeed. But before I let the implementation decide by itself, I > > would like to know if there are fundamental differences when calling > > eval and compile in this context.
> No semantic differences, just performance differences. Compile will > create a compiled function, eval will create an interpreted function in > some implementations.
COMPILED-FUNCTION-P will return T on the first kind, but NIL on the other kind! That's as big semantical change as you can get. ;-)
??>> You can use macro to generate a code which generates lambda, ??>> for example.
NE> Ok, so I use macro to generate code and then invoke the compiler at NE> runtime.
No! You use macro to transform your code at compile time. Then code you've generated does anything it wants in runtime.
NE> What is best practice for this: eval or compile ?
In case you _really_ need it, best practice is compile. But you do not need it in this case. A rule of thumb: if you think you need eval or compile, it is higly lickely you're doing it wrong.
Basically, the only case when you need eval is when code is not known at compile-time. That is, user enters it from keyboard, or you recieve it from network, or generate through GA or something. In other cases, when you just need to fill in some parameters, you can do that other way.
NE> Do they have similar meaning in this context ?
Yep they do, almost. The only difference is that eval might or might not compile functions, while compile almost certainly does. compile is a bit tidier IMHO because it accepts only lambda rather than arbitrary piece of code. There is also a third way to do it -- (coerce ... 'function), it is pretty good too, if you just need a function, not necessarily compiled.
That's how you can generate lambdas at runtime, but there is no reason to do this in this case.
Xach have shown how you can do this with lambda and without any code duplication. That was really the best way. It is actually a good practice to avoid using macros when you can just use higher-order functions.
If you absolutely want to use macros, you can do this like this (using IF at runtime):
(defun foo (value predicate) (maybe-lambda predicate (x) (eql x value)))
No runtime code generation, but maybe have lost its generality.
Or you can write a more complex macro which is more general. Your `maybe` macro just maybe inserts NOT in the beginning of the statement. My with-maybe macro can insert NOT into any place within statement user wants it to.
Macro generates a code which has IF switching between two branches. Branches are made different by different MAYBE local macros defined via MACROLET -- in first branch MAYBE is no-op, so it is vanilla statement. In second branch MAYBE adds NOT to the statement.
Unfortunately there is no tool one can use to get fully macroexpanded code (at least I don't know any), but once you'll replace macrolet:ed maybe:s, you'll get exactly function you had wrote yourself:
(defun foo (value predicate) (if predicate (lambda (x) (eq x value)) (lambda (x) (not (eq x value)))))
>>> Nice, indeed. But before I let the implementation decide by itself, I >>> would like to know if there are fundamental differences when calling >>> eval and compile in this context. >> No semantic differences, just performance differences. Compile will >> create a compiled function, eval will create an interpreted function in >> some implementations.
> COMPILED-FUNCTION-P will return T on the first kind, but NIL on the > other kind!
Not quite: It may actually return T in some implementations for the second kind. A CL implementation is required to support at least minimal compilation (as per 3.2.2.2 in the HyperSpec), but is not required to support a 'pure' interpreter (as per 3.1 in the HyperSpec) and can actually always perform (at least minimal) compilation.
> That's as big semantical change as you can get. ;-)
Nicolas Edel wrote: > On Nov 3, 9:31 am, Pascal Costanza <p...@p-cos.net> wrote: >>> Ok, so I use macro to generate code and then invoke the compiler at >>> runtime. >>> What is best practice for this: eval or compile ? Do they have similar >>> meaning in this context ? >>> (defun foo (predicate value) >>> (eval `(function (lambda (x) >>> (maybe ,predicate (eql x ,value)))))) >>> or >>> (defun foo (predicate value) >>> (compile nil `(lambda (x) >>> (maybe ,predicate (eql x ,value))))) >> The semantics are the same. Which one is better depends on context. If >> you use these functions only one or two times, or so, it's not >> worthwhile to compile them, because compilation creates a considerable >> overhead by itself. If you use them more often, then compilation is >> worthwhile (although I recently heard that in, say, ECL or GCL, >> compilation is almost always prohibitive).
> At that time, I don't know if they will be used often. But if they are > to be used in a context such as packet filtering, there will be a slow > startup time but this will be runtime efficient. I wanted to learn how > to make this now to avoid problems at experimentation time.
If you want your code to be very sophisticated, you can actually also implement your own version that uses interpreted code by default, counts how often a particular runtime-generated function is called, and after a certain threshold, compiles into a more efficient version (under the assumption that it will continue to be called even more often).
> Nicolas Edel wrote: > > On Nov 3, 9:31 am, Pascal Costanza <p...@p-cos.net> wrote: > >>> Ok, so I use macro to generate code and then invoke the compiler at > >>> runtime. > >>> What is best practice for this: eval or compile ? Do they have similar > >>> meaning in this context ? > >>> (defun foo (predicate value) > >>> (eval `(function (lambda (x) > >>> (maybe ,predicate (eql x ,value)))))) > >>> or > >>> (defun foo (predicate value) > >>> (compile nil `(lambda (x) > >>> (maybe ,predicate (eql x ,value))))) > >> The semantics are the same. Which one is better depends on context. If > >> you use these functions only one or two times, or so, it's not > >> worthwhile to compile them, because compilation creates a considerable > >> overhead by itself. If you use them more often, then compilation is > >> worthwhile (although I recently heard that in, say, ECL or GCL, > >> compilation is almost always prohibitive).
> > At that time, I don't know if they will be used often. But if they are > > to be used in a context such as packet filtering, there will be a slow > > startup time but this will be runtime efficient. I wanted to learn how > > to make this now to avoid problems at experimentation time.
> If you want your code to be very sophisticated, you can actually also > implement your own version that uses interpreted code by default, counts > how often a particular runtime-generated function is called, and after a > certain threshold, compiles into a more efficient version (under the > assumption that it will continue to be called even more often).
> Nicolas Edel wrote: > > On Nov 2, 7:24 pm, "Captain Obvious" <udode...@users.sourceforge.net> > > wrote: > >> NE> 1. Why ?
> >> Because macros work at macroexpansion time, NOT > >> runtime.
> >> NE> 2. How to use macros for generating the lambdas ?
> >> You can use macro to generate a code which generates lambda, > >> for example.
> > Ok, so I use macro to generate code and then invoke the compiler at > > runtime. > > What is best practice for this: eval or compile ? Do they have similar > > meaning in this context ?
> The semantics are the same. Which one is better depends on context. If > you use these functions only one or two times, or so, it's not > worthwhile to compile them, because compilation creates a considerable > overhead by itself. If you use them more often, then compilation is > worthwhile (although I recently heard that in, say,ECLor GCL, > compilation is almost always prohibitive).
It is not "almost always prohibitive". The cost is basically executing a C compiler. And that is worthwhile if you need the function often and if the function it is low level enough that it profits from compilation.
What is prohibitive is to compile using C a function that you are going to use only once, and do it for every use.
The alternative then is to use bytcompiled functions, which do save some time, but are not as efficient as natively compiled ones. You can always do this, without much hassle. One cheap way is to coerce a lambda form to a function type.
Juanjo wrote: > On Nov 3, 9:31 am, Pascal Costanza <p...@p-cos.net> wrote: >> Nicolas Edel wrote: >>> On Nov 2, 7:24 pm, "Captain Obvious" <udode...@users.sourceforge.net> >>> wrote: >>>> NE> 1. Why ? >>>> Because macros work at macroexpansion time, NOT >>>> runtime. >>>> NE> 2. How to use macros for generating the lambdas ? >>>> You can use macro to generate a code which generates lambda, >>>> for example. >>> Ok, so I use macro to generate code and then invoke the compiler at >>> runtime. >>> What is best practice for this: eval or compile ? Do they have similar >>> meaning in this context ? >>> (defun foo (predicate value) >>> (eval `(function (lambda (x) >>> (maybe ,predicate (eql x ,value)))))) >>> or >>> (defun foo (predicate value) >>> (compile nil `(lambda (x) >>> (maybe ,predicate (eql x ,value))))) >> The semantics are the same. Which one is better depends on context. If >> you use these functions only one or two times, or so, it's not >> worthwhile to compile them, because compilation creates a considerable >> overhead by itself. If you use them more often, then compilation is >> worthwhile (although I recently heard that in, say,ECLor GCL, >> compilation is almost always prohibitive).
> It is not "almost always prohibitive". The cost is basically executing > a C compiler. And that is worthwhile if you need the function often > and if the function it is low level enough that it profits from > compilation.
> What is prohibitive is to compile using C a function that you are > going to use only once, and do it for every use.
> The alternative then is to use bytcompiled functions, which do save > some time, but are not as efficient as natively compiled ones. You can > always do this, without much hassle. One cheap way is to coerce a > lambda form to a function type.
> What's the advantage of using the MOP over a straight-up closure in > this instance?
Good question. Maybe there is none. Originally I thought I had to store the body in a slot of the smart-function instance, but that turned out not to be necessary. So indeed, maybe you can replace this by a plain closure.
>> What's the advantage of using the MOP over a straight-up closure in >> this instance?
> Good question. Maybe there is none. Originally I thought I had to store > the body in a slot of the smart-function instance, but that turned out > not to be necessary. So indeed, maybe you can replace this by a plain > closure.
But if you're doing it via macro, you can use certain number of parameters directly:
(lambda (a b c) (let ((fn ....)) (funcall fn a b c)))
Then there is no consing. There might be some overhead from using closure's variable and funcall, but I guess funcallable-object's funcall is not totally overheadless either. So probably funcallable-object offers only a minuscule advantage in this case.
It's a poor Lisp compiler which conses a list for code which only passes the &rest argument down to another function with apply.
Pascal's code went right over your head there. The advantage of the MOP solution has nothing to do with this. And it has &rest/apply processing buried in it.
What Pascal's code does is allow the user's function to be called several times without being compiled. It counts how many calls there are. When the calls exceed a threshold, then the object compiles the function.
When the function is compiled, the object is able to replace its own closure with that compiled function. So subsequent calls go directly to that function, bypassing the threshold check.
You can't do that with a lambda closure, because a lambda closure cannot replace future calls to itself with a different closure.
> If you want your code to be very sophisticated, you can actually also > implement your own version that uses interpreted code by default, counts > how often a particular runtime-generated function is called, and after a > certain threshold, compiles into a more efficient version (under the > assumption that it will continue to be called even more often).
Uh, you can do this with the existing standard funcallable objects. Namely, instances of this highly curious SYMBOL class, which have a standard slot given by the accessor SYMBOL-FUNCTION. :)
;; tested modifications to untested code ;; fixed unbalanced parentheses in declare ;; extra levels of unquote added to backquote ;; class replaced by symbol
Kaz Kylheku wrote: > On 2009-11-03, Pascal Costanza <p...@p-cos.net> wrote: >> If you want your code to be very sophisticated, you can actually also >> implement your own version that uses interpreted code by default, counts >> how often a particular runtime-generated function is called, and after a >> certain threshold, compiles into a more efficient version (under the >> assumption that it will continue to be called even more often).
> Uh, you can do this with the existing standard funcallable objects. Namely, > instances of this highly curious SYMBOL class, which have a standard > slot given by the accessor SYMBOL-FUNCTION. :)
In principle yes, but unfortunately, there is a border case that doesn't quite work with symbols as functions:
debugger invoked on a TYPE-ERROR: The value FOO is not of type FUNCTION.
Some Common Lisp implementations accept this, some don't. Those that don't accept this are right not to do so, according to the HyperSpec. (I think this is an unnecessary restriction, symbols should be accepted as symbol-functions, but that's maybe just me... ;)
??>> Might have better performance. If you use &rest and then apply ??>> I guess most compilers will make a list, that is, you have more ??>> consing. ??>> ??>> (lambda (&rest args) ??>> (let ((fn ....)) ??>> (apply fn args)))
KK> It's a poor Lisp compiler which conses a list for code which only KK> passes the &rest argument down to another function with apply.
Can you please show an example of a compiler which optimizes it?
KK> You can't do that with a lambda closure, because a lambda closure KK> cannot replace future calls to itself with a different closure.
That's why we have not one closure, but two closures, with first calling the second one -- then we can replace second closure with a compiled version when we would want to. There is overhead of indirection, but arguably it is very low.
|
