![[Home]](http://www.bayleshanks.com/cartoonbayle.png)
http://stackoverflow.com/questions/tagged/homoiconicity
---
" You might already have a language in mind that you want to use. Technically speaking, mal can be implemented in any sufficiently complete programming language (i.e. Turing complete), however, there are a few language features that can make the task MUCH easier. Here are some of them in rough order of importance:
A sequential compound data structure (e.g. arrays, lists, vectors, etc)
An associative compound data structure (e.g. a dictionary, hash-map, associative array, etc)
Function references (first class functions, function pointers, etc)
Real exception handling (try/catch, raise, throw, etc)
Variable argument functions (variadic, var args, splats, apply, etc)
Function closures
PCRE regular expressionsIn addition, the following will make your task especially easy:
Dynamic typing / boxed types (specifically, the ability to store different data types in the sequential and associative structures and the language keeps track of the type for you)
Compound data types support arbitrary runtime "hidden" data (metadata, metatables, dynamic fields attributes)Here are some examples of languages that have all of the above features: JavaScript?, Ruby, Python, Lua, R, Clojure. " -- https://github.com/kanaka/mal/blob/master/process/guide.md
https://news.ycombinator.com/item?id=9123065 'codygman' compares various language implementations of mal using 'cloc' to see which implementations were shortest.
the shortest were:
CoffeeScript?, Clojure, OCaml, Ruby, Python
the full table was (omitting support files) was:
Language files blank comment code
-------------------------------------------------------------------------------
Rust 17 294 155 3919
C 18 446 383 3356
C# 19 472 235 3314
Visual Basic 17 322 131 2945
Java 17 322 134 2872
Go 17 299 147 2636
Bourne Shell 16 304 241 2308
Perl 19 263 165 2167
Haskell 17 328 99 1951
MATLAB 25 176 114 1880
Javascript 23 263 145 1873
PHP 17 242 118 1791
Scala 16 219 113 1731
Lua 18 227 87 1715
Python 18 218 143 1393
Ruby 17 165 87 1219
OCaml 15 105 98 1154
Clojure 17 247 117 1011
CoffeeScript 17 193 126 989(Forth is missing)
the thread https://news.ycombinator.com/item?id=9121842 discusses this and gives reasons why some of the longer implementations could have been shorter.
---
http://spinroot.com/gerard/pdf/P10.pdf rule #5 has an interesting suggestion in the context of syntax for writing assertions, where 'e' is an assertion condition (a boolean expression): "The syntax #e turns the assertion condition e into a string that is printed as part of the error message."
eg you should be able to pass around 'expressions' (which are raw ASTs, which can contain unbound variables, not necessarily written as anonymous functions with an explicit varspec), and you should be able to serialize these into strings
---
one Java metaprogramming interface in Java core:
http://docs.oracle.com/javase/7/docs/api/javax/annotation/processing/Processor.html
---
this guy claims macros are not a readability problem:
orthecreedence 2 hours ago
After programming in CL for several years, I can honestly say that when encountering new syntax that people have introduced in an app, it's no harder to follow along with it by reading the macro definition than encountering an unknown function. And when in doubt, you can just macroexpand the syntax and see what it's doing under the hood.
reply
--
kevin_thibedeau 8 hours ago
I recently needed to make a cross platform assembler for a niche 8-bit micro that only has DOS tooling. After cloning it in a small bit of Python I was looking for a simple way to extend it with macros. It turns out that m4 works really well as an asm macro language and I now have advanced facilities like a code generating expression evaluator and high level control flow statements without any significant change to the core assembler. It blows the doors off what contemporary assemblers (nasm, masm, tasm) can do and hearkens back to the high point of high level assemblers when they peaked in the 70s.
reply
---
make certain vars 'fair game' for (anaphoric/non-hygenic) macros? eg x, y, z, x1, x2, etc? or mb something more universal, such as 'anything with a _ suffix' or 'anything ending in X'?
---
extend_syntax eg
http://www.cs.rit.edu/~rpj/courses/plc/lab4/lab4df.html?lab47.html#Q12?lab4ans.html#A12
---
http://scienceblogs.com/goodmath/2007/12/17/macros-why-theyre-evil/ has a good example of scheme define-syntax/syntax-rules, it looks quite good:
(define-syntax replace (syntax-rules (initially with until just-before) ((replace initially <value> with <newvalue> until <done>) (let loop (( <value>)) (if <done> (loop <newvalue>)))) ((replace initially <value> with <newvalue> until just-before <done>) (let loop ((old #f) ( <value>)) (if <done> old (loop <newvalue>)))) ((replace <var1> <var2> initially <value1> <value2> with <newvalue1> <newvalue2> until <done>) (let loop ((<var1> <value1>) (<var2> <value2>)) (if <done> (list <var1> <var2>) (loop <newvalue1> <newvalue2>))))))
---
bentcorner 4 hours ago
I find it frustrating that languages features work actively against you when you're trying to understand something.
Wide inheritance and macro usage are probably the worst. Good naming can aid understanding, but basic things like searchability are harmed by this.
Of those two, macros are the most trouble. You can't take anything for granted, and must look at every expression with an eye for detail. Taking notes becomes essential.
reply
---
http://andymeneely.github.io/squib/
a Ruby DSL for prototyping card and board games
discussion: https://news.ycombinator.com/item?id=9796708
(a quick skim saw nothing in the HN discussion relating to programming language/DSL design)
JSR 308 type annotations:
"What are type annotations?
The Java 6 annotation syntax is useful but limited. The Type Annotations syntax permits annotations to be written in more places, such as generic type arguments: List<@NonNull? Object>. jsr Programmers can use type annotations to write more informative types, and then tools such as type-checkers can detect and prevent more errors. " -- http://types.cs.washington.edu/jsr308/
http://types.cs.washington.edu/jsr308/specification/java-annotation-design.html
---
"
Annotations
Most projects eventually need to interact with the programmer. You’ll eventually wish for annotations: some way to convey extra information from the program to your LLVM pass. There are several ways to build up annotation systems:
The practical and hacky way is to use magic functions. Declare some empty functions with special, probably-unique names in a header file. Include that file in your source and call those do-nothing functions. Then, in your pass, look for CallInst instructions that invoke your functions and use them to trigger your magic. For example, you might use calls like __enable_instrumentation() and __disable_instrumentation() to let the program confine your code-munging to specific regions.
If you need to let programmers add markers to function or variable declarations, Clang’s __attribute__((annotate("foo"))) syntax will emit metadata with an arbitrary string that you can process in your pass. Brandon Holt again has some background on this technique. If you need to mark expressions instead of declarations, the undocumented and sadly limited __builtin_annotation(e, "foo") intrinsic might work.
You can jump in full dingle and modify Clang itself to interpret your new syntax. I don’t recommend this.
If you need to annotate types—and I believe people often do, even if they don’t realize it—I’m developing a system called Quala. It patches Clang to support custom type qualifiers and pluggable type systems, à la JSR-308 for Java. Let me know if you’re interested in collaborating on this project!I hope to expand on some of these techniques in future posts. " -- http://adriansampson.net/blog/llvm.html
---
mb read:
---
(at least) two kinds of 'macros':
'reader macros' applied to string text before (or during) the transformation to AST
ordinary lisp-style macros, which are transforms of AST->AST
---
mike_hearn 23 days ago
| parent | on: Ask HN: Has anyone here programmed in Kotlin? What... |
Do you actually know Kotlin? It has many features for DSL support, including operator overloading (though not the ability to invent random new operators fortunately), infix notation, a form of exhaustive pattern matching, extension functions, annotation processing, inlineable anonymous extension functions which sounds bizarre but actually is key to defining many DSLs in the language, and so on. It's a different approach to Scala but to say there's no way to do DSLs isn't right.
As an example, check out funKtionale. It's a library that adds various functional features like function currying and partial application to Kotlin, entirely by using its DSL features.
pron 22 days ago
Or our Kotlin actor API: http://blog.paralleluniverse.co/2015/06/04/quasar-kotlin/
---
compiler warning (or scripting level requirement) if anything is REdefined with a macro (esp. if it's a language-level or stdlib-level thing)
---
so {} are like "quote". is "!" like "unquote" or is it "pass on the sideeffects"? in some sense i guess sideeffects are themselves "quoted" in a functional reactive stream. preefix vs postfix "!" to apply to the thing itslf, or to its first return value? what about impure reads (not sideeffectful)? what about sideeffecttful reads, the rest of the stream cannot be produced until these are executed against some environment? hmm.. "!" as meaning "excute against the current environment"; mb "f!e" for "exec f against environ e"; exec means not just reading from the environ, but also writing to it.. environ can "catch" these writes (and reads!) if it wants, in order to produce a functional reeactive stream instead ! =s like eval, not anttiquote; and this is not quite like quote/eval b/c blocks are opaque, they are not ASTs (and not source code text strings) that are preserved until runtime (or th end of this 'stage' at least, in the case of g compiletime macros) for us in metaprogramming
3 levels/stages: source cod text string, AST, opaque block/fn text --(parse (lex and parse))--> AST --compile--->block--eval/exec-->result and side effects (mb 'eval' to a stream and 'exec' to adtually do it? wher 'eval' is just 'exec' to an environ that catches eeverything?)
---
jerf 14 hours ago
... Haskell's great support for domain-specific languages. This is both in the sense of languages that are really "just" Haskell APIs but work in such an different manner that they are quite different, and in the sense of writing a full compiler [1] (or any subset of a compiler) for a new language.
For an example of the former, one of my favorite demonstrations is the "probability monad", in which one ends up with a sort of sublanguage inside of Haskell that is probability aware: http://www.randomhacks.net/2007/02/22/bayes-rule-and-drug-te... [2] If you want to start to really grasp the power of Haskell, you could do worse than meditate on how you'd implement that in your $FAVORITE_LANGUAGE... and then, most likely, burst into tears when someone shows you just how much of Haskell still works, unchanged, even in that environment (e.g., everything in Control.Monad), rather than having to be implemented in a parallel type hierarchy (or, put another way, producing a new "color" of functions [3]).
While I doubt they use something exactly like that, it's not hard to see how that sort of thing could make it fairly easy to write spam classification code with a lot of the basic plumbing abstracted away entirely.
There's a lot of examples of the latter, but I'm not sure of a really solid open source example to point at. I know it's a common move in industry, though.
For bonus points, these things compose together just fine; you could easily take the probability monad and build a new language on top of it.
[1]: And I do mean real compiler, by anybody's definition, all the way down to assembler if you like. Haskell's a pretty good base language for that sort thing.
[2]: Or see http://blog.plover.com/prog/haskell/probmonad-refs.html http://blog.plover.com/prog/haskell/probmonad-refs.html
[3]: http://journal.stuffwithstuff.com/2015/02/01/what-color-is-y... http://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-function/
---
cody http://docs.quantifiedcode.com/patterns/index.html
---
great example of a DSL that we'd want to enable:
http://daveyarwood.github.io/alda/2015/09/05/alda-a-manifesto-and-gentle-introduction/
the discussion mentions other more complex music languages:
https://news.ycombinator.com/item?id=10177716
including
MML, Lilypond, ChucK?, Csound, Supercollider, Max/MSP, PD, Faust, Overtone (Supercollider + Clojure) and if you want less "note C" and more abstract: Music-Suite ( http://music-suite.github.io/docs/ref ) and Overtone ( http://overtone.github.io ), along with the Leipzig library, which adds functional programming abstractions for use with Overtone ( https://github.com/ctford/leipzig )
---
note: i looked in Rust to see what representation they express their borrow checker in. I think this sort of thing would be in the source code subdirectory a common standard for program evaluation is 95% confidence with a sampling error of ± 5%. In English that means that you believe that 95 percent of the time the results from your sample, would be off by no more than 5% as compared to the results you would have gotten if you had collected data from everyone. (librustc is the main compiler, and i think 'middle' is the stuff that is neither frontend (text -> AST) nor backend (-> target code)).
It appears to just be in ordinary Rust.
The most mentioned seem to be MML, Lilypond, Overtone, ChucK?, Csound.
---
'.' is like a macro but it actually changes the case of its second argument before evaluating it, eg employeeRecord.address == (employeeRecord ADDRESS). Do we allow users to write things like that? Seems like we should...
---
i'm on board with A rationale for semantically enhanced library languages by Bjarne Stroustrup: "..a dialect created by supersetting a language using a library and then subsetting the result...when augmented by a library, a general-purpose language can be about as expressive as a special-purpose language and by subsetting that extended language, a tool can provide about as good semantic guarantees. Such guarantees can be used to provide better code, better representations, and more sophisticated transformations than would be possible for the full base language."
i'd like to build some of that into oot:
This fits in with the ideas of/that:
---
note: i looked in Rust to see what representation they express their borrow checker in. I think this sort of thing would be in the source code subdirectory https://github.com/rust-lang/rust/tree/master/src/librustc/middle (librustc is the main compiler, and i think 'middle' is the stuff that is neither frontend (text -> AST) nor backend (-> target code)).
It appears to just be in ordinary Rust.
---
Stroustrup pushes us to consider more intrusive things that metaprogrammy library writers might want to do:
"Consider a sim- ple, common, and useful example:
A=k*B+C
... Essentially all languages can handle A=k*B+C when the vari- ables denote scalar values, such as integers and floating point numbers. For vectors and matrices, things get more diffi- cult for a general-purpose language (that doesn’t have built- in vector and matrix types) because people who write that kind of code expect performance that can be achieved only if we do not introduce temporary variables for k*B and k*B+C . We probably also need loop fusion (that is doing the ele- ment * , +, and = operations in a minimal number of loops). When the matrices and vectors are sparse or we want to take advantage of known properties of the vectors (e.g. B is upper-triangular), the library code needed to make such code work pushes modern general purpose language to their limit [17, 23] or beyond — most mainstream languages can’t effi- ciently handle that last example. Move further and require the computation of A=k*B+C for large vectors and matrices to be computed in parallel on hundreds of processors. Now, even an advanced library requires the support of a non-trivial run-time support system [1]. We can go further still and take advantage of semantic properties of operations, such as “re- membering” that C was the result of an operation that leaves all its elements identical. Then, we can use much simpler add operation that doesn’t involve reading all the elements of C . For other examples, preceding the numerical calculation with a symbolic evaluation phase, say doing a symbolic differentiation, can lead to immense improvements in accuracy and performance. Here, we leave the domain where libraries have been considered useful. Reasoning like that and exam- ples like that (and many more realistic ones) have led to the creation of a host of special-purpose languages for various forms of scientific calculation." -- http://www.stroustrup.com/SELLrationale.pdf
the things he is talking about would seem to require not just macros but also:
this goes beyond simply having a macro for 'matrix multiply' (we need to match code patterns, not just expand explicitly marked points in code); then it goes beyond mere AST matching (we need to do static analysis to catch if all of the elements of C are identical, or if C is guaranteed to be diagonal); then it goes beyond adding another optimization pass (we need to alter the runtime to support parallelization); and i don't even know offhand what is needed for the symbolic evaluation phase. And, we want to do all of this 'within the language' rather than just providing hooks in the compiler implementation to add a call to some external separately compiled binary executable or shared library.
---
types of analysis:
static analysis:
runtime analysis:
---
---
Forth has an interesting convention to document what stack manipulation functions do (in the parentheses; stuff in parens is comment in Forth); eg:
dup ( a -- a a ) ?dup ( a -- a a
| 0 ) dup if dup then ; |
three notes:
---
dsls need to inject functions that (a) dont need a namespace prefix, (b) have access to 'hidden' semiglobals. eg metattrader's Bid, eg quantopian
---
using only Oot metaprograming, we should be able to add the following two features to the language:
---
we don't ever want anything in Oot (except compiler flags, and perhaps a few very special metaprogrammy compiler overrides) to globally affect a program. So no Haskell like 'only one instance of this typeclass anywhere', no C overrides, no 'open classes' that can be overridden from anywhere and that persist and globally change the way core classes work for everyone
however, we might want to allow all sorts of metaprogramming scoped to:
note that file-scoped C-like #defines would be okay, as this is a lexical scope, so these things would still be trouble: https://gist.github.com/aras-p/6224951
now, allowing arbitrary metaprogramming in any dynamic scope makes it hard for anything to be efficient, because the compiler can't really do any optimization before runtime except when it can prove that the constructs it is optimizing can't possibly lie within a dynamic scope that tries to metaprogram those constructs. So maybe don't do that, or maybe allow lexical scopes to disallow runtime metaprogramming (or maybe do that upon module load, so the compiler can make 2 versions of each module, one of them statically optimized, one of them open for runtime metaprogrammingy modification), or maybe just bite the bullet and allow it.
so, we want to make it possible for the compiler to:
(suggested by [1])
---
some various metaprogrammingish things in Ruby, which doesnt have macros:
http://stackoverflow.com/questions/17110117/macros-in-ruby
Q:
how to:
define ARGS 1,2 sum(ARGS) # returns 3
A's:
A = [1, 2] A.inject(:+) # => 3
args = [1, 2] sum(*args) # equivalent to sum( 1, 2 )
Q:
EDIT: More specifically my problem looks more like:
@button1 = FXButton.new(self, "Button 1",:opts => BUTTONPROPERTIES,:width => width, :height => height) @button2 = FXButton.new(self, "Button 2",:opts => BUTTONPROPERTIES,:width => width, :height => height) @button3 = FXButton.new(self, "Button 3",:opts => BUTTONPROPERTIES,:width => width, :height => height)
And ideally I'd want the code to look like:
@button1 = FXButton.new(self, "Button 1", ALLBUTTONPROPERTIES) @button2 = FXButton.new(self, "Button 2", ALLBUTTONPROPERTIES) @button3 = FXButton.new(self, "Button 3", ALLBUTTONPROPERTIES)
A:
ALLBUTTONPROPERTIES = ->{{opts: => BUTTONPROPERTIES, width: width, height: height}}
and within a context where the constants and variables BUTTONPROPERTIES, width, height are assigned some value, do this:
@button1 = FXButton.new(self, "Button 1", ALLBUTTONPROPERTIES.call) @button2 = FXButton.new(self, "Button 2", ALLBUTTONPROPERTIES.call) @button3 = FXButton.new(self, "Button 3", ALLBUTTONPROPERTIES.call)
---
they call this a 'macro' but i don't think it should be called that, still, is interesting:
" An implementation of attr_reader in pure Ruby
The attr_reader function should be familiar to everyone with Ruby programming experience. Implementing attr_reader is a perfect example of what are viewed as macros in Ruby, which are implemented using Ruby's metaprogramming capabilities.
But we're going to call it rm_attr_reader to avoid collision with Ruby's standard library implementation of attr_reader. (The rm_ namespaces our new macros; "rubymacros" after all.)
class Module # Define our macro here... def rm_attr_reader(name) define_method(name) do instance_variable_get("@#{name}") end end end
class Foo # And call our macro here... rm_attr_reader :bar def initialize(arg) @bar = arg end end
f = Foo.new "quux" f.bar # => "quux"
...
Parse tree substitution
The previous two above, essentially, are variations on string substitution. A "true" macro (true in the Lisp sense) can't be implemented in Ruby without extending Ruby syntax, and having (library) code capable of parsing the extended syntax. The next example uses the rubymacros gem:
macro rm_attr_reader name :( def ^name ^(RedParse::VarNode?["@#{name}"]) end ) end
(The syntax highlighter doesn't highlight this correctly because macro is not a reserved word in Ruby. Try to read it as kind of like a method. :( … ) and unary ^ constructs are also part of the new syntactical magic that rubymacros adds. You should see them as similar to double-quoted strings and the string interpolations (#{…}) within them.)
The last three versions ((we left out 2 of the 3 here)) of our new version of attr_reader share some common characteristics:
in all cases, an accessor method is defined within a quoted construct.
somehow that quoted method is interpolated into regular code so it can be executed.
the quoted method definition contains holes that get filled in by parameters that are passed in at the time of interpolation.
the quoted code and the values placed in the holes that fill it are data structures of some sort – either strings or parse trees. But they aren't code yet… they're things that will turn into code. They're… meta-code.The advantage of having data structures that represent nascent code is that they can be manipulated like any other data before you turn them into code.
The first version is also broadly similar to this approach. But instead of quoted code with holes in it, it uses magical interpreter methods to accomplish the same thing. Even with this first version, the name of the method and the instance variable are represented as strings which can be manipulated. "
---
https://en.wikipedia.org/wiki/General-purpose_macro_processor
GEMA was mentioned in passing in http://rubymacros.com/
---
todo:
---
" It is not a big deal to provide any kind of syntax on top of Lisp. In fact, much easier than with Ruby, thanks to the reader macros. – SK-logic Jun 3 '11 at 14:22
@SK-logic: True. However Lispers shy away from reader macros because once you go down that route it no longer feels like a Lisp. In fact some variants of Lisp, such as Clojure, reserve reader macros for the language designer. – btilly Jun 3 '11 at 16:38 "
---
" This bogus claim always comes up in discussions about powerful macro systems from people that haven't really used languages with powerful macro systems. Look at PLT Scheme to see the things that have been done that go far beyond lazy evaluation. OCaml's CamlP?4 is another good example. "
---
" Lisps, First-Class Special Forms, Fexprs, The Kernel Programming Language
I've been thinking about macros vs. fexprs on and off for about a year. The following bit of Scheme code exposes, what seems to me, a troublesome difference between function application and special forms.
(define (a) 1) (define (alpha) (a)) (alpha) ; 1 (define (a) -1) (alpha) ; -1
(define-syntax b (syntax-rules () ((_) 2))) (define (beta) (b)) (beta) ; 2 (define-syntax b (syntax-rules () ((_) -2))) (beta) ; 2 still!
The function being applied, a, in the first half of the code above, is dynamically determined when the application occurs. A redefinition of a means that calling alpha will reflect that change. Dynamic languages. Late binding. Yay!
In the second half of the code above, the special form, b, is determined and expanded when beta is first defined and redefining b has no affect on later calls to beta.
I know this is an old issue but looking at John Shutt's reasonably recent The Kernel Programming Language (and on wikipedia and here) he seems to have "solved" the problem by inventing $vau expressions which fall roughly into the fexprs category.
It seems a long time ago that Kent Pitman argued influentially against fexprs because they did not play well with the Lisps of 1980 with their dynamically scoped variables. The Kernel Programming Language has lexically scoped variables and the $vau form is not one considered by Pitman.
I've can only find mention of John Shutt's language once on Lambda the Ultimate and am surprised the $vau expression idea has not received more attention. It seems like a great idea enabling a more dynamic and consistently defined language. By Peter Michaux at 2009-10-15 05:07
| LtU? Forum | previous forum topic | next forum topic | other blogs | 20730 reads |
Unfortunately, there seems to be about two or perhaps three of us in the Scheme community who would care to fix this kind of problem, then a lot of others riding other hobby horses.
Hell, I just want something roughly like R4 but with FEXPRs and first class environments and I'd call Scheme "done" but... it's not a popular notion, afaict.
-t By Thomas Lord at Thu, 2009-10-15 05:48
| login or register to post comments |
" -- http://lambda-the-ultimate.org/node/3640
"
Understanding fexprs versus macros (versus functions)
The excessively short answer is: no, fexprs aren't at all like macros. Hopefully I can do better than that, though.
There are three differences floating around here between how ordinary functions work, how macros work, and how fexprs work.
First, when an ordinary function is called, its operands are evaluated; in Scheme terminology, at least, the results of these evaluations are called arguments (arguments are the results of evaluating the operands). The arguments are the inputs that will be made available to later processing by the function. Pretty much the only thing that macros and fexprs have in common is that they don't evaluate their operands: the operands become the inputs, rather than the arguments becoming the inputs as for an ordinary function.
Second, once the inputs have been chosen (either arguments or operands), an ordinary function computes a value by evaluating the expressions that make up its body, in a local environment that makes its inputs visible. In Lisp, the result of the last of these evaluations becomes the output of the function (which in C would be explicit using a return statement). Fexprs do exactly the same thing, although the inputs that are visible in that local environment are the operands rather than the arguments. Macros are the ones that are usually different at this stage: C-style macros, or hygienic Lisp macros, substitute the inputs into a template to produce an output expression. ("Traditional" Lisp macros actually do the same thing as fexprs at this stage to produce an output expression.)
Third, when an ordinary function produces its output, that output becomes the result of the call to the function. The same for a fexpr. But the output of the macro is an expression that is then evaluated in the context where the macro was called in the first place. That's what makes it a "macro", the fact that its output is then evaluated.
If you want to do some kind of manipulation on the operands before they're evaluated, you can't use an ordinary function; you have to use a fexpr or a macro. But the practical difference (IMHO, of course) between a fexpr and a macro is that with a macro, you specify how to construct an expression that specifies what you want done, whereas with a fexpr you directly specify what you want done.
One advantage of fexprs over macros is that when you're trying to make sure your code will do what you really want it to do, it's easier to do so by actually specifying what you want done (the fexpr strategy), than by specifying how to build an expression that will specify what you want done (the macro strategy). Like the difference between manipulating something with your hands, and manipulating it with tongs. (As I recall, Pitman even acknowledged this clarity advantage to fexprs in his paper, along the way to condemning fexprs for other reasons.) By John Shutt at Fri, 2009-10-23 22:32
| login or register to post comments |
"
" Macros are usually *required* to be processed at compile time, whereas fexprs and ordinary functions are, conceptually, processed at runtime. That was kind of the point of Pitman's favoring macros over fexprs, that because macros are required to be processed at compile time, they can't possibly fail to be analyzable at compile time — and the sense in which fexprs "aren't well-behaved" is exactly that they are sometimes impossible to analyze at compile time. In the extreme, you can't even tell whether the operands are going to be evaluated at all, because it depends on whether the operator evaluates *at runtime* to a fexpr, and it's impossible in general to determine at compile time what the result of a computation will be at runtime. This problem with fexprs is much, much more extreme if the language is dynamically scoped, which I believe is what caused the rejection of fexprs of which Pitman was the visible spokesman (because the mainstream Lisps in 1980 were dynamically scoped); and other facets of a Lisp language design can also be fine-tuned to maximize the analyzability of programs in the presence of fexprs.
A second reason to love fexprs (besides the fundamental clarity advantage of directly saying what you mean, which I mentioned before) is that fexprs don't require a logically separate phase of computation. That is, they don't require a phase-separation that restricts what the programmer can do. This is the flip side of the unanalyzability objection: the only way to guarantee analyzability at compile time is to impose restrictions that preclude all fexpr-based situations that can't be compile-time analyzed. Arbitrary restrictions are, of course, inherently objectionable to those of us who believe uniformity is prerequisite to maximal expressive power (in my case, I'd say, particularly, maximal *abstractive* power).
All this is about whether things *appear to the programmer* to be performed at compile time or runtime. Optimizations are beside the point(s): optimizations that cause some things to be taken care of at compile time rather than runtime (or vice versa) shouldn't even be visible, except perhaps by making things run faster, and in fact if they *are* visible then that's a bug.
The compile-time-versus-runtime issue is worrisome enough that various efforts have been made to create macros that are processed at runtime, or at least appear to be. (It generally wouldn't occur to those who do this to use fexprs instead of inventing ever-more-complicated kinds of macros, because they either aren't aware of fexprs, or mistakenly think that fexprs are unmanageable — a very understandable mistake, because it's been the folk wisdom about fexprs for decades.) Aside from individual shortcomings of particular first-class-macro efforts, there is still always a sort of impedance mismatch between the "first-class macros" (whatever exact form they take) and ordinary functions, simply because they're two separate devices with different computational properties, creating complex interactions. Fexprs, depending on just how they're integrated into the language design, may avoid the impedance mismatch as well. (I consider the way fexprs are integrated into Kernel to be especially smooth.) By John Shutt at Thu, 2009-10-29 12:45
| login or register to post comments |
" 3) You excitedly note that Scheme's hygenic macros give you essentially turing complete extensibility of the compiler - well, fexprs give you essentially turing complete extensibility of the interpreter. Why favor one over the other? The core language definition should use fexprs (and first class environments) because you build hygenic macro subsets atop that and have a perfectly workable semantics. "
"
Flexibility is nice, but...
Flexibility is nice, but I don't believe fexprs are offering much 'new and useful' flexibility, by which I refer to flexibility that is both useful in practice and not readily available by leveraging other mechanisms. Further, you pay a not insignificant price for those rather marginal benefits.
If I wanted to define 'a' such that its meaning can change later and affect earlier definitions, I'd have defined a with reference to a mutable cell then modified that cell explicitly (with, IIRC, 'set!' in Scheme). Other alternatives would include a tweak to the environment to support 'special vars', or a tweak to the syntax to thread a context object similar to a Haskell monad. In all cases, the semantics would be far more explicit and apparent. By dmbarbour at Sun, 2009-10-18 00:17
| login or register to post comments |
It sounds to me like you want to define a far more complicated core language because of some "principled" avoidance of fexprs, after which you will add library code to simulate fexprs.
Fexprs have a nice, simple, operational semantics atop which all the statically analyzable things you want can be built as subset environments and, meanwhile, fexprs perform their basic function of adding extensibility to "eval".
-t By Thomas Lord at Sun, 2009-10-18 03:47
| login or register to post comments |
"
---
so i guess there are various more-or-less metaprogrammy kinds of function calling:
very metaprogrammy: callee gets: for each input, an AST that evaluates to the input; a first-class environment representing each input's environment at the time of definition; a first-class environment representing its own environment right now
a little less: callee gets: an thunk that if evaluated, evaluates to its input; a first-class environment representing its own environment right now
a little less: callee gets: values for each input (each input is automatically evaluated); a first-class environment representing its own environment right now
a little less: callee gets: values for each input (each input is automatically evaluated)
there's also a mutability spectrum: call-by-name seems to me to only be different from call-by-need if there are side-effects; also call-by-reference only seems to be different from call-by-value if there is mutability
the bottom of both hierarchies seems to be strict call-by-value
---
Rust has cool macros:
" / Creates a `Vec` containing the arguments. / / `vec!` allows `Vec`s to be defined with the same syntax as array expressions. / There are two forms of this macro: / / - Create a `Vec` containing a given list of elements: / / ``` / let v = vec![1, 2, 3]; / assert_eq!(v[0], 1); / assert_eq!(v[1], 2); / assert_eq!(v[2], 3); / ``` / / - Create a `Vec` from a given element and size: / / ``` / let v = vec![1; 3]; / assert_eq!(v, [1, 1, 1]); / ``` / / Note that unlike array expressions this syntax supports all elements / which implement `Clone` and the number of elements doesn't have to be / a constant. / / This will use `clone()` to duplicate an expression, so one should be careful / using this with types having a nonstandard `Clone` implementation. For / example, `vec![Rc::new(1); 5]` will create a vector of five references / to the same boxed integer value, not five references pointing to independently / boxed integers.
---
rust also has syntax extensions (procedural macros) and code generation (like source filters but at the project level):
" Macros have limits. They can't execute arbitrary code at compile time. This is a good thing for security and repeatable builds, but sometimes it's not enough. There are two ways to deal with this in Rust: syntax extensions (AKA "procedural macros"), and code generation (AKA build.rs). Both of these basically give you carte-blanche to execute arbitrary code to generate source code.
Syntax extensions look like macros or annotations, but they cause the compiler to execute custom code to (ideally) modify the AST. build.rs files are a file that Cargo will compile and execute whenever a crate is built. Obviously, this lets them do anything they please to the project. Hopefully they'll just add some nice source code. "
---
comments on http://beautifulracket.com/first-lang.html :
giacomone 2 days ago
> If a datum sounds like a syntax object, that’s no coincidence — a syntax object is just a datum with some extra information attached. The #' prefix notation we used for a syntax object now connects logically: we can think of the # as representing the extra info that gets attached to the datum
This is not clear, can someone explain please?
reply
samth 2 days ago
Not sure which part isn't clear.
First, syntax objects are data, but with extra information, such as binding information, source location, etc.
Second, the #' form (written out it's called `syntax`) is similar to the ' form (written out as `quote`). But the first one includes the extra information to make a syntax object. Also, it has an extra character, which the chapter points out can be seen as going along with the extra information.
reply
---
scala supports reflection but scalajs and scalanative don't, causing probs:
acjohnson55 14 hours ago
And as the original author of Scala.js pointed out in a Scala Days talk today, unrestricted reflection means that it's impossible to do dead code elimination, which is a non-starter for real-world use.
reply
Negative1 18 hours ago
Play uses it specifically for in it’s JSON module; you can define readers and writers that allow you to parse the structure of a JSON object. To avoid having to be very explicit, you can define a case class that directly mirrors the structure of that object. e.g. given some case class that looks like this: case class SomeObj?(someInt: Int) you could do: val theObj = SomeObj?((json \ “someInt”).as[Int]) or implicit val reads: Reads[SomeObj?] = ((json \ “someInt”).read[Int])(SomeObj?.apply _) with reflection, you get: implicit reads: Reads[SomeObj?] val theObj = reads.validate(json)
Seems trivial with this small example but with large objects it keeps your code much more maintainable and consistent since reading and writing from that case class is the same.
As far as it not working in Scala.js, from my understanding it has to do with how the reflection library is shared between both runtime and compile time implementations.
You can read more about it here: http://docs.scala-lang.org/overviews/reflection/overview.html
reply
---
kyberias 3 days ago
Can you give us a couple of examples of how lisp macros "abstract away" patterns?
reply
TeMPOraL? 3 days ago
Take the example from [0]:
> Also - there are such "nonexciting patterns" in OOP languages as well, and they are much less often abused. For example "a switch inside a while" pattern. Which I think I will need to name "StateMachinePattern?" to make it cool and stop people refactoring it into strategies and stuff.
> There is value in one-screen definition of the whole machine, instead of separating it into 6 files.
In Lisp you could create a macro, say, define-state-machine, and use it like that:
(define-state-machine :my-state-machine
((:state-1
(do-something)
(do-something-else)
(if (condition)
(transition :state-2)
(transition :state-3)))
(:state-2
(do-something)
(transition :state-3))
(:state-3
(do-something)
(when (should-quit)
(quit-state-machine)))))This macro could easily expand to a "switch inside a while" (or, possibly, to a "let over lambda over cond", or maybe even into a series of low-level tagbody and go constructs). The resulting abstraction is clean and communicates its meaning well.
--
Different example - when writing macros in Common Lisp, there are two things one usually has to be wary of - unwanted variable capture, and unwanted re-evaluation. So you might end up manually assigning gensyms[1] to avoid variable capture, and manually creating lexical environments to evaluate passed forms only once. But you can also abstract it away! For instance, with with-gensyms and once-only macros from Alexandria[2]. Or you could build something like defmacro! described in Let Over Lambda[3], i.e. a macro that automatically code-walks your macro definition and extracts variables named g!foo and o!bar to apply gensyms and once-only to them, respectively.
--
Those are only two examples, but the general idea is - whenever you see yourself repeating the same boilerplate in similar places to express a concept, you can wrap that boilerplate inside a macro and make the compiler generate it for you. Since Lisp makes all of its capabilities available for you during macroexpansion, you can continue this until you're satisfied that your code is readable, boilerplate-free and clearly expresses your intentions.
--
[0] - https://news.ycombinator.com/item?id=11730248
[1] - gensyms are symbols generated on demand, that are guaranteed to have no possible collision with anything ever
[2] - https://common-lisp.net/project/alexandria/draft/alexandria....
[3] - http://letoverlambda.com/index.cl/guest/chap3.html#sec_6
reply
ArkyBeagle? 2 days ago
I use this sort of thing in Tcl ( for path-dependent reasons ) A Whole Lot. You can have arrays, indexed by state, and use the "eval" operator to "eval $fsm($state)"
fsm is an array of strings, indexed by whatever.
I have made dynamic state machine generators this way, although it gets kinda disorienting.
The tactical advantage to Tcl (for me) is that it has serial ports and sockets as first-class objects with essentially identical semantics. I work in the embedded space, and this seems 1) unusual and 2) a very nice thing to have to write comprehensive test rigs.
I rather like it better because it's all in the "string" & "list" domain rather than the lambda domain. I really should try this in Lisp just to see how wierd it gets.
reply
---
alankay1 8 hours ago
I don't think Aspects is nearly as good an idea as MOP was. But the "hyperness" of it is why the language and the development system have to be much better. E.g. Dan Ingalls put a lot of work into the Smalltalks to allow them to safely be used in their own debugging, even very deep mechanisms. Even as he was making these breakthroughs back then, we were all aware there were further levels that were yet to be explored. (A later one, done in Smalltalk was the PIE system by Goldstein and Bobrow, one of my favorite meta-systems)
reply
nickpsecurity 8 hours ago
I have a few things in my links on the topic of safe metaprogramming outside your own work. Here's a few I could remember on top of head:
Type-safe metaprogramming Sheard
Type-safe, reflective metaprogramming Microsoft
http://research.microsoft.com/apps/video/dl.aspx?id=103561
Rascal - Metaprogramming language and platform
So, given work like that, what remaining tough problems are there before you would find a metaprogramming system safe and acceptable? Or do we have the fundemantals available but you just don't like the lack of deployment in mainstream or pragmatic languages and IDE's?
Note: Just dawned on me that you might mean abstract programming in the sense of specifying, analyzing, and coding up abstract requirements closer to human language. Still interested in what gripes or goals you have on that end if so.
reply
--- " macro usage in Elixir is always explicit - you must import/use macros at the top of a module, and the implementation of those macros is easy to find by going to that module definition. It's not nearly as dangerous as say, monkey-patching in Ruby. "
" It's ok if I'm on my IDE that helps expand macros to quickly check the flow, but when it's on a webpage or in a gist somewhere, you're on your own. " ---
" https://news.ycombinator.com/item?id=12123832
... The beauty of a good DSL isn't the syntax (I know, hard to believe!), but in novel approaches to code flow execution and other deep semantics. Golang's goroutines are beautiful. Rust's ownership analyzer and prover are what _defines_ the language. Haskell's lazy evaluation open up whole new dimensions for problem solving (and headaches). If your language framework doesn't at least preserve the ability to implement those things cleanly and performantly, it's not adding much value and is basically a toy. It's not like writing a lexer for a DSL is a serious impediment. (Using Lua's LPeg, for example, you can write parsers and build an AST for most major languages in a few hundred lines of code.) "
---
lisper 9 hours ago [-]
No, you were just flat-out wrong.
> Lisp can only metaprogram its own syntax, you can't introduce a C-like syntax.
This is a flat-out false statement, and it reflects a deep but sadly common misunderstanding of how Lisp works and why it's cool. Even if Lisp did not have reader macros as a standard feature, you could still write a C compiler in Lisp more easily than you could write one in any other language. The whole concept of "metaprogramming a syntax" (so that you can talk about whether or not a language can "only metaprogram its own syntax") is non-sensical, a category error. Metaprogramming is simply writing programs that write programs. What makes Lisp cool is that it separates the syntax from the program. A Lisp program, unlike all other languages, is not text. A Lisp program is a data structure. Lisp happens to define a textual surface syntax (S-expressions) that allows you to easily convert text into the particular data structures that are Lisp programs (and also incidentally data structures that are not Lisp programs) but you can also produce these data structures in other ways, like, for example, writing programs that produce them. The textual surface syntax is a detail. Writing an infix parser in Lisp is an elementary exercise, and you could use that parser to parse C-like code whether or not you had reader macros. All reader macros let you do is seamlessly integrate that C-like syntax into the Lisp REPL rather than having to embed your new language in strings or read it from files.
reply
lisper 9 hours ago [-]
> Lisp's syntax is better because it makes an easy compile target
Not quite. Lisp's syntax is (mostly) irrelevant. Lisp's design is better because you don't have to worry about syntax to use Lisp as a compile target.
There's a huge conceptual difference between
(eval '(some code))
and (using Javascript or Python as an example)
eval("Some code")In JS/Python case you are passing a string to EVAL. In the Lisp case you are passing a data structure to EVAL. That data structure has been generated by the reader (by virtue of using QUOTE) but that is not the important part. You could as easily have written:
(eval (cons (quote some) (cons (quote code) nil)))
or
(eval (some-function-that-generates-code))
with the point being that some-function-that-generates-code does not return a string, it returns a data structure, so there is no syntax. The whole concept has evaporated because you're not dealing with strings any more. Syntax is for humans. When you have programs writing programs, syntax just gets in the way. But when your EVAL function takes a textual representation of a program as a string you have no choice but to muck with syntax. This is the reason that the misconception that syntax is essential is so widespread. It seems essential only because most programming languages don't distinguish between READ and EVAL.
reply
qwertyuiop924 7 hours ago [-]
point being, you can trivially parse code of any syntax, just like any other language. What makes lisp different is that once you have a parse tree of sexprs, you can then traverse that tree very easily: Lisp has tools built right into the language. You can then very easily transform that tree: In fact, you can write a hook whereby the code being written can provide arbitrary transformers, which are called by the compiler when seen in code: This is all a macro is: it's an API for code to hook into its own compiler: much like Rust's compiler addons. Unlike compiler addons, however, they don't depend on compiler internals: In fact, if a Lisp implementation didn't have macros and readtables, you could write a preprocessor to add them in (but gensym/hygene would be a pain to do). However, this would all be very confusing if the internal tree didn't look like the external representation. This is why Lisp is written in sexprs. The reason you see so many DSLs in lisp being written in a lispy syntax is that lisp hackers, like all other hackers, are fundamentally lazy: Since there's already a lisp syntax parser built into the language, they don't have to write their own, and macros often suffice for the most common reasons to write a DSL, which means that they didn't even have to write a compiler, they just hooked into the existing one. Besides, sexprs provide advantages even if you are writing your own language, and you can overload lisp's syntactic shortcuts for your own purposes.
reply
---
chriswarbo 9 hours ago [-]
If you want to handle a C-like language, you just need to make a parser; once you've got an AST, that is your "Lisp syntax", so you can do all of your fancy metaprogramming, interpretation, compilation, etc.
Using a parsing framework (e.g. parser combinators, parser generators, ometa, etc.) to parse a language which has a well-specified syntax (e.g. a BNF grammar) is pretty routine and mechanical; usually it just requires a one-to-one translation of the BNF form into the parser framework's syntax, then fiddling with ordering and precedence rules until your tests pass.
Starting from scratch, I could knock out a C-like parser in maybe half an hour, e.g. using parsec or ometa. I'm sure Lisps have equivalents (I know Racket has built in support for defining new concrete synax)
reply
dragonwriter 9 hours ago [-]
> Because Lisp is homoiconic, you don't code in (what I personally believe to be) reasonable syntax, you code in abstract syntax trees.
There's no reason a regular macro couldn't implement a basically C-like syntax (though it would have Lisp, not C, tokenization rules, to the extent that those are different) on what is passed as its argument. The actual call to the macro would have typical Lisp syntax, but what the macro consumes would be interpreted with whatever syntax was implemented by the macro.
(And, a reader macro could implement more deeply C-like syntax.)
reply
qwertyuiop924 7 hours ago [-]
Well, for regular macros, only if your C syntax was in a string. Otherwise, lisp would try to tokenize it, and choke. Heck, if your lisp of choice offers an eqivalent of TCL's uplevel, you wouldn't even need a macro, just a function.
reply
dragonwriter 7 hours ago [-]
> Well, for regular macros, only if your C syntax was in a string. Otherwise, lisp would try to tokenize it, and choke.
Which is why I said normal (rather than reader) macros could do a "basically C-like syntax" but with Lisp tokenization rules.
reply
---
qwertyuiop924 7 hours ago [-]
What would that look like?
reply
wcrichton 9 hours ago [-]
I've never seen something like this before--do you know of any examples of this kind of macro usage?
reply
dragonwriter 9 hours ago [-]
This article provides a simpler (but conceptually similar, where it comes to implementing syntax different than standard Lisp S-expression syntax) example of using reader macros to implement JSON literals in Lisp.
https://gist.github.com/chaitanyagupta/9324402
reply
---
ktRolster 10 hours ago [-]
fwiw IO language is a pretty good prototyping language too.....good for experimenting with different syntaxes, etc
reply
--- " Wyvern: at CMU, Jonathan Aldrich's group has been working on the Wyvern language, which incorporates a novel language feature called Type-Specific Languages (TSLs) developed by Cyrus Omar. TSLs are a means of extending the syntax of Wyvern by defining parsers as a first-class member of the language and allowing users to create their own mini-languages inside of Wyvern: This is similar to procedural macros in Rust which are essentially functions from TokenStream? → AST that define an alternate syntax for the language. TSLs, however, are notable in that they are unambiguous and composable. The general idea is that a TSL returns a value of a known type, and the compiler can do type inference to determine the expected type of a value, so it can resolve ambiguity between languages based on their types. Additionally, the TSLs can be used within each other. When you define a new TSL, you get interoperability with every other TSL defined in Wyvern for free! Imagine if you could freely mix C, OCaml, Javascript and HTML with type-safe interop. This style of composition is the future of front-end, or syntax level, interop.
Figure 3: HTML templating with SQL and CSS mixed in using Wyvern.
" -- http://notes.willcrichton.net/the-coming-age-of-the-polyglot-programmer/
---
bogdanontanu (Sol_Asm guy?) says " However from an experienced ASM programmer's point of view the greatest downside of FASM is it's macro system that is very complicated to read and understand and has a big design flaw. "
sol_asm has " Macros with local variables and argumets: MACRO/MARG/EXITM " http://www.oby.ro/sol_asm/docs/sol_asm_manual.htm#7.6
---
already skimmed and i don't have to read again, but just in case i'm looking for it, there are some details on FASM's macro system in:
---
http://www.plantation-productions.com/Webster/www.artofasm.com/Linux/HTML/Macros.html
http://www.plantation-productions.com/Webster/www.artofasm.com/Linux/HTML/DSLs.html
---
" HLA macros provide some very powerful facilities not found in other macro assemblers. One of the unique features that HLA macros provide is support for multi-part (or context-free) macro invocations. This feature is accessed via the #terminator and #keyword reserved words. Consider the following macro declaration:
program demoTerminator;
TopOfLoop?:
jmp TopOfLoop?;
LoopExit?:
static
i:int32;
begin demoTerminator;
mov( 0, i );
InfLoop?
stdout.put( "i=", i, nl );
inc( i );
endInfLoop;
end demoTerminator;
The # terminator keyword, if it appears within a macro, defines a second macro that is available for a one-time use after invoking the main macro. In the example above, the endInfLoop macro is available only after the invocation of the InfLoop? macro. Once you invoke the EndInfLoop? macro, it is no longer available (though the macro calls can be nested, more on that later). During the invocation of the # terminator macro, all local symbols declared in the main macro ( InfLoop? above) are available (note that these symbols are not available outside the macro body. In particular, you could refer to neither TopOfLoop? nor LoopExit? in the statements appearing between the InfLoop? and endInfLoop invocations above). The code above, by the way, emits code similar to the following:
_0000_HLA_:
stdout.put( "i=", i, nl );
inc( i );
jmp _0000_HLA_;
_0001_HLA_:
The macro expansion code appears in italics. This program, therefore, generates an infinite loop that prints successive integer values.
These macros are called multi-part macros for the obvious reason: they come in multiple pieces (note, though, that HLA only allows a single # terminator macro). They are also referred to as Context-Free macros because of their syntactical nature. Earlier, this document claimed that you could refer to the # terminator macro only once after invoking the main macro. Technically, this should have said "you can invoke the terminator once for each outstanding invocation of the main macro." In other words, you can nest these macro calls, e.g.,
InfLoop?
mov( 0, j );
InfLoop?
inc( i );
inc( j );
stdout.put( "i=", i, " j=", j, nl );
endInfLoop;
endInfLoop;
The term Context-Free comes from automata theory; it describes this nestable feature of these macros.
As should be painfully obvious from this InfLoop? macro example, it would be nice if one could define more than one macro within this context-free group. Furthermore, the need often arises to define limited-scope scope macros that can be invoked more than once (limited-scope means between the main macro call and the terminator macro invocation). The # keyword definition allows you to create such macros.
In the InfLoop? example above, it would be nice if you could exit the loop using a statement like brkLoop (note that break is an HLA reserved word and cannot be used for this purpose). The # keyword section of a macro allows you to do exactly this. Consider the following macro definition:
TopOfLoop?:
jmp LoopExit?;
jmp TopOfLoop?;
LoopExit?:
As with the # terminator section, the #keyword section defines a macro that is active after the main macro invocation until the terminator macro invocation. However, # keyword macro invocations do not terminate the multi-part invocation. Furthermore, # keyword invocations may occur more that once.
...
The discussion above introduced the #keyword and #terminator macro sections in an informal way. There are a few details omitted from that discussion. First, the full syntax for HLA macro declarations is actually:
statements
statements
note: additional #keyword declarations may appear here
statements
There are three things that should immediately stand out here: (1) you may define more than one # keyword within a macro. (2) # keywords and # terminators allow optional parameters. (3) # keywords and # terminators allow their own local symbols.
The scope of the parameters and local symbols isn’t particularly intuitive (although it turns out that the scope rules are exactly what you would want). The parameters and local symbols declared in the main macro declaration are available to all statements in the macro (including the statements in the #keyword and #terminator sections). The InfLoop? macro used this feature since the JMP instructions in the brkLoop and endInfLoop sections referred to the local symbols declared in the main macro. The InfLoop? macro did not declare any parameters, but had they been present, the brkLoop and endInfLoop sections could have used those parameters as well.
Parameters and local symbols declared in a #keyword or #terminator section are local to that particular section. In particular, parameters and/or local symbols declared in a #keyword section are not visible in other #keyword sections or in the #terminator section.
One important issue is that local symbols in a multipart macro are visible in the main code between the start of the multipart macro and the terminating macro. That is, if you have some sequence like the following:
InfLoop?
jmp LoopExit?;
endInfLoop;
The HLA substitutes the appropriate internal symbol (e.g., _xxxx_HLA_ ) for the LoopExit? symbol. This is somewhat unintuitive and might be considered a flaw in HLA’s design. Future versions of HLA may deal with this issue; in the meantime, however, some code takes advantage of this feature (to mask global symbols) so it’s not easy to change without breaking a lot of code. Be forewarned before taking advantage of this "feature", however, that it will probably change in HLA v3.x. An important aspect of this behavior is that macro parameter names are also visible in the code section between the initial macro and the terminator macro. Therefore, you must take care to choose macro parameter names that will not conflict with other identifiers in your program. E.g., the following will probably lead to some problems:
" [2]
---
" Crystal uses macros to achieve that while reducing boilerplate code. This example is taken from Kemal, an awesome web framework for Crystal.
HTTP_METHODS = %w(get post put patch delete options)
{% for method in HTTP_METHODS %} def [[image:method.id?]](path, &block : HTTP::Server::Context -> _) Kemal::RouteHandler::INSTANCE.add_route([[image:method?]].upcase, path, &block) end {% end %}
Here’s how the DSL declaration is done in Kemal, looping through the HTTP_METHODS array to define a method for each HTTP verb. By the way, macros are evaluated at compile-time, meaning that they have no performance penalty. "
---
some random metaprogramming used by various approaches to CUDA:
examples:
" 2. OpenACC? SAXPY
If you have been following Parallel Forall, you are already familiar with OpenACC?. OpenACC? is an open standard that defines compiler directives for parallel computing on GPUs (see my previous posts on the subject). We can add a single line to the above example to produce an OpenACC? SAXPY in C.
void saxpy(int n, float a, float * restrict x, float * restrict y) {
... Perform SAXPY on 1M elements saxpy(11>(N, 2.0, x, y);
cudaMemcpy(y, d_y, N, cudaMemcpyDeviceToHost);
CUDA C++ was the first general-purpose programming language on the CUDA platform. It provides a few simple extensions to the C language to express parallel computations. GPU functions, called kernels are declared with the__global__ specifier to indicate that they are callable from the host and run on the GPU. Kernels are run by many threads in parallel. Threads can compute their global index within an array of thread blocks by accessing the built-in variables blockIdx, blockDim, and threadIdx, which are assigned by the hardware for each thread and block. To launch a kernel, we specify the number of thread blocks and the number of threads in each thread block as arguments to the kernel function call inside 2>, which we call the execution configuration. We copy data to and from GPU device memory using cudaMemcpy().
That is CUDA C in a nutshell. As you can see, the SAXPY kernel contains the same computation as the sequential C version, but instead of looping over the N elements, we launch a single thread for each of the N elements, and each thread computes its array index using blockIdx.x*blockDim.x + threadIdx.x.
.... 5. CUDA Fortran SAXPY
module mymodule contains attributes(global) subroutine saxpy(n, a, x, y) real :: x(:), y(:), a integer :: n, i attributes(value) :: a, n i = threadIdx%x+(blockIdx%x-1)*blockDim%x if (i<=n) y(i) = a*x(i)+y(i) end subroutine saxpy end module mymodule
program main use cudafor; use mymodule real, device :: x_d(220), y_d(220) x_d = 1.0, y_d = 2.0
! Perform SAXPY on 1M elements call saxpy<<<4096, 256>>>(2**20, 2.0, x_d, y_d)end program main ...
4. Thrust SAXPY
using thrust::placeholders;
int N = 13>(N, 2.0, x, y); notation for callsite metaprogramming arguments
---
some examples of Python metaprogramming:
---
forth DSL example?:
https://github.com/jamesbowman/swapforth/blob/master/j1a/basewords.fs
---
"
kragen 371 days ago
| parent [-] | on: On being the maintainer and sole developer of SPIT... |
This is a very interesting idea, and I have been finding inspiration in it. New machine architectures by way of #include files! Or cat! I've been thinking that maybe the right compile-time programming model is something like term-rewriting (more or less like C++ templates or Prolog or Aardappel) rather than textual substitution. I wrote some more thoughts on the matter at https://lobste.rs/s/kfpsou/what_is_everyone_working_on_this_... but I still haven't gotten very far on it. "
from that link: " Also, I’ve been thinking that maybe if the Stepanov/coqasm approach is the right one, then maybe instead of Stoneknifeforth, the ideal minimal bootstrap compiler might be a simple term-rewriting system, like Aardappel, being used as a retargetable macro assembler, also inspired by this HN comment by jhallenworld about generic assemblers.
Unfortunately when I hacked together a simple term-rewriting system the other night and used it to do some simple symbolic differentiation, it was like an entire page of Scheme! (Plus 13 lines of symbolic differentiation rules.) I hoped it would be simpler. It’s possible I’ve overcomplexified the problem of term-rewriting, but I feel like maybe if it’s an entire page of Scheme then it will be probably several pages of assembly language, and so maybe term-rewriting isn’t the best bootstrap to start with after all. StoneKnifeForth? is only about a page of code, and in theory can compile itself, although I need to figure out why the ELF it generates is rejected by current Linux kernels. But maybe a term-rewriting macro-assembler is the thing to build on top of StoneKnifeForth? as a stepping stone to higher-level programming, rather than something like Ur-Scheme. "
th HN comment that inspired him is:
jhallenworld 373 days ago
| parent | favorite | on: On being the maintainer and sole developer of SPIT... |
This makes me think of a generic assembler I wrote at one point (in C++ I think, I should put it on github). The idea was that you could define the instruction set right in the assembly source. It included a backtracking BNF parser to support it with these pseudo ops:
.rule name value pattern ; Define a syntax rule
.insn pattern ; Define an instruction
... ; Macro expanded on pattern match
.end"pattern" contains literal characters, whitespace and references to other rules with "<rule-name>" or <expr> for a math expression.
"value" is a comma separated list of expressions which can contain "argN" to reference the Nth value from the pattern (as returned by embedded rules).
For example, this is how you could construct the
instructions "lda <expr>", "lda #<expr>", "ldb <expr>",
and "ldb #<expr>":.rule dual 0x01 lda .rule dual 0x02 ldb .rule mode 0xf8,arg1 <expr> .rule mode 0xfa,arg1 #<expr>
.insn <dual> <mode>
.byte arg1|arg2 ; Emit op-code
.word arg3 ; Emit argument
.endSNOBOL4 itself is not an assembler, but I think you could make one like this from it.
---
"Often, a good way to detect when internalized types are static (and thus not tags or classes) is that they are erased at runtime---at least by default[^typeable]. This means that there is no way for the program's behavior to change dynamically according to types. Essentially, this outlaws reflection and, in simple contexts, is usually A Good Thing."
i like being able to introspect at runtime, but this is probably one of the rungs on our 'meta ladder', meaning that a module must request this feature explicitly, and by doing so its 'metaprogramming level' (which is bad) is bumped up.
---
a problem with optimizing implementations in the potential presence of metaprogramming:
Consider the 'while' control structure. Optimizers (and transpilers) will need to make assumptions about how 'while' relates to control flow (to CFGs, etc). But if 'while' is just an ordinary function that takes a block, and especially if this function can itself be tweaked or overridden when it appears in a metaprogrammed context, then how can an compiler know when these assumptions are valid?
So, we need to make it easy to (a) identify when stdlib functions such as WHILE are present in their original form (not overridden or tweaked by metaprogramming), and (b) identify contexts in which normal control flow has not been overridden by metaprogramming..
---
" Lisp: It's Not About Macros, It's About Read ... read is a builtin function in most Lisps that reads an object. An object can be any kind of atom (number, string, etc.), or a data structure like a list. Here are a few examples:
(read "3") ; 3 (read "foo") ; 'foo (read "\"foo\"") ; "foo" (read "(foo bar baz)") ; (list 'foo 'bar 'baz)
Wait a second, if you take a look at Lisp code, it's really all made up of lists:
(define (foo x) (+ x 1))
Once you understand that Lisp code is data (lists of atoms), you realize that you can use read to read in Lisp code as data. And since Lisp comes with a set of functions for elegantly processing lists, suddenly it's really easy to parse Lisp code. "
---
deep vs. shallow DSL embedding (some think of it as a continuum, some binary): http://www.cs.ox.ac.uk/publications/publication7584-abstract.html
"As far as I'm aware, while you can use the strict definition (is there, in the metalanguage, the creation of an AST?) they're often discussed as more of a continuum. HOAS is a great example. There is the creation of an AST, but perhaps the most important and tricky part of any AST, the binding system, is left to the metalanguage. For that reason exactly I'm happy to say that HOAS is "shallower" than, say, a de Bruijn indexed binding system. "
---
"For that matter, you should also have a choice between Scheme-style pattern-matching macros and Lisp-style code-style macros. They're very different, and each kind is better (cleaner) in some situations. People often act as if hygiene is synonymous with define-syntax, but the pattern-template style is orthogonal to the question of hygiene. " -- [4]
---
" Syntax in general is a problem. Lisp has a little syntax, and it shows up occasionally as, for instance, '(foo) being expanded as (quote foo), usually when you least expect it. Truth be told, Lisp should probably have a skinnable syntax. That implies a canonical abstract syntax tree, which of course hasn't been defined (and in many implementations isn't even available to you, the way it is in the Io language, say). Once you've got a canonical AST defined, syntax should, in theory, be like CSS chrome. Of course, there are plenty of bodies left in the trail of this particular theory as well. Someday... " -- [5]
---
" In any case, because macros are rarely supported "well enough" by the tools, and because they're not first-class functions, and so on, they wind up being second-class citizens. The rule "you should only use a macro when nothing else will do" implies that they really are a last resort, which (to me) is synonymous with band-aid. Yes, it's wonderful that you have the band-aid " -- [6]
---
Yegge notes that Io has a "canonical abstract syntax tree", which enables a "skinnable syntax" [7]
---
on Google's Python implementation written in Go: " > It's a hard-code compiler, not an interpreter written in Go. That implies some restrictions, but the documentation doesn't say much about what they are. PyPy? jumps through hoops to make all of Python's self modification at run-time features work, complicating PyPy? enormously. Nobody uses that stuff in production code, and Google apparently dumped it.
There are restrictions. I'll update the README to make note of them. Basically, exec and eval don't work. Since we don't use those in production code at Google, this seemed acceptable. "
" Just tried this out on a reasonably complex project to see what it outputs. Looks like it only handles individual files and not any python imports in those files. So for now you have to manually convert each file in the project and put them into the correct location within the build/src/grumpy/lib directory to get your dependencies imported. Unless I missed something somewhere.. The documentation is a bit sparse. "
---
Rust custom derive and procedural macros: https://doc.rust-lang.org/book/procedural-macros.html
---
ledgerdev 4 days ago [-]
> it's pretty awful for most things that aren't Java that target the JVM, like Clojure.
Curious, how does jigsaw make things awful for clojure?
reply
lvh 4 days ago [-]
Incomplete list:
reply
---
a frame evaluation API to CPython
https://www.python.org/dev/peps/pep-0523/
---
Tezos's regexp-based macros
P(A*AI)+R, C[AD]+R, DII+P
see my notes in [[plbook-plChMiscIntermedLangs?]] or http://web.archive.org/web/20170527202511/https://tezos.com/static/papers/language.pdf
---
richard_shelton 2 days ago [-]
An interesting approach! But old META II (mentioned in the original article) system and its derivatives are looking more practical to me. I think, it's more effective to have two simple DSLs, one for parsing and another one for tree transforming like it was done in Cwic and TREE-МЕТА, which are grew from original META II (self-described in a few lines of code).
reply
---
Ethereum Serpent language macro system is powerful enough to compute factorial:
---
"
le-mark 4 hours ago [-]
TCL is pretty old school and doesn't get a lot of attention nowadays. The main use I'm aware of is in BigIP? F5 config files. There are some really interesting things about it though, for example how control flow constructs (if/else, while) are implemented as commands, using built in uplevel and upvar[1] commands to control the scope of the currently executing code. Some people say it's lisp-like, that's one of the things they mean. " [8]
cchase88 3 hours ago [-]
I wrote a blog post where I show how to define "until" using Uplevel: http://christopherchase.cc/posts/tcl-until.html
reply
---
Footnotes:
1. 20, 2.0, x, y);
A Fortran OpenACC? SAXPY is very similar.
subroutine saxpy(n, a, x, y) real :: x(:), y(:), a integer :: n, i !$acc kernels do i=1,n y(i) = a*x(i)+y(i) enddo $!acc end kernels end subroutine saxpy
... ! Perform SAXPY on 1M elements call saxpy(220, 2.0, x_d, y_d)
3. CUDA C++ SAXPY
__global__ void saxpy(int n, float a, float * restrict x, float * restrict y) { int i = blockIdx.x*blockDim.x + threadIdx.x; if (i < n) y[i] = a*x[i] + y[i]; }
... int N = 1<<20; cudaMemcpy(d_x, x, N, cudaMemcpyHostToDevice); cudaMemcpy(d_y, y, N, cudaMemcpyHostToDevice);
Perform SAXPY on 1M elements saxpy<<<4096,256
2. <
3. 20; thrust::host_vector x(N), y(N); ... thrust::device_vector d_x = x; alloc and copy host to device thrust::device_vector d_y = y;
Perform SAXPY on 1M elements thrust::transform(d_x.begin(), d_x.end(), d_y.begin(), d_y.begin(), 2.0f * _1 + _2);
y = d_y; copy results to the host vector
I wrote about Thrust in some detail in a recent post, so I will just point out a few interesting features of thrust demonstrated here. This code shows that copying a host vector to a device vector is as simple as assignment. It performes SAXPY in a single line using thrust::transform(), which acts like a parallel foreach (∥∀!), applying a multiply-add (MAD) operation to each element of the input vectors x and y. The MAD operation uses Thrust’s “placeholder” syntax (inspired by Boost placeholders), which uses the placeholder templates _1 and _2 to access the elements of the x and y input vectors, respectively.
"
-- [9]
the metaprogramming i see is:
