![[Home]](http://www.bayleshanks.com/cartoonbayle.png)
Difference between revision 49 and current revision
No diff available.http://stackoverflow.com/questions/tagged/homoiconicity
---
" You might already have a language in mind that you want to use. Technically speaking, mal can be implemented in any sufficiently complete programming language (i.e. Turing complete), however, there are a few language features that can make the task MUCH easier. Here are some of them in rough order of importance:
A sequential compound data structure (e.g. arrays, lists, vectors, etc)
An associative compound data structure (e.g. a dictionary, hash-map, associative array, etc)
Function references (first class functions, function pointers, etc)
Real exception handling (try/catch, raise, throw, etc)
Variable argument functions (variadic, var args, splats, apply, etc)
Function closures
PCRE regular expressionsIn addition, the following will make your task especially easy:
Dynamic typing / boxed types (specifically, the ability to store different data types in the sequential and associative structures and the language keeps track of the type for you)
Compound data types support arbitrary runtime "hidden" data (metadata, metatables, dynamic fields attributes)Here are some examples of languages that have all of the above features: JavaScript?, Ruby, Python, Lua, R, Clojure. " -- https://github.com/kanaka/mal/blob/master/process/guide.md
https://news.ycombinator.com/item?id=9123065 'codygman' compares various language implementations of mal using 'cloc' to see which implementations were shortest.
the shortest were:
CoffeeScript?, Clojure, OCaml, Ruby, Python
the full table was (omitting support files) was:
Language files blank comment code
-------------------------------------------------------------------------------
Rust 17 294 155 3919
C 18 446 383 3356
C# 19 472 235 3314
Visual Basic 17 322 131 2945
Java 17 322 134 2872
Go 17 299 147 2636
Bourne Shell 16 304 241 2308
Perl 19 263 165 2167
Haskell 17 328 99 1951
MATLAB 25 176 114 1880
Javascript 23 263 145 1873
PHP 17 242 118 1791
Scala 16 219 113 1731
Lua 18 227 87 1715
Python 18 218 143 1393
Ruby 17 165 87 1219
OCaml 15 105 98 1154
Clojure 17 247 117 1011
CoffeeScript 17 193 126 989(Forth is missing)
the thread https://news.ycombinator.com/item?id=9121842 discusses this and gives reasons why some of the longer implementations could have been shorter.
---
http://spinroot.com/gerard/pdf/P10.pdf rule #5 has an interesting suggestion in the context of syntax for writing assertions, where 'e' is an assertion condition (a boolean expression): "The syntax #e turns the assertion condition e into a string that is printed as part of the error message."
eg you should be able to pass around 'expressions' (which are raw ASTs, which can contain unbound variables, not necessarily written as anonymous functions with an explicit varspec), and you should be able to serialize these into strings
---
one Java metaprogramming interface in Java core:
http://docs.oracle.com/javase/7/docs/api/javax/annotation/processing/Processor.html
---
this guy claims macros are not a readability problem:
orthecreedence 2 hours ago
After programming in CL for several years, I can honestly say that when encountering new syntax that people have introduced in an app, it's no harder to follow along with it by reading the macro definition than encountering an unknown function. And when in doubt, you can just macroexpand the syntax and see what it's doing under the hood.
reply
--
kevin_thibedeau 8 hours ago
I recently needed to make a cross platform assembler for a niche 8-bit micro that only has DOS tooling. After cloning it in a small bit of Python I was looking for a simple way to extend it with macros. It turns out that m4 works really well as an asm macro language and I now have advanced facilities like a code generating expression evaluator and high level control flow statements without any significant change to the core assembler. It blows the doors off what contemporary assemblers (nasm, masm, tasm) can do and hearkens back to the high point of high level assemblers when they peaked in the 70s.
reply
---
make certain vars 'fair game' for (anaphoric/non-hygenic) macros? eg x, y, z, x1, x2, etc? or mb something more universal, such as 'anything with a _ suffix' or 'anything ending in X'?
---
extend_syntax eg
http://www.cs.rit.edu/~rpj/courses/plc/lab4/lab4df.html?lab47.html#Q12?lab4ans.html#A12
---
http://scienceblogs.com/goodmath/2007/12/17/macros-why-theyre-evil/ has a good example of scheme define-syntax/syntax-rules, it looks quite good:
(define-syntax replace (syntax-rules (initially with until just-before) ((replace initially <value> with <newvalue> until <done>) (let loop (( <value>)) (if <done> (loop <newvalue>)))) ((replace initially <value> with <newvalue> until just-before <done>) (let loop ((old #f) ( <value>)) (if <done> old (loop <newvalue>)))) ((replace <var1> <var2> initially <value1> <value2> with <newvalue1> <newvalue2> until <done>) (let loop ((<var1> <value1>) (<var2> <value2>)) (if <done> (list <var1> <var2>) (loop <newvalue1> <newvalue2>))))))
---
bentcorner 4 hours ago
I find it frustrating that languages features work actively against you when you're trying to understand something.
Wide inheritance and macro usage are probably the worst. Good naming can aid understanding, but basic things like searchability are harmed by this.
Of those two, macros are the most trouble. You can't take anything for granted, and must look at every expression with an eye for detail. Taking notes becomes essential.
reply
---
http://andymeneely.github.io/squib/
a Ruby DSL for prototyping card and board games
discussion: https://news.ycombinator.com/item?id=9796708
(a quick skim saw nothing in the HN discussion relating to programming language/DSL design)
JSR 308 type annotations:
"What are type annotations?
The Java 6 annotation syntax is useful but limited. The Type Annotations syntax permits annotations to be written in more places, such as generic type arguments: List<@NonNull? Object>. jsr Programmers can use type annotations to write more informative types, and then tools such as type-checkers can detect and prevent more errors. " -- http://types.cs.washington.edu/jsr308/
http://types.cs.washington.edu/jsr308/specification/java-annotation-design.html
---
"
Annotations
Most projects eventually need to interact with the programmer. You’ll eventually wish for annotations: some way to convey extra information from the program to your LLVM pass. There are several ways to build up annotation systems:
The practical and hacky way is to use magic functions. Declare some empty functions with special, probably-unique names in a header file. Include that file in your source and call those do-nothing functions. Then, in your pass, look for CallInst instructions that invoke your functions and use them to trigger your magic. For example, you might use calls like __enable_instrumentation() and __disable_instrumentation() to let the program confine your code-munging to specific regions.
If you need to let programmers add markers to function or variable declarations, Clang’s __attribute__((annotate("foo"))) syntax will emit metadata with an arbitrary string that you can process in your pass. Brandon Holt again has some background on this technique. If you need to mark expressions instead of declarations, the undocumented and sadly limited __builtin_annotation(e, "foo") intrinsic might work.
You can jump in full dingle and modify Clang itself to interpret your new syntax. I don’t recommend this.
If you need to annotate types—and I believe people often do, even if they don’t realize it—I’m developing a system called Quala. It patches Clang to support custom type qualifiers and pluggable type systems, à la JSR-308 for Java. Let me know if you’re interested in collaborating on this project!I hope to expand on some of these techniques in future posts. " -- http://adriansampson.net/blog/llvm.html
---
mb read:
---
(at least) two kinds of 'macros':
'reader macros' applied to string text before (or during) the transformation to AST
ordinary lisp-style macros, which are transforms of AST->AST
---
mike_hearn 23 days ago
| parent | on: Ask HN: Has anyone here programmed in Kotlin? What... |
Do you actually know Kotlin? It has many features for DSL support, including operator overloading (though not the ability to invent random new operators fortunately), infix notation, a form of exhaustive pattern matching, extension functions, annotation processing, inlineable anonymous extension functions which sounds bizarre but actually is key to defining many DSLs in the language, and so on. It's a different approach to Scala but to say there's no way to do DSLs isn't right.
As an example, check out funKtionale. It's a library that adds various functional features like function currying and partial application to Kotlin, entirely by using its DSL features.
pron 22 days ago
Or our Kotlin actor API: http://blog.paralleluniverse.co/2015/06/04/quasar-kotlin/
---
compiler warning (or scripting level requirement) if anything is REdefined with a macro (esp. if it's a language-level or stdlib-level thing)
---
so {} are like "quote". is "!" like "unquote" or is it "pass on the sideeffects"? in some sense i guess sideeffects are themselves "quoted" in a functional reactive stream. preefix vs postfix "!" to apply to the thing itslf, or to its first return value? what about impure reads (not sideeffectful)? what about sideeffecttful reads, the rest of the stream cannot be produced until these are executed against some environment? hmm.. "!" as meaning "excute against the current environment"; mb "f!e" for "exec f against environ e"; exec means not just reading from the environ, but also writing to it.. environ can "catch" these writes (and reads!) if it wants, in order to produce a functional reeactive stream instead ! =s like eval, not anttiquote; and this is not quite like quote/eval b/c blocks are opaque, they are not ASTs (and not source code text strings) that are preserved until runtime (or th end of this 'stage' at least, in the case of g compiletime macros) for us in metaprogramming
3 levels/stages: source cod text string, AST, opaque block/fn text --(parse (lex and parse))--> AST --compile--->block--eval/exec-->result and side effects (mb 'eval' to a stream and 'exec' to adtually do it? wher 'eval' is just 'exec' to an environ that catches eeverything?)
---
jerf 14 hours ago
... Haskell's great support for domain-specific languages. This is both in the sense of languages that are really "just" Haskell APIs but work in such an different manner that they are quite different, and in the sense of writing a full compiler [1] (or any subset of a compiler) for a new language.
For an example of the former, one of my favorite demonstrations is the "probability monad", in which one ends up with a sort of sublanguage inside of Haskell that is probability aware: http://www.randomhacks.net/2007/02/22/bayes-rule-and-drug-te... [2] If you want to start to really grasp the power of Haskell, you could do worse than meditate on how you'd implement that in your $FAVORITE_LANGUAGE... and then, most likely, burst into tears when someone shows you just how much of Haskell still works, unchanged, even in that environment (e.g., everything in Control.Monad), rather than having to be implemented in a parallel type hierarchy (or, put another way, producing a new "color" of functions [3]).
While I doubt they use something exactly like that, it's not hard to see how that sort of thing could make it fairly easy to write spam classification code with a lot of the basic plumbing abstracted away entirely.
There's a lot of examples of the latter, but I'm not sure of a really solid open source example to point at. I know it's a common move in industry, though.
For bonus points, these things compose together just fine; you could easily take the probability monad and build a new language on top of it.
[1]: And I do mean real compiler, by anybody's definition, all the way down to assembler if you like. Haskell's a pretty good base language for that sort thing.
[2]: Or see http://blog.plover.com/prog/haskell/probmonad-refs.html http://blog.plover.com/prog/haskell/probmonad-refs.html
[3]: http://journal.stuffwithstuff.com/2015/02/01/what-color-is-y... http://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-function/
---
cody http://docs.quantifiedcode.com/patterns/index.html
---
great example of a DSL that we'd want to enable:
http://daveyarwood.github.io/alda/2015/09/05/alda-a-manifesto-and-gentle-introduction/
the discussion mentions other more complex music languages:
https://news.ycombinator.com/item?id=10177716
including
MML, Lilypond, ChucK?, Csound, Supercollider, Max/MSP, PD, Faust, Overtone (Supercollider + Clojure) and if you want less "note C" and more abstract: Music-Suite ( http://music-suite.github.io/docs/ref ) and Overtone ( http://overtone.github.io ), along with the Leipzig library, which adds functional programming abstractions for use with Overtone ( https://github.com/ctford/leipzig )
---
note: i looked in Rust to see what representation they express their borrow checker in. I think this sort of thing would be in the source code subdirectory a common standard for program evaluation is 95% confidence with a sampling error of ± 5%. In English that means that you believe that 95 percent of the time the results from your sample, would be off by no more than 5% as compared to the results you would have gotten if you had collected data from everyone. (librustc is the main compiler, and i think 'middle' is the stuff that is neither frontend (text -> AST) nor backend (-> target code)).
It appears to just be in ordinary Rust.
The most mentioned seem to be MML, Lilypond, Overtone, ChucK?, Csound.
---
'.' is like a macro but it actually changes the case of its second argument before evaluating it, eg employeeRecord.address == (employeeRecord ADDRESS). Do we allow users to write things like that? Seems like we should...
---
i'm on board with A rationale for semantically enhanced library languages by Bjarne Stroustrup: "..a dialect created by supersetting a language using a library and then subsetting the result...when augmented by a library, a general-purpose language can be about as expressive as a special-purpose language and by subsetting that extended language, a tool can provide about as good semantic guarantees. Such guarantees can be used to provide better code, better representations, and more sophisticated transformations than would be possible for the full base language."
i'd like to build some of that into oot:
This fits in with the ideas of/that:
---
note: i looked in Rust to see what representation they express their borrow checker in. I think this sort of thing would be in the source code subdirectory https://github.com/rust-lang/rust/tree/master/src/librustc/middle (librustc is the main compiler, and i think 'middle' is the stuff that is neither frontend (text -> AST) nor backend (-> target code)).
It appears to just be in ordinary Rust.
---
Stroustrup pushes us to consider more intrusive things that metaprogrammy library writers might want to do:
"Consider a sim- ple, common, and useful example:
A=k*B+C
... Essentially all languages can handle A=k*B+C when the vari- ables denote scalar values, such as integers and floating point numbers. For vectors and matrices, things get more diffi- cult for a general-purpose language (that doesn’t have built- in vector and matrix types) because people who write that kind of code expect performance that can be achieved only if we do not introduce temporary variables for k*B and k*B+C . We probably also need loop fusion (that is doing the ele- ment * , +, and = operations in a minimal number of loops). When the matrices and vectors are sparse or we want to take advantage of known properties of the vectors (e.g. B is upper-triangular), the library code needed to make such code work pushes modern general purpose language to their limit [17, 23] or beyond — most mainstream languages can’t effi- ciently handle that last example. Move further and require the computation of A=k*B+C for large vectors and matrices to be computed in parallel on hundreds of processors. Now, even an advanced library requires the support of a non-trivial run-time support system [1]. We can go further still and take advantage of semantic properties of operations, such as “re- membering” that C was the result of an operation that leaves all its elements identical. Then, we can use much simpler add operation that doesn’t involve reading all the elements of C . For other examples, preceding the numerical calculation with a symbolic evaluation phase, say doing a symbolic differentiation, can lead to immense improvements in accuracy and performance. Here, we leave the domain where libraries have been considered useful. Reasoning like that and exam- ples like that (and many more realistic ones) have led to the creation of a host of special-purpose languages for various forms of scientific calculation." -- http://www.stroustrup.com/SELLrationale.pdf
the things he is talking about would seem to require not just macros but also:
this goes beyond simply having a macro for 'matrix multiply' (we need to match code patterns, not just expand explicitly marked points in code); then it goes beyond mere AST matching (we need to do static analysis to catch if all of the elements of C are identical, or if C is guaranteed to be diagonal); then it goes beyond adding another optimization pass (we need to alter the runtime to support parallelization); and i don't even know offhand what is needed for the symbolic evaluation phase. And, we want to do all of this 'within the language' rather than just providing hooks in the compiler implementation to add a call to some external separately compiled binary executable or shared library.
---
types of analysis:
static analysis:
runtime analysis:
---
---
Forth has an interesting convention to document what stack manipulation functions do (in the parentheses; stuff in parens is comment in Forth); eg:
dup ( a -- a a ) ?dup ( a -- a a
| 0 ) dup if dup then ; |
three notes:
---
dsls need to inject functions that (a) dont need a namespace prefix, (b) have access to 'hidden' semiglobals. eg metattrader's Bid, eg quantopian
---
using only Oot metaprograming, we should be able to add the following two features to the language:
---
we don't ever want anything in Oot (except compiler flags, and perhaps a few very special metaprogrammy compiler overrides) to globally affect a program. So no Haskell like 'only one instance of this typeclass anywhere', no C overrides, no 'open classes' that can be overridden from anywhere and that persist and globally change the way core classes work for everyone
however, we might want to allow all sorts of metaprogramming scoped to:
note that file-scoped C-like #defines would be okay, as this is a lexical scope, so these things would still be trouble: https://gist.github.com/aras-p/6224951
now, allowing arbitrary metaprogramming in any dynamic scope makes it hard for anything to be efficient, because the compiler can't really do any optimization before runtime except when it can prove that the constructs it is optimizing can't possibly lie within a dynamic scope that tries to metaprogram those constructs. So maybe don't do that, or maybe allow lexical scopes to disallow runtime metaprogramming (or maybe do that upon module load, so the compiler can make 2 versions of each module, one of them statically optimized, one of them open for runtime metaprogrammingy modification), or maybe just bite the bullet and allow it.
so, we want to make it possible for the compiler to:
(suggested by [1])
---
some various metaprogrammingish things in Ruby, which doesnt have macros:
http://stackoverflow.com/questions/17110117/macros-in-ruby
Q:
how to:
define ARGS 1,2 sum(ARGS) # returns 3
A's:
A = [1, 2] A.inject(:+) # => 3
args = [1, 2] sum(*args) # equivalent to sum( 1, 2 )
Q:
EDIT: More specifically my problem looks more like:
@button1 = FXButton.new(self, "Button 1",:opts => BUTTONPROPERTIES,:width => width, :height => height) @button2 = FXButton.new(self, "Button 2",:opts => BUTTONPROPERTIES,:width => width, :height => height) @button3 = FXButton.new(self, "Button 3",:opts => BUTTONPROPERTIES,:width => width, :height => height)
And ideally I'd want the code to look like:
@button1 = FXButton.new(self, "Button 1", ALLBUTTONPROPERTIES) @button2 = FXButton.new(self, "Button 2", ALLBUTTONPROPERTIES) @button3 = FXButton.new(self, "Button 3", ALLBUTTONPROPERTIES)
A:
ALLBUTTONPROPERTIES = ->{{opts: => BUTTONPROPERTIES, width: width, height: height}}
and within a context where the constants and variables BUTTONPROPERTIES, width, height are assigned some value, do this:
@button1 = FXButton.new(self, "Button 1", ALLBUTTONPROPERTIES.call) @button2 = FXButton.new(self, "Button 2", ALLBUTTONPROPERTIES.call) @button3 = FXButton.new(self, "Button 3", ALLBUTTONPROPERTIES.call)
---
they call this a 'macro' but i don't think it should be called that, still, is interesting:
" An implementation of attr_reader in pure Ruby
The attr_reader function should be familiar to everyone with Ruby programming experience. Implementing attr_reader is a perfect example of what are viewed as macros in Ruby, which are implemented using Ruby's metaprogramming capabilities.
But we're going to call it rm_attr_reader to avoid collision with Ruby's standard library implementation of attr_reader. (The rm_ namespaces our new macros; "rubymacros" after all.)
class Module # Define our macro here... def rm_attr_reader(name) define_method(name) do instance_variable_get("@#{name}") end end end
class Foo # And call our macro here... rm_attr_reader :bar def initialize(arg) @bar = arg end end
f = Foo.new "quux" f.bar # => "quux"
...
Parse tree substitution
The previous two above, essentially, are variations on string substitution. A "true" macro (true in the Lisp sense) can't be implemented in Ruby without extending Ruby syntax, and having (library) code capable of parsing the extended syntax. The next example uses the rubymacros gem:
macro rm_attr_reader name :( def ^name ^(RedParse::VarNode?["@#{name}"]) end ) end
(The syntax highlighter doesn't highlight this correctly because macro is not a reserved word in Ruby. Try to read it as kind of like a method. :( … ) and unary ^ constructs are also part of the new syntactical magic that rubymacros adds. You should see them as similar to double-quoted strings and the string interpolations (#{…}) within them.)
The last three versions ((we left out 2 of the 3 here)) of our new version of attr_reader share some common characteristics:
in all cases, an accessor method is defined within a quoted construct.
somehow that quoted method is interpolated into regular code so it can be executed.
the quoted method definition contains holes that get filled in by parameters that are passed in at the time of interpolation.
the quoted code and the values placed in the holes that fill it are data structures of some sort – either strings or parse trees. But they aren't code yet… they're things that will turn into code. They're… meta-code.The advantage of having data structures that represent nascent code is that they can be manipulated like any other data before you turn them into code.
The first version is also broadly similar to this approach. But instead of quoted code with holes in it, it uses magical interpreter methods to accomplish the same thing. Even with this first version, the name of the method and the instance variable are represented as strings which can be manipulated. "
---
https://en.wikipedia.org/wiki/General-purpose_macro_processor
GEMA was mentioned in passing in http://rubymacros.com/
---
todo:
---
" It is not a big deal to provide any kind of syntax on top of Lisp. In fact, much easier than with Ruby, thanks to the reader macros. – SK-logic Jun 3 '11 at 14:22
@SK-logic: True. However Lispers shy away from reader macros because once you go down that route it no longer feels like a Lisp. In fact some variants of Lisp, such as Clojure, reserve reader macros for the language designer. – btilly Jun 3 '11 at 16:38 "
---
" This bogus claim always comes up in discussions about powerful macro systems from people that haven't really used languages with powerful macro systems. Look at PLT Scheme to see the things that have been done that go far beyond lazy evaluation. OCaml's CamlP?