proj-oot-ootMetaprogrammingNotes4

oldiob on April 14, 2020 [–]

Is the committee planning on working on the preprocessor? I don't see any reason for not boosting it. It's time for C to have real meta-programming. Would be nice to have local macros that are scoped.

On another note:

Finally, stop putting C++ craps into C.

---

dmit on March 26, 2020 [–]

One cool application that cropped up in the Rust ecosystem is to use WASM for procedural macros. That way not only can they be distributed as precompiled (arch-agnostic) blobs from crates.io, but you also get all the benefits of the sandbox environment to guarantee freedom from malicious side effects. The W in "WASM" doesn't have to limit possible applications to just the web browser.

https://github.com/dtolnay/watt

---

19 icefox 26 days ago

link

---

random course on pl, esp. DSL implementation:

https://docs.google.com/document/d/e/2PACX-1vQfrlnqfBqsS2_lWSZKM_4PHlH-HvXv2gGSkxGCeLn3hgH03nJIZ2i7Ew6G5YSL8XgNmHxQbOeH7H6s/pub#h.nwy1x94v7bi3

---

a good intro to Forth metaprogramming stuff is in

http://galileo.phys.virginia.edu/classes/551.jvn.fall01/primer.htm#create section "11. CREATE ... DOES> (the pearl of FORTH)"

you should learn 'CREATE', 'DOES>', ',', '[', ']', 'IMMEDIATE' (IMMEDIATE is defined earlier, in section '4. Extending the dictionary')

---

i think i'm beginning to understand "Forth is one of the two quintessential extensible languages, the other one being Lisp" [1].

---

Starting Forth calls words like CONSTANT 'defining' words, and says that they which 'compile' new words. E.g. "80 CONSTANT MARGIN" is said to "compile" the new word MARGIN, adding it to the dictionary; when MARGIN is executed at run-time, we execute the behavior that CONSTANT compiled for it (confusingly, the convention is to call that behavior 'the run-time behavior of CONSTANT'). Starting Forth says that "Special structures like conditionals and loops are not compiled by the compiler but by the words being compiled (IF, DO, etc.)...When you’ve got an extensible compiler, you’ve got a very powerful language!" [3]

---

[4]

---

        .

5 sjamaan 2 months ago

link

Cool to see that hygienic macros are finally gaining in popularity! The syntax of new macro definition looks quite readable, too.

    5
    slightknack 2 months ago | link | 

Hygienic macros are the bomb! We’re planning to generalize the macro system to unify it with compile-time evaluation (while still preserving hygiene, using a token-tree-based macro system, like Rust). This lets us implement syntax macros (as they are now) in terms of the more powerful system.

-- [5]

---

    36
    It's Time to Get Hyped About Const Generics in Rust plt programming rust nora.codes
    authored by mtset 33 hours ago | suggest | flag | hide (hidden by 3 users) | save | cached | 8 comments
    Markdown formatting available

~ joed 29 hours ago

link flag

Does this mean type-checking in rust is about to become turing complete?

    6
    videogame_hacker 28 hours ago | link | flag | 

It already was, but there’s a stack limit

5 matklad 26 hours ago

link flag

Practically, procedural macros are in every way worse for compilers.

---

https://dev.to/baweaver/future-of-ruby-ast-tooling-9i1

---

"The clojure.tools.analyzer library does an excellent job of providing a data-oriented representation of an AST of Clojure code with static analysis annotations" [6]

---

~ jonahx 1 hour ago

link flag

I think I agreed with every single one of these except the pro-mobbing take.

    Sophisticated DSLs with special syntax are probably a dead-end. Ruby and Scala both leaned hard into this and neither got it to catch on.

As someone who’s worked with ruby more than 10 years, couldn’t agree more here. rspec is the conspicuous remnant of this delirium and is an unequivocal mistake.

-- https://lobste.rs/s/brl7ez/uncomfortable_truths_software

---

reply

TheSoftwareGuy? 1 day ago

root parent prev next [–]

Rust has an extremely powerful macro system, have you tried that?

reply

alpaca128 1 day ago

root parent next [–]

Rust macros are one of the more annoying features to me. They're great at first glance but whenever I want to build more fancy ones I constantly bump into limitations. For example they seem to be parsed without any lookahead, making it difficult to push beyond the typical function call syntax without getting compiler errors due to ambiguity.

reply

inferiorhuman 1 day ago

root parent next [–]

Procedural macros have a peek function from the syn crate. macro_rules macros can stuff this into the pattern matching.

e.g.

https://turreta.com/2019/12/24/pattern-matching-declarative-...

reply

colejohnson66 7 hours ago

root parent next [–]

But proc macros are limited by requiring another crate (unless things have changed in the last year). Sure, it’s just one extra crate in the project, but why must I be forced to?

reply

xigoi 19 hours ago

root parent prev next [–]

But there's the weird limitation that procedural macros have to be in a separate crate.

reply

efaref 19 hours ago

root parent next [–]

Why is that weird? Procedural macros are compiler plugins. They get compiled for the platform you're building on, not the one you're building for, and so they need to be a separate compilation unit. In Rust, the crate is the compilation unit.

reply

xigoi 18 hours ago

root parent next [–]

Because you can't just throw together a simple procedural macro to use in a specific project, as you can in other languages.

reply

---

https://news.ycombinator.com/item?id=14167620

LightMachine? on April 22, 2017

parent context prev next [–]on: Type Systems as Macros

eli5?

chriswarbo on April 22, 2017

parent next [–]

Macros are run at compile time, and are used to rewrite source code prior to interpretation or compilation. They're mostly used to make nicer syntax, e.g. a macro could turn an expression like `(with-file (f "/tmp/foo") ...)` into a working implementation like:

    (let* ((f (open "/tmp/foo"))
           (result ...))
      (close f)
      result)

Here, the authors make macros for type annotations, i.e. they take expressions like `(my-value : my-type)`, check whether the types unify correctly, and spit out an untyped expression implementing that value if they do.

By using macros, the type system becomes modular: new type system features (subtyping, kinds, etc.) can be written by the programmer as a set of macros, rather than as a whole new language (and all of the work that entails). They test their claim by implementing a bunch of features, combining/reusing them in a bunch of mini languages, and write some semi-plausible programs in these languages, to see if they're realistic.

stchang on April 26, 2017

... Typed Racket uses macro expansion to translate its typed surface language into a typed core language, and then type checks that core language.
parent next [–]

This approach works well because the surface and core languages are similar, and thus the type checker need only handle a small number of core forms.

This approach is limited, however, to constructs that are translatable into the core typed language. For example, a few Racket `for/X` comprehension forms are not supported in Typed Racket because they are difficult to translate.

Our approach alternatively uses macro expansion to type check the surface language and translates it into an untyped core language. Thus it's less limited in the typed languages one may implement. The tradeoff is that the programmer must implement type rules for all forms in the surface language, rather than just the core language.

juliangamble on April 23, 2017

prev next [–]

How powerful is the type system? Can you do dependent types?

stchang on April 26, 2017

parent next [–]

> Can you do dependent types?

Yes. For example, see https://github.com/wilbowma/cur

---

passerine does something like what we want to do in terms of defining few primitives, the using macros in a prelude:

https://lobste.rs/s/fvaycc/passerine_programming_language#c_13ifys

---

"Rust...proc. macros even cannot manipulate an abstract syntax tree, because syn::Item, a common structure used to write proc. macros, is indeed known as a concrete syntax tree, or “parse tree”... Ideally, a programming language should have either no macros or only a lightweight form of syntax rewriting rules (like Scheme’s extend-syntax or syntax extensions of Idris), in order to keep the language consistent and well-suited to solve expected tasks." -- [7]

---

---

https://users.cs.northwestern.edu/~robby/pubs/papers/jfp2012-fcdf.pdf Macros that Work Together

---

https://handlebarsjs.com/

---

" I largely don’t want these features that metaprogramming generalizes. Not in the compiler, and not as a library. The one that I do is type constraints – which is very difficult to implement as a library, because it implies attaching extra data to a type when type checking – but it’s simple to implement within the compiler with a bit and, once the extra data is part of the type. (Simple, as in it adds about 20 lines to the type checker, last I remember). " [9]

--- " ShadowRealms? – an ECMAScript proposal for a better eval() ... Each instance has its own global JavaScript? scope. Code is evaluated in that scope. If it changes global data, that only affects the ShadowRealm?, but not the real global data. " -- [10]

---

https://jack.wrenn.fyi/blog/deflect/ https://news.ycombinator.com/item?id=34001435

"Today, I’m releasing deflect, an implementation of reflection for Rust. Deflect can be used to recover the concrete types of trait objects, inspect the internal state of async generators, pretty-print arbitrary data, and much more."

kp995 23 hours ago

prev next [–]

Can’t we rely more on Rust’s Pattern Matching and it’s strong type system?

Reflection seems more helpful when the programming language is little unsounded.

reply

jswrenn 22 hours ago

parent next [–]

Absolutely! That's the approach that frunk [0] takes. Frunk (and other reflection libraries like it) are suitable for most use cases, and make better use of Rust's affordances.

My crate is suitable for cases where you cannot know (or control) the set of types you might need to reflect on in advance. It's primary use-cases are related to debugging.

[0]: https://docs.rs/frunk

reply

halfmatthalfcat 22 hours ago

root parent next [–]

Is Frunk Rust's Shapeless (from Scala)?

reply

jswrenn 21 hours ago

root parent next [–]

Yep!

reply

---

https://stackoverflow.com/questions/3704372/how-does-clojures-syntax-quote-work?rq=1

---

https://clojuredocs.org/clojure.core/spit https://clojuredocs.org/clojure.core/slurp

---

one of the top answers on https://stackoverflow.com/questions/8920137/clojure-caret-as-a-symbol is interesting:

""" ^ is "the meta character" it tells the reader to add the symbol starting with ^ as metadata to the next symbol (provided it is something that implements IMetas)

user=> (def x ^:IamMeta? [1 2 3])

  1. 'user/x user=> x [1 2 3] user=> (meta x) {:tag :IamMeta?} user=>

You can learn a lot about how clojure works under the hood by looking at the meta of things, for instance functions:

user=> (meta foo) {:ns #<Namespace user>, :name foo, :file "NO_SOURCE_PATH", :line 5, :arglists ([s])}

this is very often used for type hints

(defn foo [^String s] (.charAt s 1))

it is generally a good idea to turn on reflection warnings (set! *warn-on-reflection* true) and then add type hints until the warnings go away. without these Clojure will look up the type of the function operands at run-time, which saves you the trouble of fussing with types though at a slight cost.

PS: My next favorite reader character is the "dispatch" character #, it is well worth learning about it next :)

PPS: this is different in clojure 1.2.x vs clojure 1.3.x in Clojure 1.2.1 metadata does not compose when you use the meta-character:

user=> (def foo ^:foo ^:bar [1 2 3])

  1. 'user/foo user=> (meta foo) {:tag :foo}

and in 1.3 it "does the right thing" and also keywords are options instead of "tags":

user=> (def foo ^:foo ^:bar [1 2 3])

  1. 'user/foo user=> (meta foo) {:foo true, :bar true} """

---

https://bluishcoder.co.nz/2007/11/28/embedded-grammars-in-factor.html

---

https://lobste.rs/s/kbss1k/i_can_t_believe_i_m_praising_tcl_2008#c_jyssyq

    7
    Sietsebb 4 years ago | link
    I hadn’t done much/anything with it before today, but my work today has made me like it a lot. Writing bare words instead of quoted strings improves (interactive) usability far more than it ought to. I always suspected as much, because R’s wonderful tidyverse does a similar thing; but I never dared to trust that gut feeling. Now that I’ve experienced it in the tidyverse, the shell, and Tcl I’m pretty sure that feature is just that nice (for me).
    (For an example of unquoted strings (actually column names) in the tidyverse, see this analysis of the diamonds dataset. It’s the code block that starts with library(modelr), just below the scatter plot of Old Faithful data. Fantastic for exploratory data analysis.)

https://r4ds.had.co.nz/exploratory-data-analysis.html#patterns-and-models

library(modelr)

mod <- lm(log(price) ~ log(carat), data = diamonds)

diamonds2 <- diamonds %>% add_residuals(mod) %>% mutate(resid = exp(resid))

ggplot(data = diamonds2) + geom_point(mapping = aes(x = carat, y = resid))

---

on tcl:

" smrtinsert 4 months ago

parent prev next [–]

The real power was unlocked with uplevel, upvar and switch regexp for a variety of reasons. I miss sane language extensibility. It was meh in clojure by comparison and too much in scala. " -- https://news.ycombinator.com/item?id=35834844

"`uplevel` permitted modifying the calling environment, enabling injection of locally scoped dsls." -- https://news.ycombinator.com/item?id=35812063

---

so with the addition of tcl, we have a few "small metaprogrammable" languages:

lisp (macros; Racket is the exemplar), forth, tcl

then you have things like ruby's method missing, c++ templates, python's metaclasses, C preprocessors

this guy mentions M4: https://news.ycombinator.com/item?id=30329516

---

this is another post by that guy that was (accidentally?) uninvited to headline rustconf and was upset about that:

[11]

interestingly, graydon2 says that Rust used to have a system sort of like that, and he would prefer it: https://graydon2.dreamwidth.org/307291.html

~ ssokolow 16 hours ago

link flag
    While creating such data structures is pretty straightforward in Zig, creating any of these examples in Rust using proc macros is basically impossible - the reason being that proc macros don’t have access to type information like size or alignment. While you could have a proc macro generate a const fn that computes the clusters for a particular enum, this function cannot be used to specify the length of an array for a generic type.

As a Python expat who doesn’t trust himself to write memory-unsafe code, all I can say is “here’s hoping people qualified to solve these problems in Rust are paying attention”.

    15
    riking 14 hours ago | link | flag | 
            This is the kind of work that the RustConf fiasco’s talk subject would have enabled: instead, we made sure syn/quote/serde remain dominant in the proc macro space, instead of creating a whole new reflection mechanism that would need new libraries.

---