proj-plbook-plChSciLangs

Difference between revision 34 and current revision

No diff available.

Table of Contents for Programming Languages: a survey

Languages for scientific, numerical, or mathematical computation

MATLAB and Octave

Tutorials:

Internals and implementations

Core data structures: todo

Number representations

Integers

Floating points todo

array representation

variable-length lists: todo

multidimensional arrays: todo

limits on sizes of the above

string representation

Representation of structures with fields

todo

R

features:

free tutorials:

books:

best practices and style guides:

libraries:

Retrospectives:

Opinions:

Lists and tours of libraries:

Gotchas:

is.list(x) to test if an object is actually a vector." [65]

Types [74]:

R internals

Julia

Julia tutorials

Julia features

"

Julia: Dynamism and Performance Reconciled by Design section 4.2.1 shows how multiple dispatch enables a library to compute the derivative of another program without modifying that other program to be aware of the derivative-computing library.

JuliaLang: The Ingredients for a Composable Programming Language

Julia retrospectives

Julia best practices

Julia opinions

    Performance issues including long startup time and JIT lags
    Somewhat obscure syntax and problems with interoperability with other languages
    Poor text formatting facilities in the language and lack of good unit testing frameworks
    Unsafe interface to native APIs by default
    Unnecessarily complicated codebase and insufficient attention to bug fixing

Despite all this, I think the language can find its niche as an open-source alternative to MATLAB because its syntax might be appealing to MATLAB users. I doubt it can seriously challenge Python as the de-facto standard for numerical computing." [85]

lenticular 42 days ago [-]

It's extremely expressive. Notably, Julia is homoiconic, with full lisp-style macros. It also has multiple dispatch, which is a far more general technique that OO single-dispatch. This makes it very easy to define modular interfaces that work much like statically-typed type classes in Haskell. This allows you, for example, to define a custom matrix type for your bespoke sparse matrix layout and have it work seamlessly with existing linear algebra types.

I've done a lot of work in both Python with Scipy/Numpy, and Julia. Python is painfully inexpressive in comparison. Not only this, but Julia has excellent type inference. Combined with the JIT, this makes it very fast. Inner numerical loops can be nearly as fast as C/Fortran.

Expanding on the macro system, this has allowed things like libraries that give easy support for GPU programming, fast automatic differentiation, seamless Python interop, etc.

tombert 42 days ago [-]

I'm not sure how I feel about multi-dispatch...I've had a few headaches chasing down problems with multimethods in Clojure...I'd have to try using Julia full-time to see how it feels.

I was unaware that Julia was homoiconic...I'm somewhat of a Lisp fanboy so I might need to give the language another chance.

lenticular 42 days ago [-]

There's pretty big differences in usage between multimethods in Clojure and Julia. I've used both a decent amount. All functions in Julia are multimethods by default. If you don't use type annotations, a new method will be generated whenever you call the function with new argument types. This explicit type specialization is a very important part of why Julia can have such a consistently fast JIT despite its dynamicity.

Errors from missing or conflicting methods tend to not happen much in practice.

https://docs.julialang.org/en/v0.7.0/manual/methods/

nicoburns 42 days ago [-]

> If you don't use type annotations, a new method will be generated whenever you call the function with new argument types.

Damn. That's a pretty clever trade off between dynamic and static types.

vanderZwan 42 days ago [-]

I vaguely recall a talk by Stefan Karpinsky where he mentions meeting one of the big names in compiler land (working on V8 or something) and they said Julia's JIT is just a lazily evaluated AOT compiler, and as a result much simpler than the JITs typically seen in other languages.

tombert 42 days ago [-]

Forgive a bit of ignorance here, but that doesn't sound terribly different than overloading functions in C++. Am I way off on that?

KenoFischer? 42 days ago [-]

The difference is run-time (semantically) vs compile time. If you had overloading at runtime in C++, you wouldn't need virtual functions or any of the thousands of OO "patterns" (visitor, factory, etc.), that are working around the lack of this capability. " -- [106] todo digest this

"

kgwgk 42 days ago [-]

> Julia is homoiconic

It depends on what you understand by homoiconic:

https://stackoverflow.com/questions/31733766/in-what-sense-a...

eigenspace 42 days ago [-]

This is why the language creators usually avoid using the word 'homoiconic' because every time one uses that word there's a finite probability of being bogged down in an incredibly uninteresting semantic argument.

Instead, people prefer to say that julia code is just another (tree-like) data-structure in the language and it can be manipulated at runtime with functions or compile time with macros or at parse time with string macros and now with Cassette.jl[1] we can even manipulate the form of code that has already been written and shipped by other packages all with first class metaprogamming tools. It seems to me that even if Julia is not 'truly homoiconic', that we seem to get the touted benefits of homoiconicity to the point that it seems like an unimportant distinction.

[1] https://github.com/jrevels/Cassette.jl

Athas 41 days ago [-]

Then why not just say 'Julia has macros'? Lightly perusing the description of Julia's features, that seems like a clear way of expressing what it is.

(I also vaguely recall Julia describing its type system as "dependent" in way that goes against convention. Maybe they just liked controversy in the early days!)

eigenspace 41 days ago [-]

What I just said is why Julia people don’t tend to call it homoiconic or dependantly typed. It lead to so many semantic arguments that most just talk about actual features such as macros and various multiple dispatch features instead of using words like homoiconicity and dependant typing.

zem 41 days ago [-]

"macros" are unfortunately used by C and lisp to describe two different things, and both usages are as widely popular as their parent languages (i.e. very). "

-- [107] todo digest this

Julia opinionated comparisons

Julia Internals and implementations

Core data structures: todo

Links:

Number representations

Integers

Floating points todo

array representation

variable-length lists: todo

multidimensional arrays: todo

limits on sizes of the above

string representation

Representation of structures with fields

Julia type system

Julia: dynamism and performance reconciled by design section 4.1 describes some aspects of Julia's type system, and references Julia Subtyping: A Rational Reconstruction for more detail.

Julia variants and variant implementations

GENESIS

LabView

Mathematica

"symbolic manipulation and solving"

Simulink

"continuous-time dynamic systems"

Modelica

"mechanical, electrical, etc systems"

Verilog-AMS

"analog and mixed-signal electronics"

Esterel

"reactive control systems"

SBOL

"synthetic biology systems"

Church

http://v1.probmods.org/

WebPPL

http://probmods.org/

Relay IR

https://docs.tvm.ai/langref/index.html

"Relay is a functional, differentiable programming language designed to be an expressive intermediate representation for machine learning systems. Relay supports algebraic data types, closures, control flow, and recursion, allowing it to directly represent more complex models than computation graph-based IRs can. Relay also includes a form of dependent typing using type relations in order to handle shape analysis for operators with complex requirements on argument shapes."

part of the TVM project

Weld IR

https://www.weld.rs/

"A Common Runtime for High Performance Data Analytics"

"Weld is a runtime for improving the performance of data-intensive applications. It optimizes across libraries and functions by expressing the core computations in libraries using a small common intermediate representation, similar to CUDA and OpenCL?...For example, for Spark, NumPy?, and TensorFlow?, porting over a few Weld operators can increase performance by up to 30x even on some simple workloads!"

https://www.weld.rs/docs/latest/weld/ast/index.html https://www.weld.rs/docs/latest/weld/ast/index.html#enums

https://cs.stanford.edu/~matei/papers/2017/cidr_weld.pdf

https://github.com/weld-project/weld/tree/master/python/grizzly/grizzly https://www.weld.rs/grizzly https://pypi.python.org/pypi/pygrizzly/0.0.1 Grizzly is a subset of the Pandas data analytics library integrated with Weld

https://www.weld.rs/weldnumpy WeldNumpy? is a subset of the NumPy? numerical computing framework integrated with Weld

"Split annotations are a system that allow annotating existing code to define how to split, pipeline, and parallelize it. They provide the optimization that we found was most impactful in Weld (keeping chunks of data in the CPU caches between function calls rather than scanning over the entire dataset), but they are significantly easier to integrate than Weld because they reuse existing library code rather than relying on a compiler IR. This also makes them easier to maintain and debug, which in turn improves their robustness. Libraries without full Weld support can fall back to split annotations when Weld is not supported, which will allow us to incrementally add Weld support based on feedback from users while still enabling some new optimizations." -- [137] https://github.com/weld-project/split-annotations https://shoumik.xyz/static/papers/mozart-sosp19final.pdf

TensorFlow MLIR

https://github.com/tensorflow/mlir/blob/master/g3doc/LangRef.md https://www.tensorflow.org/mlir/overview

https://llvm.org/devmtg/2019-04/slides/Tutorial-AminiVasilacheZinenko-MLIR.pdf

https://github.com/tensorflow/mlir/blob/master/g3doc/Dialects/Standard.md

Standard Types: "Standard types are a core set of dialect types that are defined in a builtin dialect and thus available to all users of MLIR."

"Instead of using phi nodes, MLIR uses a functional form of SSA where terminators pass values into block arguments defined by the successor block." [138]. See also [139]

Links:

VAST

https://github.com/trailofbits/vast

" VAST is a new compiler front/middle-end designed for program analysis. It transforms parsed C and C++ code, in the form of Clang ASTs, into a high-level MLIR dialect. The high level dialect is then progressively lowered all the way down to LLVM IR. This progression enables VAST to represent the code as a tower of IRs in multiple MLIR dialects. " -- [140]

SAL

Sal/Svm: An Assembly Language and Virtual Machine for Computing with Non-Enumerated Sets