notes-computer-programming-programmingLanguageDesign-prosAndCons-ffi

I've never met a FFI I didn't come to loathe. I've had java, ruby, and python FFI libraries fail to satisfy my needs despite half a dozen bugfixes between them. What this man says about having to write wrappers, despite abundant and loud promises to the contrary, is completely true. You don't have to wander far off the beaten path before a typical FFI goes belly-up. POD structs usually suffice (Sure, we support POD structs! Oh, you want to nest them / align them / make arrays of them / have them hold pointers / ...? We don't support that "yet". Worse: they support it but it's buggy.). Heavens help you if your argument has (gasp) an initializer or one of the arguments is a reference. Maybe things have changed in the last ~5 years, but I doubt it.

If Julia's intimate connection with LLVM makes it practical to implement a better FFI or hybridize FFI + wrapper code when necessary, it will have a very valuable advantage over python for purposes of scientific computing. Maybe even enough to displace it in the long run.

EDIT: By "hybridize" I mean that the ability to embed asm,C,C++ in Julia with the same ease that you can embed asm in C/C++ would be a KILLER feature.

reply

simonster 3 days ago

link

That's in the works: https://github.com/JuliaLang/julia/pull/5046 for inline LLVM IR in Julia, with potential future extension to other languages.

reply

digikata 3 days ago

link

> I've never met a FFI I didn't come to loathe

I've felt that pain. With so many new programming languages popping up, I've been wondering if the next killer programming improvement isn't strictly a programming language at all, but rather something that rethinks the linker, manages execution, and facilitates interfaces between larger blocks of code (maybe in multiple languages).

reply

jjoonathan 2 days ago

link

I've heard many very smart people (professors, leaders of large HPC efforts, "language geeks") express the same sentiment over the years. Unfortunately, it's a btch of a problem, as attested to by the veritable graveyard of half-baked solutions out there. Apple's "Bridge Support" is the closest thing I've seen to success (haven't worked with MS's CIL) and it leaves much to be desired.

I'm not sure there is any way around the impedance mismatches between languages. They're all slightly different for a reason. Perhaps clean C APIs are the best we can hope for.

reply

digikata 2 days ago

link

I keep hoping that some accessibility upgrade comes to the linker akin to how LLVM made compiler infrastructure more accessible, might make FFI bridging more automated. For example, if GDB can access and take apart C structs an automated way, why is FFI coding always seem to have programmers do a lot of mechanical interfacing work.

Not that all the other JVM, CIL, etc aren't approaches that yield some improvements, but something that upgrades capability or accessibility at the level of the ABI linkage level is going to be a wider impact.

reply

pjmlp 2 days ago

link

> I'm not sure there is any way around the impedance mismatches between languages.

CLR has an approach for that by defining what is known as CLS, Common Language Subsystem.

Likewise they have a similar approach on WinRT?, known as Type Providers.

However they also have their impedance mismatches, as you are only allowed to use types that are usable by all languages that target such runtimes.

The benefit is that they still allow for an higher level of code abstractions than pure C functions.

reply

Toenex 2 days ago

link

Yep me too. People are too quick to dismiss the cross language issues which can in my experience can slowly erode the benefit. Typically you use an FFI to access particular functionality offered in libraries/classes only offered in another language. In my experience it is only a temporary solution, a does-it-work test. After that you need to recode. Having said that, in my experience Lua(JIT) does this pretty well with C libraries but Lua was designed as the scripting companion to C from day one.

The JVM offers a better place to tackle these issues. Scala and Java cross calling is often more than adequate. Perhaps a Julia compiler for the JVM could be a step in the right direction.

reply

--

sdegutis 3 days ago

link

Wait a minute! Can you embed Julia into a C program like Lua? Can it interface with complex C types cleanly?? This might be the scripting language I'v been looking for in my side project!

reply

sdegutis 3 days ago

link

Wow. Okay, yes. It can be embedded[1]. It can call C code[2].

Julia may have just saved my project (which was dying because it needed a good scripting language that was fast)!

[1]: http://docs.julialang.org/en/latest/manual/embedding/

[2]: http://docs.julialang.org/en/latest/manual/calling-c-and-for...

reply

3JPLW 3 days ago

link

And embedding (and its documentation) will likely get much better very soon.

https://github.com/JuliaLang/julia/pull/4997

reply

sdegutis 3 days ago

link

One big pain point so far is that it won't be easy to actually embed Julia into my app. So users of my app will have to install Julia (probably via homebrew) before they can script the app with it.

reply

--

astrieanna 3 days ago

link

Julia does not compile the C code; it links against shared libraries. It helps that Julia types can model C structs easily.

Manual entry on calling C: http://docs.julialang.org/en/latest/manual/calling-c-and-for...

Blog post on passing Juila callback functions to C code: http://julialang.org/blog/2013/05/callback/

reply

--

" import sys import ctypes pyint_p = ctypes.POINTER(ctypes.c_byte*sys.getsizeof(5)) five = ctypes.cast(id(5), pyint_p) print(2 + 2 == 5) # False five.contents[five.contents[:].index(5)] = 4 print(2 + 2 == 5) # True (must be sufficiently large values of 2 there...) " -- https://gist.github.com/Jach/1208215

--

https://news.ycombinator.com/item?id=7136604

Thrift, Protocol Buffers, Haris, D-Bus

--