notes-computer-programming-programmingLanguageDesign-prosAndCons-erlang

good post: http://www.unlimitednovelty.com/2011/07/trouble-with-erlang-or-erlang-is-ghetto.html

--- the ingredients of Erlang: " Fast process creation/destruction Ability to support » 10 000 concurrent processes with largely unchanged characteristics.

A programming model where processes are lightweight values -- and a good scheduler -- make concurrent programming much easier, in a similar way to garbage collection. It frees you from resource micro-management so you can spend more time reasoning about other things.

    Fast asynchronous message passing.

This is what ZeroMQ? gives you. But it gives you it in a form different to that of Erlang: in Erlang, processes are values and message-passing channels are anonymous; in ZeroMQ?, channels are values and processes are anonymous. ZeroMQ? is more like Go than Erlang. If you want the actor model (that Erlang is based on), you have to encode it in your language of choice, yourself.

    Copying message-passing semantics (share-nothing concurrency).

Notably, Erlang enforces this. In other languages, shared memory and the trap of using it (usually unwittingly) doesn't go away.

    Process monitoring.

Erlang comes with a substantial library, battle-tested over decades, for building highly concurrent, distributed, and fault-tolerant systems. Crucial to this is process monitoring -- notification of process termination. This allows sophisticated process management strategies; in particular, using supervisor hierarchies to firewall core parts of the system from more failure-prone parts of the system.

    Selective message reception.

" http://www.rabbitmq.com/blog/2011/06/30/zeromq-erlang/ (from http://www.zeromq.org/whitepapers:multithreading-magic ) --

davidw 11 hours ago

I'm interested in tracking Go as a replacement for Erlang. Some things that should probably happen:

• GC works on 32 bit systems.
• Scheduler improvements so that things are a bit more robust ( http://code.google.com/p/go/issues/detail?id=543 )
• A fairly solid library implementing supervision trees or something like it.
• Probably some other things, but those strike me as the big ones. The important thing is for the system to be able to run for months at a time.

I think they'll get there eventually. Maybe not 100%, but 'good enough' sooner or later.

reply

jerf 7 hours ago

Right now, as near as I can tell, you basically can't implement an Erlang supervision tree in Go. You don't have anything like linking, which really has to be guaranteed by the runtime to work properly at scale, so bodging something together with some 'defer' really doesn't cut it. You also can't talk about "a goroutine", because you can't get a reference or a handle to one (no such thing), you can only get channels, but they aren't guaranteed to map in any particular way to goroutines.

I've done Erlang for years and Go for weeks, so I'm trying to withhold judgement, but I still feel like Erlang got it more right here; it's way easier to restrict yourself to a subset of Erlang that looks like a channel if that is desirable for some reason than to implement process features with channels in Go.

reply

JulianMorrison? 8 hours ago

That issue is quite simply a misinterpretation of goroutines.

Erlang: perform a maximum number of reductions, then switch, or switch on IO. The number of reductions is cleverly adjusted so that a process which is swamping other processes will be throttled.

Go: switch on IO.

Go's design is much simpler, and closer to Ruby Fibers than to Erlang processes, except that goroutine scheduling can use multiple threads. To cooperatively switch without doing IO, call runtime.Gosched().

reply

trailfox 10 hours ago

Akka is also a viable Erlang alternative.

reply

waffle_ss 6 hours ago

Looks like a nice library but I don't think it's a serious contender to replace Erlang because the JVM just isn't made for the level of concurrency that Erlang's VM is. Off the top of my head as an Erlang newbie:

• You're still using Java threads, so the potential to access [shared memory][1] is there (i.e. "shared nothing" can't be guaranteed).
• You're still subject to global GC pauses, whereas the Erlang VM has per-process GC.

[1]: http://doc.akka.io/docs/akka/snapshot/general/jmm.html

reply

An Erlang process has its own heap, so when it blows up, its state just goes away, leaving your remaining program's state untouched. With Go, there is no way to recover sanely; even if your goroutines are designed to copy memory, Go itself has a single heap.

Now, this is a very odd design decision for a language that claims it's designed for reliability. Perhaps Go's authors thinks it's better just for the entire program to die if a single goroutine falls over; well, that's one way, but it's a crude one. Erlang's design is simply better.

I wonder if Go can ever adopt per-goroutine heaps, or whether it's too late at this stage. I was happy to see that Rust has chosen to follow Erlang's design by having per-task heaps, even if all the pointer mechanics (three times of pointers, ownership transfer, reference lifecycles and so forth) result in some fairly intrusive and gnarly syntax.

reply

masklinn 1 day ago

> Go allows you to share memory between goroutines (i.e. concurrent code).

Go will share memory, by default, and special attention must be taken preventing or avoiding it. It's not an allowance.

> In fact, the Go team explicitly tells you not to do that

And yet they have refused to implement a correct model, even though they have no problem imposing their view when it fits them (and having the interpreter get special status in breaking them, see generics).

reply

burntsushi 1 day ago

> Go will share memory, by default, and special attention must be taken preventing or avoiding it.

Not really. If you use channels to communicate between goroutines, then the concurrency model is that of sequential processes, even if channels are implemented using shared memory under the hood.

That is, the default concurrency model militated by Go is not shared memory, but that of CSP. It's disingenuous to affix Go with the same kind of concurrency model used in C.

> And yet they have refused to implement a correct model

What is a correct model? Erlang's model isn't correct. It's just more safe.

> (and having the interpreter get special status in breaking them, see generics)

What's your point? Purity for purity's sake?

reply

masklinn 19 hours ago

> Not really. If you use channels to communicate between goroutines, then the concurrency model is that of sequential processes

Except since Go has support for neither immutable structures not unique pointers, the objects passed through the channel can be mutable and keep being used by the sender. Go will not help you avoid this.

> That is, the default concurrency model militated by Go is not shared memory, but that of CSP. It's disingenuous to affix Go with the same kind of concurrency model used in C.

It's not, go passes mutable objects over its channel and all routines share memory, you get the exact same model by using queues in C.

Erlang's core concept of concurrency seems like something that'd be better suited as a library and app server than a whole language and runtime.

I've yet to hear of any Erlang-specific magic that cannot be implemented inside another language.

reply

SoftwareMaven? 1 day ago

How would you get per-actor heaps that cannot be violated by other actors? That is critical to Erlang's ability to recover from processes dying. I spent a lot of time doing Java and can't think how you could (you could in the JVM if you had language constructs for it, but then we are back to a new language).

There's a reason Stackless Python's actors aren't just a library on top of Python.

reply

reeses 11 hours ago

TLABs are a partial step in that direction at the JVM level. You could also use object pools/factories keyed to the thread.

Those are the first two "ghetto" hack solutions I can think of that wouldn't require significant code changes on a going-forward basis.

reply

SoftwareMaven? 7 hours ago

But those hacks wouldn't provide the same guarantees that language-level changes provide. Sure, you can try not to impact other thread's heaps, but nothing is stopping me, which means a simple programming error has the potential to impact multiple threads. As a result, you can't just "reboot" that thread (a critical piece of what makes Erlang interesting), because you have no guarantees its errors didn't impact other threads. You also have no guarantees that the underlying libraries aren't mucking up all of your carefully crafted memory management.

It's like the kernel protecting memory so applications can't overwrite each other. Sure, applications could just write to their own memory, but nobody actually trusts that model[1]. Instead, they want something below that level enforcing good behavior.

reeses 11 hours ago

Obviously, you can do the same things in Java, as people have demonstrated with alternative languages that target the JVM.

It's the expressiveness at the language level that is really the "magic". For example, doing the equivalent of OO is not intuitive in Erlang, but completely possible (actually easy, but it looks...wrong) whereas it's supported by every Java tool. By the same token, pattern-matched message passing, lightweight green threads, and hot code deployment are primary concepts in Erlang.

reply

dumael 12 hours ago

You can at best implement a mimicry of Erlang's message passing in Java.

With sufficient effort, you can have the equivalent of no shared mutable data.

What you cannot have is completely separate heaps, so that if one thread crashes for whatever reason it doesn't take your application down.

Also, good luck trying to find a garbage collector that supports completely separate heaps that isn't a direct copy/near-identical implementation of the BEAM[1] VM's GC.

[1] The virtual machine that is the stock VM for Erlang. (In fact I don't know of any others but I have never looked.)

reply

reeses 11 hours ago

We've been doing it in Java since the nasty old days of RMI or "our own weird RPC over HTTP implementation". Lots of lightweight services with a supervisor to manage them and a directory to find them.

Now it's even easier with ESBs. Write a ten-line grails service to expose a bin-packing facility with Drools and then never touch it again.

reply

Evbn 1 day ago

Java's syntactic overhead stops most humans. "final" everywhere, big method names for every primitive message passing operation, etc

reply

---

seanmcdirmid 1 day ago

Erlang is not designed for parallel programming; it is designed for concurrent programming. These are two very different programming domains with different problems.

Every time someone conflates parallelism with concurrency...everyone gets very confused.

reply

unoti 23 hours ago

Isn't it really fair to say that it's designed for both? The way it uses immutable state and something-similar-to-s-expressions to express data make it very straightforward (or even transparent) to distribute work between multiple processes and separate computers, in addition to how it makes it practical and simple to break work into small chunks that can be interleaved easily within the same thread. It's really designed for doing both very well, wouldn't you say?

reply

seanmcdirmid 23 hours ago

Not at all. Erlang isn't useful for modern parallel computing as we know it, which is usually done as some kind of SIMD program; say MapReduce? or GPGPU using something like CUDA. The benefit doesn't just come from operating on data all at once, but these systems (or the programmer) also do a lot of work to optimize the I/O and cache characteristics of the computation.

Actor architectures are only useful for task parallelism which no one really knows how to get much out of; definitely not the close-to-linear performance benefits we can get from data parallelism. Task parallelism is much better for when you have to do multiple things at once (more efficient concurrency), not for when you want to make a sequential task faster.

Maybe this will help

http://jlouisramblings.blogspot.com/2011/07/erlangs-parallel...

and

https://news.ycombinator.com/item?id=2726661

reply

reeses 7 hours ago

SIMD is a specialized form of parallelism. It is not the only definition of the term.

It should also be clear that task parallelism (or concurrency from your perspective) has not had the benefit of billions of engineer-hours focused on improving its performance. It is within recent memory that if you wanted 20+ CPUs at your disposal, you'd have to build a cluster with explicit job management, topologically-optimized communications, and a fair amount of physical redundancy.

As many of the applications requiring low-end clusters tended to involve random numbers or floating point calculations, we also had the annoyance of minor discrepancies such as clock drift affecting the final output. This would present, for example, in a proportional percentage of video frames with conspicuously different coloration.

seanmcdirmid 4 hours ago

Task parallelism was something used to work on 20 years ago when we thought it was the solution to scaling. But then we found that the supercomputer people were right all along, that the only thing that really scales very well is data parallelism. So the focus in the last 5/10 years has been finding data parallel solutions to the problems we care about (say deep neural network training), and then mapping them to either a distributed pipeline (MapReduce?) or GPU solution.

> It is within recent memory that if you wanted 20+ CPUs at your disposal, you'd have to build a cluster with explicit job management, topologically-optimized communications, and a fair amount of physical redundancy.

You are still thinking about concurrency, not parallelism. Yes, the cluster people had to think this way, they were interested in performance for processing many jobs; no the HPC people who needed performance never thought like this, they were only interested in the performance of one job.

> As many of the applications requiring low-end clusters tended to involve random numbers or floating point calculations, we also had the annoyance of minor discrepancies such as clock drift affecting the final output.

Part of the problem, I think, is that we've been confused for a long time. Our PHBs saw problems (say massive video frame processing) and saw solutions that were completely inappropriate for it (cluster computing). Its only recently that we've realized there are often other/better option (like running MapReduce? on that cluster).

reply

seanmcdirmid 15 hours ago

Yes, erlang is great for concurrency, GPUs are great for significant scalable parallelism. They both solve different problems, I agree, and that's my point.

reply

seanmcdirmid 4 hours ago

> but MIMD is more easily applied. Almost any application can be refactored to spawn lightweight threads for many calculations without any explicit knowledge of the π-calculus.

MIMD hasn't been shown to scale, and its not locking that is the problem, but I/O and memory.

> To your point and for a well-understood example, make -j doesn't always result in faster compilations. It may if you have the ability to leverage imbalances in CPU and storage hierarchies, but you can also kill your performance with context switches (including disk seeking).

When Martin Odersky began pushing actors as a solution to Scala and multi-core, this was my immediate thought: the Scala compiler is slow, can this make it go faster? It was pretty obvious after some thought that the answer was no. But then we have no way yet of casting compilation as a data-parallel task (a point in favor of task parallelism, but it doesn't help us much).

reply

---

And in a lot of ways Erlang sucks, too. The syntax is outdated and stupid (Prolog lol), it has weird type coercion, memory management isn't handled that well (and many more). Of course, since Facebook uses it, people think it's a magic bullet (Erlang is to Facebook like Python is to Google).

> How is it any more or less stupid than curly bracket.

When you move a few lines around to refactor, you have to pay a lot of attention to the "ant turd token" line endings. I like Erlang, but still find this aspect of it annoying.

--- " However, in my opinion separate heaps was absolutely 100% the correct design decision. The main benefit is not having to worry about the effects of a long-running, stop-the-world garbage collection, which can have catastrophic effects on user interaction, server response times, request queue lengths, etc. An additional benefit is that the language implementers can use "dumb" algorithms for garbage collection and avoid a large class of difficult-to-track bugs.

"

There is some magic in how it has separate heaps and how it maps schedulers n:m (n cpus say 2 to 24 to m processes say 2 to 300k), how it provides concurrent garbage collection without stopping the world, how it provides hot code reloaded if that is what you need.

Back to your point though, I think once you understand the language it is actually a lot simpler to understand what is going on. Modules are usually very self contained, and you don't get the layers upon layers of indirection you see in other frameworks (I'm looking at you Rails). I think the functional style of programming as well helps to keep things simple, it doesn't make sense to have a 20 line function in Erlang.

We used Erlang several years ago. The code base has ~100k lines of code so it should be representative. We abandoned it later and switched to C++ because of performance (mostly in mnesia) and quality issues (some drivers in OTP). We didn't expect too much from performance considering it is functional (which seldom does in place update) but it is still below expectation.

It is understandable though. Just think about how much resources have been put into development of Erlang VM and the runtime/libraries(OTP), and compare it with JVM/JDK. There is just no magic in software development. When talking about high concurrency and performance, the essential things are data layout, cache locality and CPU scheduling etc for your business scenario, not the language.

reply

acqq 1 day ago

The tests here confirm your experience, Erlang is for many algorithms significantly slower, even when more cores are used:

http://benchmarksgame.alioth.debian.org/u64q/erlang.php

reply

artsrc 1 day ago

Excel solves this right problem, and Erlang does not.

Erlang allows you to create concurrent programs, i.e.: programs where the result is schedule dependent.

One right problem is allowing people to write deterministic parallel programs. This gives you the speed (from parallel) with the reliability (from deterministic).

http://joearms.github.io/2013/05/31/a-week-with-elixir.html

https://news.ycombinator.com/item?id=5798049

--

" Anthony EdenAugust? 19, 2013 at 7:08 AM

I use both Go and Erlang, depending on my needs. I prefer Erlang's design and I adore OTP, however I'll be the first to admit that learning to think in Erlang was significantly more challenging than learning to think in Go, the latter being very similar to almost every other imperative language I've already used. Additionally when it comes to resource usage Go is pretty damn amazing (using very little CPU and memory compared to a lot of other languages I've used, like Ruby and Java) whereas the Erlang VM tends to use quite a bit of both RAM. "

--

"

zaphar 2 days ago

(full disclosure: I work at google and also like erlang)

Erlang has fantastic facilities for robustness and concurrency. What it does not have is type safety and it's terrible at handling text in a performant fashion. So if you don't care about either of those things and only care about robustness and concurrency then Erlang is great. There were internal discussions about Erlang here but the upshot was. We had already basically duplicated Erlangs supervision model in our infrastructure, only we did it for all languages and Erlang didn't offer any benefits in performance for us. It's only benefit would have been the concurrency model. That's much less benefit than Go gives.

Go gives you Erlangs concurency model, a similar philosophy of robustness, type safety, all batteries included, and performance. Equating the two languages works in 1 or 2 dimensions but not on all the dimensions google cares about.

reply

derefr 2 days ago

Interesting, thanks for that; it's pretty much what I guessed (especially the bit about the supervision tree and hot-code-upgrade advantages being mooted by your infrastructure.)

On a tangent, though:

> What it does not have is type safety

I've tried to work this out before (I'm designing a new language for Erlang's VM), but as far as I can tell, type safety is in practice incompatible with VM-supported hot code upgrade.

If you have two services, A and B, and you need to upgrade them both, but you can't "stop the world" to do an atomic upgrade of both A and B together (because you're running a distributed soft real-time system, after all), then you need to switch out A, and then switch out B.

So, at some point, on some nodes, A will be running a version with an ABI incompatible with B. In a strongly-typed system, the VM wouldn't allow A's new code to load, since it refers to functions in B with type signatures that don't exist.

On the other hand, in a system with pattern-matching and a "let it crash" philosophy, you just let A's new code start up and repeatedly try-and-fail to communicate with B for a while, until B's code gets upgraded as well--and now the types are compatible again.

It's an interesting problem.

reply

laureny 2 days ago

> type safety is in practice incompatible with VM-supported hot code upgrade.

That's not true.

First, it's very easy to hot reload changes that have been made to the code that are backward compatible. The JVM spec describes in very specific details what that means (adding or removing a method is not backward compatible, modifying a body is, etc...).

This is how Hotswap works, the JVM has been using it for years.

As for changes that are backward incompatible, you can still manage them with application level techniques, such as rolling out servers or simply allow two different versions of the class to exist at the same time (JRebel does that, as do other a few other products in the JVM ecosystem).

Erlang doesn't really have any advantages over statically typed systems in the hot reload area, and its lack of static typing is a deal breaker for pretty much any serious production deployment.

reply

rdtsc 1 day ago

> lack of static typing is a deal breaker for pretty much any serious production deployment.

Are you talking about Google only where they made it a mandate or in general? There are serious production deployments on Python, Ruby, Erlang and Javascript.

I will trade expressiveness and less lines of code with a strong but dynamically typed language + tests over more a static typed language with more lines of code all being equal.

Or put it another way, if strong typing is the main thing that protects against lack of faults and crashes in production, there is a serious issue that needs to be addressed (just my 2 cents).

reply

derefr 2 days ago

> As for changes that are backward incompatible, you can still manage them with application level techniques, such as rolling out servers or simply allow two different versions of the class to exist at the same time (JRebel does that, as do other a few other products in the JVM ecosystem).

Neither of these allow for the whole reason Erlang has hot code upgrade in the first place: allowing to upgrade the code on one side of a TCP connection without dropping the connection to the other side. Tell me how to do that with a static type system :)

reply

pmahoney 1 day ago

Tomcat (and other app servers) has support for doing hot reloads of Java web apps while not reloading the HTTP layer (and not dropping TCP connections).

http://www.javacodegeeks.com/2011/06/zero-downtime-deploymen...

I have implemented a similar system for JRuby apps running inside a Servlet container. There are many caveats. I don't actually recommend it because for a while you're using nearly twice the memory (and JRuby is particularly memory hungry). Also there are many ways to leak the old class definitions such that they are not GC'd (e.g. thread locals). But it's certainly possible.

I suspect that Erlang, Java, and all languages are in the same boat: some parts can be upgraded live in the VM while other parts require a full restart (maybe coordinating with multiple nodes and a load balancer to achieve zero-downtime).

reply

banachtarski 21 hours ago

Erlang is not in that boat. Generally, you can upgrade the entire thing (except the lowest level libraries obviously) without too much fuss.

reply

lenkite 1 day ago

Out of curiosity, where/why would such an exotic feature be needed in today's internet architectures where you always front a group of servers with a load balancer ?

reply

banachtarski 21 hours ago

Not everything is a website! Also, not everything is stateless. Consider writing a chat application for the web for example and letting users on one page communicate with another one.

reply

butterflyhug 1 day ago

Not all Internet protocols are HTTP. If you're running a service where long-lived connections are the norm, "simply fronting a bunch of servers with a load balancer" can require a pretty smart load balancer. E.g. IMAP connections often last hours or even days, and are required to maintain a degree of statefulness.

reply "

--

DanWaterworth? 2 days ago

Go gives you Erlangs concurency model

There are a number of significant differences between Erlang's and Go's concurrency models: Asynchronous vs synchronous communication, per-thread vs per-process heaps, send to process vs send to channel.

reply

zaphar 2 hours ago

Go has asynchronous communication and synchronous communication.

And the other things you mention are in practice not significant differences. The model both use is based on Hoares CSP and the same general ways of using apply. Some of the specifics must accomodate differences but those are differences of implementation not the general model.

reply

--

" It is too bad though. Erlang, I think, has some features I like better such as hot code reloading, better supervision strategies, separate heap per lightweight process (hence no stop-the-world GC), and is battle tested for longer. Also, at least to me, actors with pattern matched receive, as central building blocks, makes more sense than channels. So I'll keep Erlang as my concurrent server side go-to language for now. "

--

AYBABTME 2 days ago

When people ask «Why not Erlang instead of Go, Erlang is X and Y and Z...» they seem to be oblivious to the fact that Go is C-like and Erlang has a pretty weird Prolog-like syntax.

I've had some experience with Prolog before touching any Erlang. It's not a syntax or a programming style that I liked, not at all. When I came to do some Erlang [1], I found the same style and it was not a pleasant surprise.

I learned C-like languages first, so maybe that's why my view is flawed. But most people also learn C-like languages as their first language, so it might be that Erlang looks like an ugly beast when they come across it. And thus as a concurrent language, Go seems to be first of it's kind.

Meanwhile, Go has a very simple style that pretty much everybody can read out of the box.

[1]: CS Games in Canada, one challenge was a 'debugging' competition where programs in ~10 languages had bugs we had to find and fix in 90 mins. One of them was in Go, another one in Erlang. Out of about 20 teams participating, me and another one (2/20) managed the Erlang one while the vast majority of the other teams managed to do the Go one. FWIW, this can speak to people's ability to understand Go vs. Erlang.

--

rdtsc 1 day ago

> they seem to be oblivious to the fact that Go is C-like and Erlang has a pretty weird Prolog-like syntax.

Yeah it has Prolog-like syntax. However, when designing and working on large distributed system looking at just the syntax is a kind of shortsighted. The problem is not syntax (which is different, and is actually pretty simple, and a lot less ambiguous than say Javascript or has less "features" than C++), the problem is _semantics_. And by that I mean structuring your programs as a set of multiple concurrent actors. That is the hard part.

Another way to put it. Erlang is probably getting looked at because someone wants to either scale, build a distributed system or a highly fault tolerant system. At that point, if dots vs semicolons is a major stumbling block what are they going to do when they hit a netsplit.

Now, Erlang like any tool has trade-offs. But those are about isolation and private heaps vs raw sequential performance. Hot code upgrades and compiled static code and pointers referenced everywhere in the code is not going to work well. Stuff like that. Single assignment is also another common one.

(And if syntax is a major stumbling block, there is Elixir or a lisp like languages LFE and Joxa that all take advantage of the actor model and run on the BEAM VM).

reply

--

lbarrow 2 days ago

One of the big projects they're working on in the next version of Go is to make the internal scheduler more Erlang-esque =)

reply

MetaCosm? 2 days ago

Yep, full preemptive scheduler, I am very excited about it!

reply

--

--

continuations 2 days ago

The author did say CPU efficiency is key, and Erlang isn't exactly known for its raw speed.

In general search is very CPU intensive, maybe that's why Google never adopted Erlang.

reply

banachtarski 21 hours ago

Yes but intentionally so. There is an intrinsic tension between latency and throughput. Erlang chooses willfully to optimize for the former rather than the latter. This works when the majority of the tasks occurring concurrently are typically small and lightweight (aka, a web server).

reply

--

"

pekk 2 days ago

Are Go and Erlang that fungible with each other? Honest question.

reply

derefr 2 days ago

Not in theory, but yes in practice.

For just one example, Go produces static native binaries, while Erlang produces bytecode for a virtual machine. But the Erlang virtual machine is tiny and it's standard practice (with tool support) to ship it with your application as a "release", so either way you get the effects of having one self-sufficient blob of code in a folder that you can "just run" without having to think about runtimes or libraries.

What I would say is that, for every IO-bound highly-concurrent C++ project Google is rewriting into Go, the same project could have been rewritten into Erlang, and they'd see most of the same advantages relative to C++: better, more transparent concurrency; "batteries included" for solving various client-server and distribution problems; being able to just drop a code package on a server and run it; etc.

reply

zaphar 2 days ago

You're assuming IO-bound highly-concurrent C++ server don't have other requirements besides those two. Maybe it's IO-bound highly concurrent text processing. Erlang will suck at this despite the two pieces it's excellent at. Go is pretty fast at processing text and google does a lot of text processing.

reply

derefr 2 days ago

What exactly does "text processing" mean, by the way? Erlang is very good at processing streams of bytes--you can pattern match on binaries to get new sub-binaries (which are basically equivalent to Go's array slices) to pass around, etc. It just gets awkward when you have to convert those streams into codepoints to do case-insensitive comparisons and such.

But to reply more directly, "IO-bound" means something specific--that the problem will be using a negligible amount of CPU, no matter what constant coefficient of overhead the language adds, and so scaling will never be a problem of "oops it's using too much CPU to do the text processing" but rather "oops the gigabit links are saturated we need to add more boxes."

reply

zaphar 2 hours ago

You answered your own question when you acknowledged the areas Erlang get awkward in. Google supports more than 50 different languages. Doing that requires performant analysis of more than just bytes but of unicode codepoints.

reply

Daishiman 1 day ago

Google has both CPU-bound and IO-bound problems at massive scales. A language that is not highly performant in either area would be insufficient.

reply "

--

" Some requirements in that set (e.g. serving data on dl.google.com) could be solved perfectly well by Erlang, and could have been for years, but can also now be solved perfectly well by Go. That they are now being solved by Go is likely an effect of the "Golang advocacy group" that has formed at Google.

Others can't be solved by Erlang, but can be solved by Go (e.g. CPU-bound matrix-multiplications for PageRank? index calculation), or vice-versa (e.g. deploying new Google Talk daemon code without dropping the XMPP connection.) "

-- haskell vs erlang talk

http://research.microsoft.com/en-us/um/people/simonpj/papers/haskell-retrospective/Haskell-Erlang-Jun09.pdf

--

marshray 1 day ago

Erlang seems to pay a small performance penalty for that ability though. The VM is not allowed to run more than a few thousand instructions without checking for a signal to task switch.

But if that kind of guaranteed cooperation isn't needed, I'd expect that lightweight threads could be added to a language like this in a straightforward way.

reply

--

"

Jessica Darko wtpayne • 4 hours ago

    −

No. Golang is not capable of concurrency. Sorry. If you try to write a distributed system in it, eventually you will run into very painful, intractable problems, becuase golang is not competently architected (which is to say, they didn't take concurrency seriously when they made the language). This is probably because they wanted to compile down to machine code and didn't want to bother with a virtual machine, but as a result, golang is just a better threading model.

Erlang is the only solution I'm aware of that does it right, and can scale in that way.

Unfortunately, almost everyone you meet who advocates golang has not actually learned erlang and come to understand what is necessary for concurrency (and it's not one thing, but a collection of things, including immutability, and no shared memory) .... and so most of the advocacy for golang you see out there is written by people who don't know what they're talking about.

Plus, on top of it, golang is written by guys who were such flaming aholes that they decided to force everyone to follow their one True Way for braces by putting it in the language. Why write in a language written by people who show such disrespect for programmers? 1 • Reply • Share ›

" --

" I’m tempted to write a comparison between BEAM and Dis, Inferno’s VM, since there are some very close similarities (e.g., message-passing, cheap concurrency, a secure VM) and some interesting differences (the type system, Inferno’s channels versus Erlang’s messages, the speed difference between the two VMs, Inferno’s sh versus the Erlang shell, and the language choice in Inferno versus on BEAM). I think that might tend towards evangelism, though. "

--

Still, when I say "Erlang", I don't mean the syntax, I mean mostly the VM and stdlib (the platform semantics, in other words.) You can get those with Elixir or LFE or any number of growing projects, just like you can get Java's platform semantics with Scala or Clojure. Everyone agrees Erlang's syntax is hideous, after all--even its developers.

--

"

When compared to Java/C++, Clojure's approach to concurrency is much simpler and easier to "get right" when you're coding concurrent programs. However, Clojure isn't the only language to provide "native" support for concurrency. It's interesting to compare the approach that Clojure has taken with the approach that Erlang (another language that has built-in support for concurrency) has taken. I've posted a bit about Erlang in the past on my blog; so, although I'm not an Erlang expert, I do know something about the language. I'll focus on three main areas as a basis for comparison:

    Autonomous/independent threads of execution: Actors in Erlang, Agents in Clojure. Erlang's actors are light-weight "user-land" processes (e.g. - "green threads") that are autonomous and can scale tremendously. Clojure's agents are native threads that are managed in thread pools for performance. As explained in the documentation, "Clojure's Agents are reactive, not autonomous - there is no imperative message loop and no blocking receive". Rich Hickey has summarized the Actor/Agent differences and his design rationale:

        "There are other ways to model identity and state, one of the more popular of which is the message-passing actor model, best exemplified by the quite impressive Erlang. In an actor model, state is encapsulated in an actor (identity) and can only be affected/seen via the passing of messages (values). In an asynchronous system like Erlang's, reading some aspect of an actor's state requires sending a request message, waiting for a response, and the actor sending a response. It is important to understand that the actor model was designed to address the problems of distributed programs. And the problems of distributed programs are much harder - there are multiple worlds (address spaces), direct observation is not possible, interaction occurs over possibly unreliable channels, etc. The actor model supports transparent distribution. If you write all of your code this way, you are not bound to the actual location of the other actors, allowing a system to be spread over multiple processes/machines without changing the code.

        I chose not to use the Erlang-style actor model for same-process state management in Clojure for several reasons:
            It is a much more complex programming model, requiring 2-message conversations for the simplest data reads, and forcing the use of blocking message receives, which introduce the potential for deadlock. Programming for the failure modes of distribution means utilizing timeouts etc. It causes a bifurcation of the program protocols, some of which are represented by functions and others by the values of messages.
            It doesn't let you fully leverage the efficiencies of being in the same process. It is quite possible to efficiently directly share a large immutable data structure between threads, but the actor model forces intervening conversations and, potentially, copying. Reads and writes get serialized and block each other, etc.
            It reduces your flexibility in modeling - this is a world in which everyone sits in a windowless room and communicates only by mail. Programs are decomposed as piles of blocking switch statements. You can only handle messages you anticipated receiving. Coordinating activities involving multiple actors is very difficult. You can't observe anything without its cooperation/coordination - making ad-hoc reporting or analysis impossible, instead forcing every actor to participate in each protocol.
            It is often the case that taking something that works well locally and transparently distributing it doesn't work out - the conversation granularity is too chatty or the message payloads are too large or the failure modes change the optimal work partitioning, i.e. transparent distribution isn't transparent and the code has to change anyway.
        Clojure may eventually support the actor model for distributed programming, paying the price only when distribution is required, but I think it is quite cumbersome for same-process programming. YMMV of course."

    Safe use/update of shared data: Mnesia in Erlang, STM in Clojure. In Erlang, control of updates to thread-shared data is handled by Mnesia, a distributed DBMS. This provides full ACID data integrity; however, Erlang is designed to be a distributed concurrency language and Clojure is not, so the cost/benefit of using a DBMS for non-distributed data integrity has to be taken into consideration. Clojure's STM-based approach is an effective, well-performing solution that doesn't require any out-of-process storage. Although STM has recently had some vocal detractors, most of the criticisms of STM would appear to apply more in an environment where mutable variables/structures are the norm. There is a thread on the Clojure mailing list that details a number of reasons why Clojure's use of STM avoids many of the things that STM has been criticized for. As concurrency-guru Cliff Click says in the thread:

        "I think Clojure is on to something good here - Immutability IS the Big Hammer For Concurrency here, and the STM is just one of several flavors of 'Big Hammer isn't working so I need another approach'. Given the Immutability-By-Default, STM has a much better chance of being performant than in other languages so it makes sense to give it a go."

    Distributed concurrency: This one really subsumes the other two (as well as some others that I won't be discussing here). Providing native support for distributed concurrency (also called parallel computing) affects quite a few design choices when you're building a language. Erlang was designed from the outset to support both distributed and non-distributed (same-address-space) concurrency. Therefore, Erlang uses the same actor-based messaging model in both distributed and non-distributed processing environments and Mnesia is designed to provide a shared data store for Erlang processes that are not necessarily running on the same machine. These choices make sense for Erlang because Erlang has a unified approach to both distributed and non-distributed concurrency. Clojure deliberately does not have any unified, "built-in" support for distributed concurrency. This decision means that distributed concurrency can be more difficult to program in Clojure than in Erlang; however, it also means that non-distributed concurrency (which, for many applications, will be the more common concurrency scenario) can be catered for in an easier manner. In a comparison of the two languages, Rich Hickey explained his reasons for not designing Clojure with support for a distributed concurrency model:

        "I have some reservations about unifying the distributed and non-distributed models (see e.g. http://research.sun.com/techrep/1994/smli_tr-94-29.pdf), and have decided not to do so in Clojure, but I think Erlang, in doing so, does the right thing in forcing programmers to work as if the processes are distributed even when they are not, in order to allow the possibility of transparent distribution later, e.g. in the failure modes, the messaging system etc. However, issues related to latency, bandwidth, timeouts, chattiness, and costs of certain data structures etc remain. My experiences with transparent distribution were with COM and DCOM, and, quite frankly, not happy ones. I think Erlang has a much better story there, but the fact is that distributed programming is more complex, and I personally wouldn't want to incur that complexity all the time. I think it is ok for a system designer to decide one part of a system will be distributed and another not, and incur some rewrite if they are wrong. If I wrote phone switches I might think otherwise :) "

--- http://www.tbray.org/ongoing/When/200x/2009/10/26/Messaging

" Not all the prac­ti­cal sane lock-free so­lu­tions to the con­cur­rency co­nun­drum in­volve mes­sage-pass­ing; for ex­am­ple, see Clo­jure’s Atoms and Refs. But I do think that mes­sage-pass­ing is im­por­tant; it pre­sents a rel­a­tively sim­ple men­tal model that isn’t ter­ri­bly hard to think about, it scales nat­u­rally out­side the shared-global-mem­ory space, and it maps rea­son­ably well onto the se­man­tics ac­tu­ally pro­vided by the un­der­ly­ing op­er­at­ing sys­tems. ¶

Cer­e­mony · Clo­jure has a lot less. You don’t have to cre­ate threads or take care of send­ing and re­ceiv­ing mes­sages. That counter func­tion in­volves three lines of code, as op­posed to 29 in Er­lang. This is good. ¶

To be fair, a user of the Counter mod­ule would only ever have to say incr(Counter) and count(Counter). But still, the whole im­ple­men­ta­tion of both sides is just three lines of Clo­jure.

I do won­der if there are cases where I’d miss Er­lang’s di­rect con­trol over the process in­ven­tory? Some­times I might be count­ing twelve things and some­times twelve thou­sand.

Cou­pling · Er­lang has a lot less. I don’t need to know what kind of a thing I’m talk­ing to, the mes­sages I send it are just chunks of data, and it gets to de­cide how to deal with them. In par­tic­u­lar, it can triv­ially ig­nore those it doesn’t care about. ¶

This seems good to me. But maybe I’m kid­ding my­self here; As a de­vel­oper, nor­mally wouldn’t I be con­trol­ling both sides of the con­ver­sa­tion? And as I look at ac­tual Er­lang code, I no­tice that in prac­tice, a lot of the mes­sages are along the lines of { opcode, argument, argument, ... } — they smell a lot like dressed-up func­tion calls.

Pat­tern Match­ing · I think this is ac­tu­ally the im­por­tant dif­fer­ence be­tween the ap­proaches. The mes­sage you send to a Clo­jure agent is just a func­tion, and the sys­tem goes about ap­ply­ing it like any other. On the other hand, the re­ceiv­ing end in Er­lang gets to pat­tern-match, which feels nat­ural and looks read­able. Is this the prover­bial Com­puter Sci­ence one-more-level-of-in­di­rec­tion which turns out to solve an im­por­tant prob­lem? ¶

Scal­ing · My use-case here, count­ing in­te­gers, is aw­fully lim­ited. I’ve built up a “feel” for what Er­lang mes­sage wran­gling feels like in prac­tice at sig­nif­i­cant-pro­gram scale. It’s one of the rea­sons I’m pos­i­tive on Er­lang: it’s easy to ex­plain and easy to un­der­stand (well, com­pared to lock­ing). When I stress Clo­jure a bit, if I find it even eas­ier, that’d be a big sell­ing point. ¶

If only Clo­jure weren’t a Lisp... "

--

http://joearms.github.io/2014/02/01/big-changes-to-erlang.html

--

Ask HN: Why isn't Erlang more popular?

https://news.ycombinator.com/item?id=7277797

jerf 1 day ago

Honestly, I think what kills it is that it's not an Algol-descended language [1]. If Erlang was written with an Algol-esque syntax it would have taken off years ago. But instead it has this weird syntax, which it then doesn't really do that much with.

--

Nogwater 1 day ago

Not knowing enough about Erlang/OTP or Go: Would it be possible for more of OTP to be replicated in a Go 2.0? What's missing? What would it take? What can never be replicated?

reply

jerf 1 day ago

You can recover a substantial portion, but there's a few things you can't quite get back. Since Go is a mutable-state language, and also a shared-state language (isolation is by convention, not by language design), you have to live with the consequences of that. In my supervisor tree implementation, the restart of a monitored goroutine is just to fire off a new goroutine on the exact same object again; if you still have bad state lying around and immediately crash again, well, too bad, the supervisor can't do anything about that. (Except not thrash the processor with endless restarts.) In contrast, when an Erlang supervisor restarts a process, it is guaranteed to be a fresh process with no shared state from the one that just crashed. It's much more likely to not immediately crash.

--

waffle_ss 1 day ago

Besides the issues that jerf mentioned, Erlang also has per-process garbage collection built right into the VM. Last time I checked Go's garbage collection was still stop-the-world, mark-and-sweep, making it less suitable for soft real-time systems than Erlang.

reply

--

Jtsummers 1 day ago

I'd say it works for small scale as well. The concurrency is awesome, but (and this may just be from limited experience) I've yet to see another language that handles binary matching and manipulation as well. I tried to sell my current office on erlang for one project. Take a dozen mutually incompatible protocols (used by systems that can't be updated easily to a new or common protocol, that's actually been tried and is why there are a dozen protocols now) and create a proxy/translator server for all of them. In essence we have a dozen networks that can't talk to each other, but send the same information using their own formats. Writing a translator from one protocol to another (or my suggestion of one protocol to intermediate representation to second protocol, need only 24 translation modules instead of 12! translation modules) was trivial for my proof of concept sytem. The concurrency was really just a nice bonus, in this case.

reply

--

davidw 1 day ago

That's the hype, but if you ask the people who built it, it's actually more about fault tolerance than scaling. And indeed, I'm using it for a semi-embedded system where stability is important, something that Go and Node.js, for instance, didn't seem quite ready for when we evaluated them a year ago. Erlang is pretty rock solid and is something that we'll be happy to have running without worries.

reply

--

technomancy 1 day ago

My theory: writing network servers that are not web servers is a relatively uncommon problem to have these days.

One of the hardest parts about learning a new language is coming up with a learning project that showcases the unique strengths of the language without being intimidating to a newcomer or too contrived to actually be useful. This is difficult in any language, but it's especially so in Erlang.

Obviously "it's different; people don't like things that are different" has a lot to do with it, but we've seen other FP languages experiencing faster growth recently, so I don't think that can be the only cause.

I've been using it for a few months, for what it's worth.

Edit: obviously there are lots of people who need to write concurrent network server clusters, but I'd argue that the benefits of the Erlang approach are difficult to grasp before you've actually deployed something written in Erlang; simple toy projects (which are a prerequisite to learning a language) don't usually play to its strengths. A language that's really good at web apps is going to grow more quickly simply because its advantages are easier to appreciate from the start.

-- "lists for strings, aargh!" -- https://news.ycombinator.com/item?id=7277797

"It is also worth a note that Elixir adds language features, such as default UTF8 binary strings, real macros, Clojure-style protocols, tooling, etc, on top of the Ruby-style syntax." -- https://news.ycombinator.com/item?id=7277797

"..sub-standard string handling.." -- https://news.ycombinator.com/item?id=7278993

"While I like some of the advantages of Erlang, the lack of general support for a lot of common operations (and yes, string manipulation is a huge deal)..." -- https://news.ycombinator.com/item?id=7278026

--

lincolnq 1 day ago

I've tried using Erlang once or twice. I inherited an open source project written in it, wanted to maintain it a bit but I couldn't make head nor tail of it, and I'm a good programmer (but very busy with other things). There were too many things to learn to get started working with the Erlang ecosystem and I didn't really have the time.

I didn't find any good tools.

The error messages were obtuse.

I didn't understand how simple shit like configuration files worked. I couldn't find any place where the file was 'opened' from code. I chalked it up to the magic of the underlying framework or whatever.

--

pbnjay 1 day ago

I tried it for approximately 6 months (a few years ago). I translated some slow python into fairly idiomatic (IMHO) erlang and got some pretty significant speedups. I had vary little functional programming experience before that, so my comments here come with that caveat.

My hangups:

• lack of easy-to-use string libraries. far and away the biggest pain point. working in bioinformatics, I deal with a lot of poorly-formatted text.
• installing erlang itself was fine, but installing (and finding) any other packages was a PITA.
• documentation could have been better. I can't remember what specifically I disliked but remember being frustrated trying to find info about builtin nuances.
• syntax. this one seems silly from the outside, but the whole commas-here-but-definitely-not-there and other idiosyncracies really made tweaking code and debugging a pain. Go has similar pains around the "unused variable" errors so I know this type of thing isn't particular to erlang.

--

• no package manager
• no list of open source packages

-- https://news.ycombinator.com/item?id=7277797

---

gordonguthrie 1 day ago

Pierre Fenoll has provided a link to an extensive list of links (and summarised them) about why it has not taken off: https://github.com/fenollp/kju/blob/master/criticisms.md

reply

gcb0 1 day ago

If you skip the syntax part and the "query" pet peevy, it is as if he was reading my mind! Spot on

The string part specially... will have to check out his binary string code.

reply

--

https://github.com/fenollp/kju/blob/master/criticisms.md

---

captainmuon 1 day ago

It's just not suited for what I do. At work, I do number crunching. Essentially a bunch of simple calculations in a for loop. C++ is excellent for this. I use a lot of Python for rapid prototyping, and as glue code. When I need to do concurrency, it's usually the easiest thing to run multiple instances in parallel, or to submit jobs to a cluster. There is already a well-tested infrastructure in place, and I don't need all the fancy stuff erlang has in this area.

Second, but more important, my colleagues know C++ (good enough at least). If I started to use something else, we couldn't collaborate. We're physicists, not computer scientists or professional programmers.

Then, I often find myself writing GUI code. I wouldn't know how to start writing a GUI in Erlang, but it's trivial in Python or C++.

If anything, I would need a language centered around mutability, so almost the opposite of erlang. A language where everything is mutable, where you can databind to any object. Where you can just make an array of objects and bind it to a graphical widget, and get create, edit, update, delete operations for free. You never have to write `listview.insert(...)`. Maybe the command pattern is part of the language and it is trivial to write an undo function. And finally, it would include multithreading, with only a simple syncronization primitive, the transaction. The goal of concurrency here would not be speed, but GUI responsiveness.

So, I have two very different use cases, for one I can use C++, for the other the ideal language has not been invented (but C# and Python are both not bad). I just don't know what to do with Erlang (and Haskell, Clojure, and all the other hip languages).

reply

--

thedufer 1 day ago

I haven't used or looked at Erlang, but bear with me.

• no package manager

This is _huge_. For example: I currently work very heavily with node.js. I understand all of the many, many problems with javascript. NPM single-handedly makes up for all of them put together, in my eyes.

Which is to say - an amazing package manager can make a poor language. A decent package manager (pip, for example) allows a nice language to shine, but won't make or break it. Lack of a package manager could probably kill just about any new languages these days (and Erlang - correct me if I'm wrong - appears to have the popularity of a pretty early-stage language right now).

--

paperwork 1 day ago

Like Clojure, I like that Erlang has lots of interesting ideas. I may never use either in production, but Erlang's OTP sounds very interesting as a way to handle real-world scenarios like failure handling. Similarly, Clojure's datomic, core async, etc. are interesting ideas, implemented by people with good taste.

However, too many examples show trivial things like mapping over a list or calculating something recursively. As a professional programmer, I get how those things wrok. What sets these languages apart from Javas of the world is how state is handled. It isn't easy to dig through tutorials and docs to find the best way of keeping state, updating it, referencing it, etc.

I once attempted to implement a small order matcher (from the stock trading world) in Erlang. I know how to do it in imperative languages, but it was pretty painful to do so in Erlang. It was getting very verbose, I wasn't sure how to create a priority queue, how to modify elements in a data structure, etc. Since this wasn't a simple transformation of data, I had a hard time finding references in documentation spread across the web.

I realize that if I was committed to learning Erlang, I would work through a book or two. Perhaps find a small open source project and work through the implementation. However, I, like so many others, wasn't committed. I was merely trying it out and when I couldn't make progress, I decided to use my precious free time on something else.

reply

jamii 1 day ago

I spent a total of about 11 months (spread over a few years) consulting on a betting exchange in erlang. It was pretty painful even with lots of erlang experience.

I sat down one week and prototyped a rewrite of the core exchange in clojure. 40x better throughput and 10x better latency despite using only two threads, naive `pr-str` serialisation and storing all the exchange state in one huge immutable data structure (for easy rollback on errors). Much easier to debug and to wire up different setups without modifying code (eg swapping out the network layer without touching the io and batching code).

There are some interesting ideas in erlang but they are hidden behind an inexpressive language and poor libraries. I would think more than twice before taking on another large erlang project.

reply

paperwork 1 day ago

As non-mainstream languages go, in the financial industry, Scala is picking up steam. However, clojure is at the top of my list as the language I want to learn, despite obstacles of limited time. Lisp has an impressive pedigree and Rich Hickey is a smart, pragmatic dude who seems to have spent a great deal of time thinking about reducing complexity in real-world software engineering.

reply

jamii 1 day ago

I think that's the real strength of clojure - not the language itself but that the community around it is focused on radically reducing complexity.

Every project I have ever worked on that struggled or failed did so because the complexity outgrew the developers ability to manage it. It kind of scares me that scala is growing so quickly - from my experience working on a large scala project it seems to breed complexity like nothing else I've ever worked with (haskell, python, ocaml, erlang, clojure).

reply

---

RogerL? 1 day ago

I looked at it, even bought a couple books. A guy here at work was a huge booster for it, and used it in some small piece of production code. It looked cool, and I love parallel and distributed processing, so the idea of 'fail fast' appealed to me because of how it seemed to simplify a lot of thinking about such things. Hot swapping sounds awesome.

But

1) I don't currently do much that requires any of that.

2) I don't want to learn a completely different 'kind' of language just to hit one pain point.

3) Resources are quite limited (re the website, training, books)

4) and this is the biggee: I just don't want to do functional programming. Our Erlang booster? Want to know how much time he sometimes spent on mailing lists and such asking "how do you do X"? I mean for pretty basic stuff. I get the value of functional programming and immutability ("oh no you don't" you'll respond, but bear with me), but in the end it is too high a price to pay FOR ME. I work in imperative languages, I am comfortable in them, I can't get away from them (I need C++ for speed, for example), I'm just not going to take on a language that kicks my legs out from under me like that. Y'all can argue about the finer points of functional vs whatever, but I'll abstain. I'm interested in solving my problems, and the imperative model works very well for my brain and needs.

So, for me, an interesting toy. I'm 47 and don't have time to chase 'yet another language' which is all this is for me.

Now, if I was trying to solve issues like the ones that got Armstrong to write Erlang in the first place, perhaps I'd revisit it. But as it is? No. No time. Not enough return on the investment.

reply

---

rvirding 1 day ago

I am not going to enter into a syntax argument I have already given my views on that here http://rvirding.blogspot.se/2014/01/erlang-syntax-again-and-... . My main point is that the syntax is simple and that it fits the semantics much better than anything based on an Algol like syntax would, even one which uses ';' in the "normal" way.

And one reason it works is because it has built-in those features for building fault-tolerant systems which, for example, Go lacks. Borrowing from NASA "Failure is not an option".

reply

http://rvirding.blogspot.se/2014/01/erlang-syntax-again-and-again-and-again.html

"Every once in a while there is a blogg complaining about Erlang and its syntax. To save you the trouble here are the main syntax complaints:

• The use of ';' (and ',' and '.') is strange, different and very confusing. I.e. it's not like Java.
• 'if' is very strange. I.e. it is not like Java.
• Variables must start with an uppercase letter. I.e. it is not like Java.
• The record syntax is very strange. "

--

waffle_ss 1 day ago

I fall under the "tried it & stopped" category.

I stumbled upon Erlang in 2011, and did the full "Learn You Some Erlang for Great Good!" tutorial and about 1/2 of the Erlang Programming book (and exercises). Where I ended up is that I had no real-world use cases that would benefit from Erlang, so I just kind of atrophied with it. It seems to me like Erlang is really really good at building programs that can be modeled with message passing (like chat), but a lot of problems are difficult to map to that paradigm (or perhaps my mind just has trouble envisioning how to do it yet). The closest I got was starting to use it to write a distributed Web crawler, but I ended up using scrapy (a Python framework) because the string manipulation was annoying with Erlang, and scrapy already had a lot of features that I would have had to re-implement on my own. But as I'm starting to try and manage crawler distribution with Celery and RabbitMQ?, I find myself starting to think more and more about how Erlang would probably do this bit in particular better, so I might return to it.

The language that I've since discovered I really want for most tasks is Haskell. I've been programming Ruby full-time for the last few years, and I've grown really jaded with it. I get sick of having to write so many unit tests around everything in what I see as a poor fix for its loosy-goosyness. I'm still pretty early on in learning Haskell, but I find the approach towards correctness first to be exactly what I want, and while so far some of the material is pretty challenging I can see it will be very rewarding as well.

reply

--

prodigal_erik 1 day ago

To me it sounds like Erlang's biggest departure from the mainstream is http://c2.com/cgi/wiki?LetItCrash, casually letting processes die and be recreated. When I put effort into actually using a niche language (as opposed to merely learning it for fun) I expect it to enable me to write software that's less embarrassingly defective than everyone else's, where Erlang only seeks to reduce the pain from my doing bad work.

reply

bjourne 1 day ago

That doesn't mean you shouldn't handle errors if you can. However, there are always more errors and coding bugs left than what you think. Systems with 0 bugs just doesn't exist in the real world. So if you are writing a critical system that is supposed to run 24/7 and your code encounters a "one in a million" bug that got through testing, such that a obscure race condition or something, then to handle it the best strategy is to reset the involved processes and try again.

It's Erlang's answer to the question "What if there is a bug in the system?" Most languages doesn't answer it ("just dont write buggy code!") but Erlang does and tries to come up with a reasonable solution.

reply

---

bunderbunder 1 day ago

Based on my (limited) experience. (Learned it for the sake of learning it, thought it was fun to work with, not considering using it for any serious projects.):

I think it's similar to the reason why DSP's aren't very popular compared to general-purpose CPUs for most tasks. It's a very specialized tool, designed to solve a very specialized problem. If you don't have that problem, it's not a practical choice because it's more difficult to use and often ill-suited to more common problems. I'm sure plenty of others have mentioned strings already.

Even if you do have that problem, but only a little bit, it might not be a practical choice. There are a lot of tasks where a DSP might perform better but a general-purpose CPU's still preferable because it's good enough, and the skills necessary to work with one are more common. The story's similar for choice of programming language in many applications that require concurrency and fault tolerance, but not to any particularly great extent.

reply

---

501 1 day ago

I've posted these before[1]:

• No public bug tracker
• epmd's security (or lack thereof)
• Can't insist that epmd be started separately (`erl -no_epmd` won't start epmd but it also won't start epmd's gen_server)
• Can't swap out epmd because while `erl -epmd_module foo` is there, `net_kernel:epmd_module()` is barely used. (Although I don't think its interface is documented anyway)
• No built-in way to hook UNIX signals
• While the documentation itself is pretty good, it's presentation is lacking and it's difficult to quickly correct mistakes as you run into them
• It can be difficult to reason about when a shared binary will be garbage collected
• OS packaging (I'm thinking of Debian/Ubuntu) of Erlang and Erlang apps tends to be more harmful than helpful (old packages, namespace conflicts, etc)

To them I'd add:

• No agreed upon build tool (rebar while prevalent isn't universally accepted)
• Community libraries often have no support for upgrades or aren't packaged properly for releases
• Can't upgrade SSL if you're using it as a carrier
• No standard code format tool like gofmt (yes I know about erl_tidy, no it doesn't provide the same functionality as gofmt or it'd see similar use)
• Not enough infrastructure around built around edoc; where's the godoc or godoc.org work-a-likes? (yes I know about erldocs.com, it's no godoc.org)

[1] https://news.ycombinator.com/item?id=5801706

reply

--

jwatte 1 day ago

We've developed and run a rather large erlang cluster (> 100k simultaneous users) for several years. It's pretty clear that OTP is not ready for that scale -- most of our big bugs have been in the libraries and OTP, not in our code. Couple that with the poor type safety at compile time, and a "in place update" model that doesn't actually work for "many times a day" continuous deployment, and I feel it's not lived up to the hype. We also run PHP, C++ and Haskell stacks, each of which has had less environment-based problems.

reply

---

jeffdavis 1 day ago

Shared state is great as long as you control it.

Databases are a giant blob of shared state, albeit a carefully-managed one. People complain about them all the time, but databases completely take over in huge areas of the application development landscape. There's a reason for that -- consistency is a very powerful simplifying assumption (giant understatement). Getting consistency in erlang may be possible, and of course you can still use a database from erlang, but the philosophy doesn't quite line up.

Databases already offer isolation between transactions. And they naturally work with immutable structures similar to functional programming.

For most applications out there, HA means replicating a database, and when it goes down (which is actually quite rare), you lose a little time doing a failover.

Hot code loading means uploading a PHP file with a new extension and "mv"ing it over the old one. Need schema changes? PostgreSQL? offers transactional DDL (e.g. ALTER TABLE).

Any error in the database usually just causes that one transaction for that one request to fail. Any error in the application usually just crashes that one process serving that one request.

Philosophically, using erlang is trading consistency for availability. Given that it's easy to get good availability using normal applications connected to a database, using erlang is somewhat of a niche.

I have spent some effort learning erlang, and I really like it in many ways. I bought the new book and I like it so far. I have no problem with the syntax and I find it enjoyable to write in (though I haven't written any large programs). There are certain projects where I think erlang would be a great choice, like management and control of a cluster system. Obviously it works for telco-like things, too, but I've never developed anything like that.

reply

---

stdbrouw 1 day ago

As an everyday language, it seems too different to get used to. As a language you learn to expand your horizons, it's not weird/different enough when compared to e.g. Haskell and Lisp.

Also, Erlang has long had an excellent reputation for performant, parallelizable code, but Go and node.js have stolen some of their thunder on that front. (I'm talking purely marketing-wise, I don't know enough to compare them technologically speaking.)

I'm not sure if "no platform / no package manager" really matters.

People first learn a language and get excited about it, and then if something irks them, they'll scratch that itch. It's like that famous de Saint Exupéry quote: "If you want to build a ship, don't drum up the men to gather wood, divide the work and give orders. Instead, teach them to yearn for the vast and endless sea."

The node.js ecosystem was absolutely awful at first, and the deployment story really only got fixed about a year or longer after the first release of NPM.

reply

---

eliah-lakhin 1 day ago

In my opinion, there are three main issues:

1) The Language actually is not general purposes. Or at least it is not promoted this way. If you really need Actor based model and lightweight threads, you can choose Scala/Akka, that is also well suitable for a wide range of different objectives.

2) Standard library(I mean OTP) is simply ugly.

3) Language's syntax is ugly too. Pascal again? No, thanks. :) I know there is Elixir that fixes most of the issues, but too few people know about it outside of the Erlang community. When someone mention Erlang he/she probably thinks about "vanilla" Erlang, which is ugly again.

P.S. I had been using Erlang for a while and then stopped.

reply

---

etrepum 1 day ago

I found Erlang a lot easier to learn than Haskell. I don't really find them to be very similar, especially because they have a completely different evaluation strategy. The syntax and semantics are also quite different. Specifically, Haskell's non-strict (lazy) evaluation turns everything inside out so you tend to write many things in a different way than you would in Erlang.

reply

---

room271 1 day ago

I like a lot of things about Erlang. But the main reason I would be wary of using it is that it seems like an all-or-nothing kind of thing. The distributed actor model effectively means Erlang everywhere. This contrasts with RPC, queues, http, etc. where a variety of technologies (and languages) can be combined together. And an all-in approach seems laden with risk - all devs and relevant technologies need to fit in with the Erlang stack.

And if you are not using distributed actors, then why bother with Erlang at all?! (It's a nice language to be sure, but there are lots of nice functional languages available so the competition is pretty fierce).

reply

---

pessimizer 1 day ago

Tons of people mention Elixir in threads like these. Is anybody actually using it in production, or is it just something that Rubyists use as a foil to criticize Erlang's syntax?

I feel the algol-ization of (functional, prototypical)javascript is a confusing mess and a weakness that makes js harder to use. Making Erlang look object-oriented just seems awkward.

Syntax: comma means AND, semicolon means OR, period means "done." Is that so crazy? There's no comparison to how badly Perl does my head in.

reply

bitwalker 1 day ago

I've started using Elixir on my projects over the last few months, and I absolutely love it. It's not just the cleaner syntax, the language adds features on top of what Erlang already provides, and provides easy interop between the two languages, the documentation is better, the build tool (mix) is great, and I think the community is awesome.

It's a new language, and it won't be 1.0 until later this year, but if you're willing to live on the edge a bit, I think the rewards are well worth it.

reply

rvirding 1 day ago

The erlang syntax is actually much simpler than Elixir's which both contains more syntax and alternate ways of writing things.

reply

---

ssmoot 1 day ago

I found learning Erlang difficult since pattern matching is poorly justified in the material I came across. You don't see the benefit right off the bat.

Also, the "processes only" model is pretty off-putting compared to Scala IMO where you're able to choose between threads and processes. You lose _some_ benefit in reliability (I assume, since it's said so often), but I've never seen that actually play out in the real world. What I have seen is comparably slow IPC exactly a real world development cost. So I'm personally more of a "in-process first" kinda guy.

I may be completely wrong on that one BTW. It's just what I recall from reading half of some Erlang book and studying online material trying to pick up the language.

It wasn't until Scala and Akka that the benefits of both pattern-matching and actors really clicked with me. Though I still prefer Scala's versions on both counts so I don't feel a great need to revisit Erlang at this point since I'm not working with Telco equipment. ;-)

reply

--

_halgari 1 day ago

Before finding Clojure I dabbled in Erlang. In the end I found the "share nothing" model to be too limiting. Sure its great for highly fault-tolerant systems. But the fact is I just don't need that most of the time. It's just easier to setup a AWS autoscaling cluster of web servers running Clojure and be done with it.

Oh yeah, and Clojure beats the pants off Erlang when it comes to performance. Even Erjang is faster than stock Erlang.

reply

waffle_ss 1 day ago

> Clojure beats the pants off Erlang when it comes to performance

Source?

reply

_halgari 1 day ago

http://benchmarksgame.alioth.debian.org/u32/benchmark.php?te...

But I actually meant in general. Having real strings and good floating point support helps performance quite a bit.

reply

--

twunde 17 hours ago

I've actually considered learning Erlang, but never gone through with it as other languages seem more useful and/or enjoyable.

For me to seriously consider learning a new language/framework it should have:

Good documentation. Python is the gold standard. Good SEO for that documentation. Multiple good FREE learning resources, preferably with something useful being built. Good tooling (package manager, testing, etc)

Most importantly there should be a reason to learn it. For Node, I get easy asynchronous code. Clojure I get an enterprise-acceptable lisp. Go I get C-like performance + concurrency in an enjoyable language with modern tooling. Hell, learning Java allowed me to program for the Android.

Erlang? Well it's got some great concurrency and it's highly available. But I don't know anyone using it (and when I hear about it, it's usually because someone has stopped using it), it has a high learning curve and most of what makes it interesting can be found elsewhere.

reply

--

bfrog 1 day ago

It has a huge learning curve compared to many other languages.

Ask the simple question of how do you write the equivalent of

int main(....) { ... } and well... there isn't one!

You are almost forced, from the start, to learn about releases and a ton of other really complicated stuff just to write a program you can share. Its sort of being fixed with relx.

Secondly no one seems to understand pattern matching at first. Its just such an alien concept when you look at it compared to all the other mainstream procedural/oo languages people usually already know. Again a huge cliff to climb to really grasp the possibilities and usage.

It really is quite a great language, and an even better runtime. The learning curve to making great things with it is just quite high in my opinion. I've done two major projects with it, one shipped as an embedded web app in some lab equipment out there. I think it was a good choice!

reply

seancribbs 1 day ago

1. !/usr/bin/env escript

main(_) -> io:format("hello, world").

reply

bfrog 1 day ago

Fair enough, but thats not how the books and documentation present erlang. They present it with the shell and c()

reply

--

Also, the learning curve: I took me half a day to be good enough in Python and years to learn the basics in Erlang. It is way more complex and the standard library is plagued with inconsistent/messy interfaces. (They say it is because of legacy code support or other BS.) But under the ugly surface lies the hidden beauty: single assignment is great when reading other people's code. Patching several running servers without stopping with one command is thrilling. Or the built-in nice abstractions for distributed programming like rpc:multicall [2].

-- https://news.ycombinator.com/item?id=7280054

--

9999 1 day ago

I've actually always wanted to use Erlang and I've worked on quite a few projects where it probably would have been the best solution possible (massively concurrent websocket backends for example). The two things that made Erlang a non-starter on those projects were:

1) It's hard to find people that have production experience with Erlang 2) The perception is that batteries are not included (even in comparison with Go... if you have some information to refute that notion I would be more than happy to learn more)

reply

swvist2 1 day ago

I have tried it and (try to) use it when appropriate. I am fairly comfortable with it and my only disappointment is that I do not get to use it often.

The ability to solve a problem in a particular programming language is often dependent on the ability to express the problem in the constructs provided by the language. A very good example is the problem of concurrency. Doing something concurrently in a primarily procedural/OOP language often seems hacky and it is something that does not feel natural. The actor model that erlang implements is a very powerful model for certain class of problems and concurrency is one of them. Erlang the language, minus the OTP, is extremely small and once you get over the culture shock experienced when it comes to syntax, things will seem pretty easy. OTP isn't magic. Its years and years of erlang experience packaged into one neat library, solving commons problems, so that you can concentrate on your work instead of reinventing the wheel.

Reading Joe Armstrong thesis 'Making reliable distributed systems in the presence of software errors'[1] is highly recommended if you want to understand why things are the way they are.

[1] : http://www.erlang.org/download/armstrong_thesis_2003.pdf

reply

--

etrepum 1 day ago

Erlang requires a fair amount of boilerplate for a small project (rebar, app file, application behaviour, gen_server behaviour, etc.) and escript doesn't work very well.

It lacks a good way to do abstract data types (records don't count).

The compiler's ability to optimize is limited by the lack of purely functional guarantees and the metaprogramming facilities (parse transforms) aren't easy to use to work around that.

There's no facility like Haskell's ST or clojure transients to encapsulate mutable stuff, just a hole to write code in C and who really wants to do that? Yes, I know about the process dictionary and ets but those aren't appropriate for most algorithms I've wanted mutability for.

That said, I still use Erlang, but only in the domains where it really shines.

reply

awnird 1 day ago

Compared to other languages, Erlang has a huge learning gap between "playing around in the REPL" and "deploying an application". There's a much larger base of knowledge needed to deploy Erlang apps, than an equivalent app in another language such as Python or Java.

reply

--

seiji 1 day ago

Short answer: You have to be slightly smarter than average to use it properly.

Longer answer: The only proper way to learn Erlang is to read the documentation. Then re-read the documentation. All of it. It takes 3-6 months to get proficient, then another 6 months to stop doing things the native or outright bad way.

If you are primarily an "erlang programmer," you can't interview for jobs. Everybody hires for either java, ruby, or python. If you do see a rare "Erlang job" position, they probably actually mean "We want you to understand ejabberd because we based our company around it and it's actually unusable at scale, so you get to fix other people's problems all day long." (Of course, building your own company/services around Erlang stops the "need to interview in other languages" problem.)

But, why can't you interview in Erlang? I guess you can, but the way I work, I have an editor in one window and API docs in the other. Interviewers, sitting up there on their oh-so-high perches, don't like it when candidates want to do quick API lookups for things like parameter order or return values. (Does it take (Fun, List) or (List, Fun)? Does it return Value or {ok, Value} or {value, Value}?)

Short conclusion: Erlang is a system and understanding systems takes effort and practice. People, in general, don't want to learn, they want to do. It's the whole "one year of experience 15 times over" instead of 15 years of growth and advancement problem. It's the "person with 20 years in computing can't write a tail recursive function" problem. It's just a problem.

Alternative question: why don't people understand defmacro (and recursive defmacro) and write their programs from the bottom up?

Bonus analysis: In a world where people just want to learn one thing and use it forever, Erlang doesn't fit. With Python, you can learn it once then keep "extending" it to pretend to get concurrency and other fancy features Python actively rejects at the implementation level. So, you learn Python once, then feel productive because you're duplicating functionality given to you for free in other languages.

Erlang has so much "done right the first time" built into its VM you don't have to reinvent basic parallel computing every time you want to get two webpages at once or serve more than one client at a time from a basic five line server. But—Erlang? people know for other tasks, say something better served by numpy, they should jump over to Python or Lua or something else better suited to the task without reimplementing all of the "they did it right the first time" code in Erlang just because they refuse to learn any other language.

reply

--

mpd 1 day ago

There's a lot that I like about Erlang the language, but using it in production left a sour taste in my mouth due to difficult-to-find documentation, difficulty in testing the code, and (especially) the misery that is mnesia.

It was a poor fit with our cloud-based infrastructure, and once the mnesia database began corrupting itself weekly, requiring a full rebuild, it was an easy decision to move to another solution for what we were using it for.

reply

seancribbs 1 day ago

mnesia is indeed a frequent choice by newbies for queryable storage in Erlang, but I've found few production systems use it for similar reasons to the ones you describe. Nowadays you're more likely to see external datastores (PostgreSQL?, MySQL?, etc) used except for transient/in-memory stuff which can be stuffed in ETS.

reply

--

cordite 1 day ago

Coming from Haskell, the actor model does not seem composable. The state machine style is interesting, but the pattern that it comes out as distracts from the concept / algorithm.

This comes from reading rabbitmq and riak-core sources.

reply

--

bsder 1 day ago

Erlang isn't more popular because to be popular you need to be at least passably good at a lot of things. There is a lot of stuff where Erlang makes life hard. Operating system integration, lack of IDE, imperative programming, UTF-8 manipulation, and maps (which just hit the language) are all good examples of things that Erlang gets stomped on by most more "mainstream" languages.

That having been said, anybody who is serious about concurrent network programming knows about Erlang. The problem is that those people are far outnumbered by the people doing CRUD all the time.

reply

--

https://github.com/fenollp/kju/blob/master/criticisms.md :

No native string type.

Package manager

It isn't batteries included and doesn't appear to have a community working on correcting this. If it does, it isn't highly visible.

 it looks alien from anyone used to the C family of languages

(Foo, Bar) <- blah... ... later ... (Baz, Bar) <- blah... The third line intends to shadow Bar, but in Erlang it will attempt a match of the previously bound Bar with the right-hand side. A mismatch will result in a run-time error, which is of course a terrible time to find out that you re-used a variable name.

Syntax

making the statement separators more uniform, and making the anonymous function syntax a bit more lightweight. (Though Erlang's anonymous functions isn't terribly worse than ML or Lisp: I suppose I've been spoiled by Haskell)

no struct/map data structure

Not really open

While Erlang is an open source project, its implementation and release cycle are managed by Ericsson, the company that created it, and Ericsson just doesn't seem to care. I'm not sure what Ericsson's priorities are when it comes to adding features to Erlang, but in my opinion they're doing a worse job of engaging the community than Oracle has been doing with Java.

No clever mutability

Immutable state languages force object creation whenever anything changes. This can be partially mitigated by persistent data structures, which are able to share bits and pieces of each other because they're immutable. This works, for example, when attempting to create a sublist that consists of the last N elements of a list. But what if you want the first N elements? You have to make a new list. What if you want elements M..N? You have to make a new list. In mutable state languages, performance problems can often be mitigated by mutating local (i.e. non-shared) state instead of creating new objects. To give an example from the Ruby language, combining two strings with the + operator, which creates a new string from two old ones, is significantly slower than combining two strings with the concatenating >> operator, which modifies the original string. Mutating state rather than creating new objects means there's fewer objects for the garbage collector to clean up and helps keep your program in-cache on inner loops. If you've seen Cliff Click's crash course on modern hardware, you're probably familiar with the idea that latency from cache misses is quickly becoming the dominating factor in today's software performance. Too much object creation blows the cache.

The standard library is inconsistent, ugly, and riddled with legacy

Should module names in the standard library be plural, like "lists"? Or should they be singular, like "string"? Should we count from 1, as in most of the functions found in things like the lists module, or should we count from 0 like the functions found in the array module? How do I get the length of a list? Is it lists:length/1? No, it's erlang:length/1. How do I get the Nth element of the tuple? Should I look in the tuple module? Wait, there is no tuple module! Instead it's erlang:element/2. How about the length of a tuple? It's erlang:tuple_size/1. Why is the length of a list just "length" whereas the length of a tuple is "tuple_size"? Wouldn't "list_length" be more consistent, as it calls out it works on lists? When we call erlang:now() to get the current time, it returns {1311,657039,366306}. What the hell does that mean? It's a tuple with three elements. How could time possible need three elements? A quick look at the documentation reveals that this tuple takes the form {Megaseconds, Seconds, Microseconds}. Separating out Microseconds makes sense... Erlang has no native decimal type so using a float would lose precision. But why split apart Megaseconds and Seconds? Once upon a time Erlang didn't support integers large enough to store the combination of Megaseconds and Seconds, so they were split apart. The result is a meaningless jumble of three numbers, which you have to run through the confusingly named calendar:now_to_local_time/1 function to get a human meaningful result, which doesn't tell you what time it is now, but instead takes the tuple that erlang:now/0 returns as an argument and will spit back meaningful {Year, Month, Day} and {Hour, Minute, Second} tuples

No strings

Erlang has no way of differentiating lists of integers that represent strings from lists of integers that are actually lists of integers. much of the tooling and string functions are designed to work with list-based strings. To leverage these functions, you have to convert a binary to a list before working with it.

Syntax

caommas, ifs Warts

strings, records, single assigment even though SSA Code Organization

The only code organization offered is the source file module, there are no classes or namespaces. I'm no OO fanatic (not anymore), but I do see the real value it has: code organization. Every time time you need to create something resembling a class (like an OTP generic process), you have to create whole Erlang file module, which means a whole new source file with a copyright banner plus the Erlang cruft at the top of each source file, and then it must be added to build system and source control. The extra file creation artificially spreads out the code over the file system, making things harder to follow. What I wish for is a simple class facility. I don't need inheritance or virtual methods or static checking or monkey patching. I'd just like some encapsulation, the ability to say here is a hunk of data and you can use these methods to taste the tootsie center. That would satisfy about 90% of my unmet project organization needs. Uneven Libraries and Documentation

Most of core Erlang is well documented and designed, but too many of the included modules are buggy, overly complex, poorly documented or all three. The Inets httpd server we've found incredibly frustrating to use in CouchDB? and are discarding it for a 3rd party Erlang HTTP library. The XML processor (Xmerl) is slow, complicated and under documented. Anything in Erlang using a GUI, like the debugger or process monitor, is hideous on Windows and pretty much unusable on OS X. The OTP build and versioning system is complicated and verbose and I still don't understand why it is like it is. And crufty. I know Erlang has been evolving for real world use for a long time, but that doesn't make the cruftyness it's accumulated over the years smell any better. The coding standards in the core Erlang libraries can differ widely, with different naming, argument ordering and return value conventions. It's tolerable, but it's still there and you must still deal with it.

Another vast and poorly lit area is that of exceptions. Today, there is seldom any way of knowing what sorts of exceptions a function can throw, and the exceptions are usually uninformative (e.g., "badmatch" when you have half a dozen pattern matches in the indicated function). Thus, using the exit reason to recover from an error is difficult. In practice, you print, log or ignore an exit today.

-- https://github.com/fenollp/kju/blob/master/criticisms.md

--

http://www.howistart.org/posts/erlang/1

--

SkyMarshal? 175 days ago

Agreed, in fact what's surprising is that anyone thinks it's surprising that the JVM with its billions of man hours and primary usage in mission critical finance/banking/etc is anything but one of the most performant, scalable, robust, well-tooled platforms around.

>Aside from terrific performance, the JVM gives you the best concurrency platform out there (though not the easiest to use)

Though I'd probably still reserve that accolade for Erlang OTP/BEAM.

yummyfajitas 175 days ago

Though I'd probably still reserve that accolade for Erlang OTP/BEAM.

I would absolutely not. Erlang/OTP/BEAM might be the best distributed computing platform (though Akka is making huge inroads), but Erlang is actually a pretty terrible at single-box concurrency.

Erlang has one concurrency primitive - message passing. Which is great, but it's not always the right one.

Consider a a producer process which solves a PDE or does a big Bayesian computation (output is an Array[Float] or Array[Double]). You have consumer processes which read from the array and compute sums over subsets of the elements.

In Erlang you are just passing around copies of arrays. In the JVM, you'll probably have a single mutable array. You'll either wrap the appropriate calls in arr.synchronize {...}. Or you can use memory barriers and let the consumers freely read from the array, and invalidate their computation if the producer wrote to the array between the start and end of their read.

http://www.jerf.org/iri/post/2930

tel 6 days ago

It would be interesting to see what it takes to get this functionality (at least locally) into Haskell given that there are asynchronous exceptions... even ones with arguably greater safety than their Erlang compatriots.

A similar effort is happening with the Cloud Haskell program, but my understanding is that they're pouring a lot of effort into transmitting arbitrary Haskell functions over the wire between computers. This is pretty unnecessary for supervision and "Let it Crash"-style error handling alone.

reply

jeremyjh 6 days ago

That is a core primitive in Cloud Haskell but not the only one - you also send and receive data as in Erlang. distributed-process-platform provides Erlang style supervisors, gen_server, etc. It's all there and very much captures the features and spirit of OTP in a powerful static type system.

reply

tel 5 days ago

I think I just missoveremphasized the importance of a particular paper on it that I read.

reply

thinkpad20 6 days ago

Oh, that's a bit disappointing. I thought Cloud Haskell was focused on developing actor-pattern abstractions like lightweight, concurrent threads and fault tolerance. Serializing functions seems like a real red herring.

reply

jeremyjh 6 days ago

You can totally do that in Cloud Haskell - serializing functions can be useful too but it's not the draw for me.

reply

exo762 5 days ago

Somewhat different, but there is an attempt to implement BEAM in Haskell: https://github.com/gleber/erlhask

This problem (sync/async) is one of the things that is being solved in this project.

reply

Addendum: Why Go Will Never Have Asynchronous Exceptions

As mentioned above, Erlang allows you to remotely kill a target process. This is accomplished with the exit function, which throws an asynchronous exception into the target PID, including the possibility of throwing an uncatchable exception in that forces termination, much like kill -9 in UNIX.

To understand why these are called "asynchronous", you have to look at the exception from the point of view of the process receiving the exception; it is not the usual sense of the term that programmers are used to. Most exceptions are synchronous, in that they either occur or don't occur at a particular point in the program; they are "synchronous" (in its original meaning of at the same time as) with the code that produced them. For instance, if your language is going to throw an exception for "file not found", it will occur when you try to open it, not at a random time. By contrast, from a thread's point of view, an "asynchronous exception" can occur at any time, and from the thread's accounting of time, it is completely unrelated to anything it is currently doing.

This is a subtle thing in the Erlang paradigm; since a process shares no state with any other process, and even most resources are held at arm's length via ports, it is feasible to asynchronously kill Erlang processes reasonably safely. The killed process will automatically drop all its values. Any resource it has open will be via a "port", which as an Erlang process, will be linked to the using process, and thus, when the using process dies, that process or port will also "die", so the killed process has a well-defined way of cleaning up its resources even when asynchronously killed. It's still not perfectly safe; some resources may leak depending on how it interacts with other threads, etc, but it is reasonably safe. In Erlang, arguably writing code that isn't safe to kill would be a bug.

Erlang gets away with this by having rigidly partitioned processes. That is, I don't think immutability enters into it; it is the rigid partitioning that accomplishes this. Languages with immutable values do have an easier time providing asynchronous exceptions, though I would observe it took Haskell several iterations to get it correct. In this case, arguably it was the laziness making it harder, but it still is not clear that a strict immutable language with shared values would have a trivial time either. However, it is a flamingly bad idea to have asynchronous exceptions in a shared-state mutable language, and despite the fact that Go uses convention to try to avoid sharing state, it is a shared-state mutable language.

It is not possible to program correctly in a mutation-based language when an "asynchronous exception" can happen at any time. In particular, you do not know what operation the thread was in the middle of that was never supposed to be observed; for instance, if the goroutine was in the middle of a critical section protected by a mutex, it is possible to clean up the mutex while the goroutine is dying, but there's no way to roll back anything the goroutine half did. There's a lot of other more subtle issues that arise, too. For instance, trying to protect a goroutine with a top-level defer doesn't prevent asynchronous exceptions from ruining your day... what if you get an asynchronous exception in the middle of the deferred function itself? Code that is safe in a world without asynchronous exceptions can end up bubbling a panic up past the top of a goroutine stack due to something out of the control of the running goroutine... in the current semantics, that's morally indistinguishable from a segfault, and your program terminates. Any attempts to get around that brings their own further problems. I went on for a couple of paragraphs here and deleted them due to being redundant. Thar be fractal fail here! If you feel like exploring the space yourself, remember to treat this as a hostile environment, like any other threading case. It's helpful to imagine a hostile adversary trying to find the worst possible time for your thread to experience an asynchronous exception, and remember: You can receive an arbitrary number of them, and the exception is itself important, it may be implementing some other guarantee... just ignoring it for any reason is itself a failure.

If easy answers are immediately leaping to your mind, bear in mind they've all been tried and they didn't work. This is a venerable problem faced by all the CLispScript? languages, and it's fairly well established there's no practical solution. Even Java eventually had to pull them out, and I mention Java not necessarily as the paragon of software engineering, but as a project that demonstrably has had massive efforts poured into it, and every motivation in the world to make that functionality work for reverse compatibility. If they couldn't do it, and given the fundamental nature of the problems in a mutable state language, probably nobody else can either.

Therefore, there's no point in waiting for this functionality to exist before writing a supervision tree library.

---

lucaspiller 5 days ago

Exactly. One of the big ideas of Erlang is to have many lightweight processes (processes in the VM, not OS).

As an example say you have a chat system, you will have a process for each user [0]. If you have a million users, you will end up with a million processes. The supervisor tree watches these processes, so if one crashes a single user may get logged out but everybody else will be unaffected.

[0] In reality you'll probably have it even more fine grained than that, so users will end up with multiple processes.

reply

---

wcummings 6 days ago

Why not just use Erlang?

reply

jerf 6 days ago

I'm comfortable with it, but none of my team wants to use it.

Even after 7 years of experience, it is very klunky to program in, and in reimplementing the Erlang program I had in hand, I immediately added some features that were very hard to implement in Erlang. And note the architecture of the code didn't actually change, this isn't one of those cases where the second version was also radically different than the first so no comparison is possible, it was actually hard to implement in Erlang. Erlang is a very klunky language to work in. Raw Java might be klunkier, but it's easily the klunkiest of the languages I use.

Erlang's clustering, which my code heavily depends on, just sort of breaks sometimes, on machines that are on the same switch, with every bit of hardware switched out over time. It is very opaque to figure out why it isn't working, or how to make it work again. Mnesia breaks even more often on loads that shouldn't be a problem. After 7 years of working with it, at this point even if the solution was just to turn on this one flag somewhere and all my problems would go away, I still would not consider that a positive.

I don't "hate" it, but I've been waiting for years for a more conventional language that had pervasive microthreading in it that I could use for work. Go is not quite it... I'd still like the goroutines to be completely isolated, and I've got a handful of other quibbles... but it's close enough that I might be able to get out of the hell of trying to multiprocess in most other languages, while being something I might actually be able to get adopted. In general, the company I work for has not been overwhelmed by the effectiveness of Erlang.

In practice, the biggest loss of fundamental capability is loss of true asynchronous exceptions, and, well... in practice that's only an annoyance, not an OH MY GOSH I WILL NEVER USE A LANGUAGE WITHOUT ASYNCHRONOUS EXCEPTIONS sort of thing. In most cases if you're throwing an async exception you've really already lost and now we're just discussing exactly how you're losing.

Every language student should study Erlang, and learn from it, and possibly use it. It's a solid language. And there's nothing wrong with trying to port some of that solidity elsewhere. (I'd pick a real DB over Mnesia, though.)

reply

stock_toaster 6 days ago

> In most cases if you're throwing an async exception you've really already lost and now we're just discussing exactly how you're losing.

That is a exceptionally quotable. :)

reply

tormeh 6 days ago

Have you taken a look at Akka (with Scala)? You sacrifice code hot-swapping and I think little else, but you get JVM performance and one of the most extensible non-Lisp and safe non-Haskell languages in existence.

reply

wcummings 6 days ago

Or Akka with Java

reply

tormeh 6 days ago

Yeah, but the reasons for choosing Java over Scala are mostly organizational. If your organization can tolerate Erlang it can tolerate Scala.

reply

mratzloff 6 days ago

It seems he wanted a static type system, for starters.

reply

rubiquity 6 days ago

There's a running joke in the Erlang community that every Erlang programmer will at some point try and re-implement Supervisors and do it horribly wrong. This isn't because the person is a bad programmer, it's just easy to underestimate the amount of time and incredible programming that has gone into Erlang's supervisors.

By sticking with Go, the author gets to keep the static types but likely gets faux-Supervisors at best.

reply

jerf 6 days ago

I describe exactly what I get. Whether they're "faux" depends on your definition.

No, seriously, that's really profoundly true. Erlang's definition isn't the only one. But you're free to take it. If you can bend a bit, though, these may not be "faux".

One thing that's easier when it comes to implementing supervisors in not-Erlang though is that the vast majority of languages have something better than "behaviors" to work with, which are bizarrely klunky things. I actually implemented something more like a "gen_server" initially, but it wasn't a win. Some of that "incredible programming" is essential complexity (in Fred Brooks' terminology), but some of it is accidental complexity. Plus Erlang had to build it from true, total scratch; Go is a pre-existing runtime, and "defer" is simple, yet powerful.

reply

rvirding 6 days ago

Did you try to implement your own behaviours? Or processes that aren't behaviours as such but still function correctly in supervision trees and in applications?

reply

jerf 5 days ago

What I found is that if you implement Serve() and Stop(), honestly, you're done, in the world of Go. If you want a state machine, just write it. Many of the things I'm supervising are goroutines that wrap an API around a channel that implements a server.

reply

polskibus 6 days ago

While not exactly static typing, you can add type annotations in Erlang and use dialyzer as a step in compilation to catch typing bugs.

reply

jerf 6 days ago

In the spirit of rubiquity's post, I'll take a faux supervisor tree and static typing over a "real" supervisor tree (sorry, gotta scare quote it) and faux static typing.

It should be pointed out specifically that Erlang's type system is incredibly, annoyingly weak, more so than merely "weak typing". It's a language that doesn't permit any sort of user-created type whatsoever. There are good reasons for this which would be its own interesting blog post, mostly related to the need for a clear serialization for cluster communication, but I think those good reasons have been superceded by later work in the general programming world on serialization of objects and the tradeoff is wrong in 2014. (But, no sarcasm, a good one when it was created. It is no real criticism of a language to observe that it couldn't use lessons learned only a decade after it was created.)

reply

davidw 5 days ago

Here's a recent Erlang type annoyance:

Given two tuples with 3 integers, {X, Y, Z}, without knowing it a priori, you can't tell which one is an erlang:now() and which is an erlang:date().

reply

---

"Rust is inspiring for many reasons. The biggest reason I like it is because it's practical. I tried Haskell, I tried Erlang and neither of those languages spoke "I am a practical language" to me. I know there are many programmers that adore them, but they are not for me. Even if I could love those languages, other programmers would never do and that takes a lot of enjoyment away. " -- http://lucumr.pocoo.org/2014/10/1/a-fresh-look-at-rust/

" ...

the Erlang language and libraries exploit a few basic data types (tuples and lists being composed into proplists and iolists) very well, to the point that a lot of practical Erlang code is simply shuffling data by pattern matching on those types. It's not a Lisp-level code-data correspondence, but it's in a similar league.

reply

jerf 13 hours ago

...

Erlang has no user-definable data structures. I don't recall anything I've read explicitly stating this, but I've believed for a while that this was the case because it made moving terms across the network simpler. Thus, libraries aren't "exploiting" a few basic data types... they're stuck with a few basic data types. There's no alternative.

That really has nothing to do with 'homoiconicity'. Erlang code is not a data structure, and it is not spelled in Erlang terms. And that would generally not be a good thing! Erlang's already plenty verbose without also having to be expressed as legal Erlang data terms.

(The generally loosy-goosy typing that results is one of the reasons I don't like Erlang and am moving away from it. But YMMV.)

reply

"

---