notes-computer-programming-programmingLanguageDesign-prosAndCons-erlang

good post: http://www.unlimitednovelty.com/2011/07/trouble-with-erlang-or-erlang-is-ghetto.html

--- the ingredients of Erlang: " Fast process creation/destruction Ability to support » 10 000 concurrent processes with largely unchanged characteristics.

A programming model where processes are lightweight values -- and a good scheduler -- make concurrent programming much easier, in a similar way to garbage collection. It frees you from resource micro-management so you can spend more time reasoning about other things.

    Fast asynchronous message passing.

This is what ZeroMQ? gives you. But it gives you it in a form different to that of Erlang: in Erlang, processes are values and message-passing channels are anonymous; in ZeroMQ?, channels are values and processes are anonymous. ZeroMQ? is more like Go than Erlang. If you want the actor model (that Erlang is based on), you have to encode it in your language of choice, yourself.

    Copying message-passing semantics (share-nothing concurrency).

Notably, Erlang enforces this. In other languages, shared memory and the trap of using it (usually unwittingly) doesn't go away.

    Process monitoring.

Erlang comes with a substantial library, battle-tested over decades, for building highly concurrent, distributed, and fault-tolerant systems. Crucial to this is process monitoring -- notification of process termination. This allows sophisticated process management strategies; in particular, using supervisor hierarchies to firewall core parts of the system from more failure-prone parts of the system.

    Selective message reception.

" http://www.rabbitmq.com/blog/2011/06/30/zeromq-erlang/ (from http://www.zeromq.org/whitepapers:multithreading-magic ) --

davidw 11 hours ago

I'm interested in tracking Go as a replacement for Erlang. Some things that should probably happen:

• GC works on 32 bit systems.
• Scheduler improvements so that things are a bit more robust ( http://code.google.com/p/go/issues/detail?id=543 )
• A fairly solid library implementing supervision trees or something like it.
• Probably some other things, but those strike me as the big ones. The important thing is for the system to be able to run for months at a time.

I think they'll get there eventually. Maybe not 100%, but 'good enough' sooner or later.

reply

jerf 7 hours ago

Right now, as near as I can tell, you basically can't implement an Erlang supervision tree in Go. You don't have anything like linking, which really has to be guaranteed by the runtime to work properly at scale, so bodging something together with some 'defer' really doesn't cut it. You also can't talk about "a goroutine", because you can't get a reference or a handle to one (no such thing), you can only get channels, but they aren't guaranteed to map in any particular way to goroutines.

I've done Erlang for years and Go for weeks, so I'm trying to withhold judgement, but I still feel like Erlang got it more right here; it's way easier to restrict yourself to a subset of Erlang that looks like a channel if that is desirable for some reason than to implement process features with channels in Go.

reply

JulianMorrison? 8 hours ago

That issue is quite simply a misinterpretation of goroutines.

Erlang: perform a maximum number of reductions, then switch, or switch on IO. The number of reductions is cleverly adjusted so that a process which is swamping other processes will be throttled.

Go: switch on IO.

Go's design is much simpler, and closer to Ruby Fibers than to Erlang processes, except that goroutine scheduling can use multiple threads. To cooperatively switch without doing IO, call runtime.Gosched().

reply

trailfox 10 hours ago

Akka is also a viable Erlang alternative.

reply

waffle_ss 6 hours ago

Looks like a nice library but I don't think it's a serious contender to replace Erlang because the JVM just isn't made for the level of concurrency that Erlang's VM is. Off the top of my head as an Erlang newbie:

• You're still using Java threads, so the potential to access [shared memory][1] is there (i.e. "shared nothing" can't be guaranteed).
• You're still subject to global GC pauses, whereas the Erlang VM has per-process GC.

[1]: http://doc.akka.io/docs/akka/snapshot/general/jmm.html

reply

An Erlang process has its own heap, so when it blows up, its state just goes away, leaving your remaining program's state untouched. With Go, there is no way to recover sanely; even if your goroutines are designed to copy memory, Go itself has a single heap.

Now, this is a very odd design decision for a language that claims it's designed for reliability. Perhaps Go's authors thinks it's better just for the entire program to die if a single goroutine falls over; well, that's one way, but it's a crude one. Erlang's design is simply better.

I wonder if Go can ever adopt per-goroutine heaps, or whether it's too late at this stage. I was happy to see that Rust has chosen to follow Erlang's design by having per-task heaps, even if all the pointer mechanics (three times of pointers, ownership transfer, reference lifecycles and so forth) result in some fairly intrusive and gnarly syntax.

reply

masklinn 1 day ago

> Go allows you to share memory between goroutines (i.e. concurrent code).

Go will share memory, by default, and special attention must be taken preventing or avoiding it. It's not an allowance.

> In fact, the Go team explicitly tells you not to do that

And yet they have refused to implement a correct model, even though they have no problem imposing their view when it fits them (and having the interpreter get special status in breaking them, see generics).

reply

burntsushi 1 day ago

> Go will share memory, by default, and special attention must be taken preventing or avoiding it.

Not really. If you use channels to communicate between goroutines, then the concurrency model is that of sequential processes, even if channels are implemented using shared memory under the hood.

That is, the default concurrency model militated by Go is not shared memory, but that of CSP. It's disingenuous to affix Go with the same kind of concurrency model used in C.

> And yet they have refused to implement a correct model

What is a correct model? Erlang's model isn't correct. It's just more safe.

> (and having the interpreter get special status in breaking them, see generics)

What's your point? Purity for purity's sake?

reply

masklinn 19 hours ago

> Not really. If you use channels to communicate between goroutines, then the concurrency model is that of sequential processes

Except since Go has support for neither immutable structures not unique pointers, the objects passed through the channel can be mutable and keep being used by the sender. Go will not help you avoid this.

> That is, the default concurrency model militated by Go is not shared memory, but that of CSP. It's disingenuous to affix Go with the same kind of concurrency model used in C.

It's not, go passes mutable objects over its channel and all routines share memory, you get the exact same model by using queues in C.

Erlang's core concept of concurrency seems like something that'd be better suited as a library and app server than a whole language and runtime.

I've yet to hear of any Erlang-specific magic that cannot be implemented inside another language.

reply

SoftwareMaven? 1 day ago

How would you get per-actor heaps that cannot be violated by other actors? That is critical to Erlang's ability to recover from processes dying. I spent a lot of time doing Java and can't think how you could (you could in the JVM if you had language constructs for it, but then we are back to a new language).

There's a reason Stackless Python's actors aren't just a library on top of Python.

reply

reeses 11 hours ago

TLABs are a partial step in that direction at the JVM level. You could also use object pools/factories keyed to the thread.

Those are the first two "ghetto" hack solutions I can think of that wouldn't require significant code changes on a going-forward basis.

reply

SoftwareMaven? 7 hours ago

But those hacks wouldn't provide the same guarantees that language-level changes provide. Sure, you can try not to impact other thread's heaps, but nothing is stopping me, which means a simple programming error has the potential to impact multiple threads. As a result, you can't just "reboot" that thread (a critical piece of what makes Erlang interesting), because you have no guarantees its errors didn't impact other threads. You also have no guarantees that the underlying libraries aren't mucking up all of your carefully crafted memory management.

It's like the kernel protecting memory so applications can't overwrite each other. Sure, applications could just write to their own memory, but nobody actually trusts that model[1]. Instead, they want something below that level enforcing good behavior.

reeses 11 hours ago

Obviously, you can do the same things in Java, as people have demonstrated with alternative languages that target the JVM.

It's the expressiveness at the language level that is really the "magic". For example, doing the equivalent of OO is not intuitive in Erlang, but completely possible (actually easy, but it looks...wrong) whereas it's supported by every Java tool. By the same token, pattern-matched message passing, lightweight green threads, and hot code deployment are primary concepts in Erlang.

reply

dumael 12 hours ago

You can at best implement a mimicry of Erlang's message passing in Java.

With sufficient effort, you can have the equivalent of no shared mutable data.

What you cannot have is completely separate heaps, so that if one thread crashes for whatever reason it doesn't take your application down.

Also, good luck trying to find a garbage collector that supports completely separate heaps that isn't a direct copy/near-identical implementation of the BEAM[1] VM's GC.

[1] The virtual machine that is the stock VM for Erlang. (In fact I don't know of any others but I have never looked.)

reply

reeses 11 hours ago

We've been doing it in Java since the nasty old days of RMI or "our own weird RPC over HTTP implementation". Lots of lightweight services with a supervisor to manage them and a directory to find them.

Now it's even easier with ESBs. Write a ten-line grails service to expose a bin-packing facility with Drools and then never touch it again.

reply

Evbn 1 day ago

Java's syntactic overhead stops most humans. "final" everywhere, big method names for every primitive message passing operation, etc

reply

---

seanmcdirmid 1 day ago

Erlang is not designed for parallel programming; it is designed for concurrent programming. These are two very different programming domains with different problems.

Every time someone conflates parallelism with concurrency...everyone gets very confused.

reply

unoti 23 hours ago

Isn't it really fair to say that it's designed for both? The way it uses immutable state and something-similar-to-s-expressions to express data make it very straightforward (or even transparent) to distribute work between multiple processes and separate computers, in addition to how it makes it practical and simple to break work into small chunks that can be interleaved easily within the same thread. It's really designed for doing both very well, wouldn't you say?

reply

seanmcdirmid 23 hours ago

Not at all. Erlang isn't useful for modern parallel computing as we know it, which is usually done as some kind of SIMD program; say MapReduce? or GPGPU using something like CUDA. The benefit doesn't just come from operating on data all at once, but these systems (or the programmer) also do a lot of work to optimize the I/O and cache characteristics of the computation.

Actor architectures are only useful for task parallelism which no one really knows how to get much out of; definitely not the close-to-linear performance benefits we can get from data parallelism. Task parallelism is much better for when you have to do multiple things at once (more efficient concurrency), not for when you want to make a sequential task faster.

Maybe this will help

http://jlouisramblings.blogspot.com/2011/07/erlangs-parallel...

and

https://news.ycombinator.com/item?id=2726661

reply

reeses 7 hours ago

SIMD is a specialized form of parallelism. It is not the only definition of the term.

It should also be clear that task parallelism (or concurrency from your perspective) has not had the benefit of billions of engineer-hours focused on improving its performance. It is within recent memory that if you wanted 20+ CPUs at your disposal, you'd have to build a cluster with explicit job management, topologically-optimized communications, and a fair amount of physical redundancy.

As many of the applications requiring low-end clusters tended to involve random numbers or floating point calculations, we also had the annoyance of minor discrepancies such as clock drift affecting the final output. This would present, for example, in a proportional percentage of video frames with conspicuously different coloration.

seanmcdirmid 4 hours ago

Task parallelism was something used to work on 20 years ago when we thought it was the solution to scaling. But then we found that the supercomputer people were right all along, that the only thing that really scales very well is data parallelism. So the focus in the last 5/10 years has been finding data parallel solutions to the problems we care about (say deep neural network training), and then mapping them to either a distributed pipeline (MapReduce?) or GPU solution.

> It is within recent memory that if you wanted 20+ CPUs at your disposal, you'd have to build a cluster with explicit job management, topologically-optimized communications, and a fair amount of physical redundancy.

You are still thinking about concurrency, not parallelism. Yes, the cluster people had to think this way, they were interested in performance for processing many jobs; no the HPC people who needed performance never thought like this, they were only interested in the performance of one job.

> As many of the applications requiring low-end clusters tended to involve random numbers or floating point calculations, we also had the annoyance of minor discrepancies such as clock drift affecting the final output.

Part of the problem, I think, is that we've been confused for a long time. Our PHBs saw problems (say massive video frame processing) and saw solutions that were completely inappropriate for it (cluster computing). Its only recently that we've realized there are often other/better option (like running MapReduce? on that cluster).

reply

seanmcdirmid 15 hours ago

Yes, erlang is great for concurrency, GPUs are great for significant scalable parallelism. They both solve different problems, I agree, and that's my point.

reply

seanmcdirmid 4 hours ago

> but MIMD is more easily applied. Almost any application can be refactored to spawn lightweight threads for many calculations without any explicit knowledge of the π-calculus.

MIMD hasn't been shown to scale, and its not locking that is the problem, but I/O and memory.

> To your point and for a well-understood example, make -j doesn't always result in faster compilations. It may if you have the ability to leverage imbalances in CPU and storage hierarchies, but you can also kill your performance with context switches (including disk seeking).

When Martin Odersky began pushing actors as a solution to Scala and multi-core, this was my immediate thought: the Scala compiler is slow, can this make it go faster? It was pretty obvious after some thought that the answer was no. But then we have no way yet of casting compilation as a data-parallel task (a point in favor of task parallelism, but it doesn't help us much).

reply

---

And in a lot of ways Erlang sucks, too. The syntax is outdated and stupid (Prolog lol), it has weird type coercion, memory management isn't handled that well (and many more). Of course, since Facebook uses it, people think it's a magic bullet (Erlang is to Facebook like Python is to Google).

> How is it any more or less stupid than curly bracket.

When you move a few lines around to refactor, you have to pay a lot of attention to the "ant turd token" line endings. I like Erlang, but still find this aspect of it annoying.

--- " However, in my opinion separate heaps was absolutely 100% the correct design decision. The main benefit is not having to worry about the effects of a long-running, stop-the-world garbage collection, which can have catastrophic effects on user interaction, server response times, request queue lengths, etc. An additional benefit is that the language implementers can use "dumb" algorithms for garbage collection and avoid a large class of difficult-to-track bugs.

"

There is some magic in how it has separate heaps and how it maps schedulers n:m (n cpus say 2 to 24 to m processes say 2 to 300k), how it provides concurrent garbage collection without stopping the world, how it provides hot code reloaded if that is what you need.

Back to your point though, I think once you understand the language it is actually a lot simpler to understand what is going on. Modules are usually very self contained, and you don't get the layers upon layers of indirection you see in other frameworks (I'm looking at you Rails). I think the functional style of programming as well helps to keep things simple, it doesn't make sense to have a 20 line function in Erlang.

We used Erlang several years ago. The code base has ~100k lines of code so it should be representative. We abandoned it later and switched to C++ because of performance (mostly in mnesia) and quality issues (some drivers in OTP). We didn't expect too much from performance considering it is functional (which seldom does in place update) but it is still below expectation.

It is understandable though. Just think about how much resources have been put into development of Erlang VM and the runtime/libraries(OTP), and compare it with JVM/JDK. There is just no magic in software development. When talking about high concurrency and performance, the essential things are data layout, cache locality and CPU scheduling etc for your business scenario, not the language.

reply

acqq 1 day ago

The tests here confirm your experience, Erlang is for many algorithms significantly slower, even when more cores are used:

http://benchmarksgame.alioth.debian.org/u64q/erlang.php

reply

artsrc 1 day ago

Excel solves this right problem, and Erlang does not.

Erlang allows you to create concurrent programs, i.e.: programs where the result is schedule dependent.

One right problem is allowing people to write deterministic parallel programs. This gives you the speed (from parallel) with the reliability (from deterministic).

http://joearms.github.io/2013/05/31/a-week-with-elixir.html

https://news.ycombinator.com/item?id=5798049

--

" Anthony EdenAugust? 19, 2013 at 7:08 AM

I use both Go and Erlang, depending on my needs. I prefer Erlang's design and I adore OTP, however I'll be the first to admit that learning to think in Erlang was significantly more challenging than learning to think in Go, the latter being very similar to almost every other imperative language I've already used. Additionally when it comes to resource usage Go is pretty damn amazing (using very little CPU and memory compared to a lot of other languages I've used, like Ruby and Java) whereas the Erlang VM tends to use quite a bit of both RAM. "

--

"

zaphar 2 days ago

(full disclosure: I work at google and also like erlang)

Erlang has fantastic facilities for robustness and concurrency. What it does not have is type safety and it's terrible at handling text in a performant fashion. So if you don't care about either of those things and only care about robustness and concurrency then Erlang is great. There were internal discussions about Erlang here but the upshot was. We had already basically duplicated Erlangs supervision model in our infrastructure, only we did it for all languages and Erlang didn't offer any benefits in performance for us. It's only benefit would have been the concurrency model. That's much less benefit than Go gives.

Go gives you Erlangs concurency model, a similar philosophy of robustness, type safety, all batteries included, and performance. Equating the two languages works in 1 or 2 dimensions but not on all the dimensions google cares about.

reply

derefr 2 days ago

Interesting, thanks for that; it's pretty much what I guessed (especially the bit about the supervision tree and hot-code-upgrade advantages being mooted by your infrastructure.)

On a tangent, though:

> What it does not have is type safety

I've tried to work this out before (I'm designing a new language for Erlang's VM), but as far as I can tell, type safety is in practice incompatible with VM-supported hot code upgrade.

If you have two services, A and B, and you need to upgrade them both, but you can't "stop the world" to do an atomic upgrade of both A and B together (because you're running a distributed soft real-time system, after all), then you need to switch out A, and then switch out B.

So, at some point, on some nodes, A will be running a version with an ABI incompatible with B. In a strongly-typed system, the VM wouldn't allow A's new code to load, since it refers to functions in B with type signatures that don't exist.

On the other hand, in a system with pattern-matching and a "let it crash" philosophy, you just let A's new code start up and repeatedly try-and-fail to communicate with B for a while, until B's code gets upgraded as well--and now the types are compatible again.

It's an interesting problem.

reply

laureny 2 days ago

> type safety is in practice incompatible with VM-supported hot code upgrade.

That's not true.

First, it's very easy to hot reload changes that have been made to the code that are backward compatible. The JVM spec describes in very specific details what that means (adding or removing a method is not backward compatible, modifying a body is, etc...).

This is how Hotswap works, the JVM has been using it for years.

As for changes that are backward incompatible, you can still manage them with application level techniques, such as rolling out servers or simply allow two different versions of the class to exist at the same time (JRebel does that, as do other a few other products in the JVM ecosystem).

Erlang doesn't really have any advantages over statically typed systems in the hot reload area, and its lack of static typing is a deal breaker for pretty much any serious production deployment.

reply

rdtsc 1 day ago

> lack of static typing is a deal breaker for pretty much any serious production deployment.

Are you talking about Google only where they made it a mandate or in general? There are serious production deployments on Python, Ruby, Erlang and Javascript.

I will trade expressiveness and less lines of code with a strong but dynamically typed language + tests over more a static typed language with more lines of code all being equal.

Or put it another way, if strong typing is the main thing that protects against lack of faults and crashes in production, there is a serious issue that needs to be addressed (just my 2 cents).

reply

derefr 2 days ago

> As for changes that are backward incompatible, you can still manage them with application level techniques, such as rolling out servers or simply allow two different versions of the class to exist at the same time (JRebel does that, as do other a few other products in the JVM ecosystem).

Neither of these allow for the whole reason Erlang has hot code upgrade in the first place: allowing to upgrade the code on one side of a TCP connection without dropping the connection to the other side. Tell me how to do that with a static type system :)

reply

pmahoney 1 day ago

Tomcat (and other app servers) has support for doing hot reloads of Java web apps while not reloading the HTTP layer (and not dropping TCP connections).

http://www.javacodegeeks.com/2011/06/zero-downtime-deploymen...

I have implemented a similar system for JRuby apps running inside a Servlet container. There are many caveats. I don't actually recommend it because for a while you're using nearly twice the memory (and JRuby is particularly memory hungry). Also there are many ways to leak the old class definitions such that they are not GC'd (e.g. thread locals). But it's certainly possible.

I suspect that Erlang, Java, and all languages are in the same boat: some parts can be upgraded live in the VM while other parts require a full restart (maybe coordinating with multiple nodes and a load balancer to achieve zero-downtime).

reply

banachtarski 21 hours ago

Erlang is not in that boat. Generally, you can upgrade the entire thing (except the lowest level libraries obviously) without too much fuss.

reply

lenkite 1 day ago

Out of curiosity, where/why would such an exotic feature be needed in today's internet architectures where you always front a group of servers with a load balancer ?

reply

banachtarski 21 hours ago

Not everything is a website! Also, not everything is stateless. Consider writing a chat application for the web for example and letting users on one page communicate with another one.

reply

butterflyhug 1 day ago

Not all Internet protocols are HTTP. If you're running a service where long-lived connections are the norm, "simply fronting a bunch of servers with a load balancer" can require a pretty smart load balancer. E.g. IMAP connections often last hours or even days, and are required to maintain a degree of statefulness.

reply "

--

DanWaterworth? 2 days ago

Go gives you Erlangs concurency model

There are a number of significant differences between Erlang's and Go's concurrency models: Asynchronous vs synchronous communication, per-thread vs per-process heaps, send to process vs send to channel.

reply

zaphar 2 hours ago

Go has asynchronous communication and synchronous communication.

And the other things you mention are in practice not significant differences. The model both use is based on Hoares CSP and the same general ways of using apply. Some of the specifics must accomodate differences but those are differences of implementation not the general model.

reply

--

" It is too bad though. Erlang, I think, has some features I like better such as hot code reloading, better supervision strategies, separate heap per lightweight process (hence no stop-the-world GC), and is battle tested for longer. Also, at least to me, actors with pattern matched receive, as central building blocks, makes more sense than channels. So I'll keep Erlang as my concurrent server side go-to language for now. "

--

AYBABTME 2 days ago

When people ask «Why not Erlang instead of Go, Erlang is X and Y and Z...» they seem to be oblivious to the fact that Go is C-like and Erlang has a pretty weird Prolog-like syntax.

I've had some experience with Prolog before touching any Erlang. It's not a syntax or a programming style that I liked, not at all. When I came to do some Erlang [1], I found the same style and it was not a pleasant surprise.

I learned C-like languages first, so maybe that's why my view is flawed. But most people also learn C-like languages as their first language, so it might be that Erlang looks like an ugly beast when they come across it. And thus as a concurrent language, Go seems to be first of it's kind.

Meanwhile, Go has a very simple style that pretty much everybody can read out of the box.

[1]: CS Games in Canada, one challenge was a 'debugging' competition where programs in ~10 languages had bugs we had to find and fix in 90 mins. One of them was in Go, another one in Erlang. Out of about 20 teams participating, me and another one (2/20) managed the Erlang one while the vast majority of the other teams managed to do the Go one. FWIW, this can speak to people's ability to understand Go vs. Erlang.

--

rdtsc 1 day ago

> they seem to be oblivious to the fact that Go is C-like and Erlang has a pretty weird Prolog-like syntax.

Yeah it has Prolog-like syntax. However, when designing and working on large distributed system looking at just the syntax is a kind of shortsighted. The problem is not syntax (which is different, and is actually pretty simple, and a lot less ambiguous than say Javascript or has less "features" than C++), the problem is _semantics_. And by that I mean structuring your programs as a set of multiple concurrent actors. That is the hard part.

Another way to put it. Erlang is probably getting looked at because someone wants to either scale, build a distributed system or a highly fault tolerant system. At that point, if dots vs semicolons is a major stumbling block what are they going to do when they hit a netsplit.

Now, Erlang like any tool has trade-offs. But those are about isolation and private heaps vs raw sequential performance. Hot code upgrades and compiled static code and pointers referenced everywhere in the code is not going to work well. Stuff like that. Single assignment is also another common one.

(And if syntax is a major stumbling block, there is Elixir or a lisp like languages LFE and Joxa that all take advantage of the actor model and run on the BEAM VM).

reply

--

lbarrow 2 days ago

One of the big projects they're working on in the next version of Go is to make the internal scheduler more Erlang-esque =)

reply

MetaCosm? 2 days ago

Yep, full preemptive scheduler, I am very excited about it!

reply

--

--

continuations 2 days ago

The author did say CPU efficiency is key, and Erlang isn't exactly known for its raw speed.

In general search is very CPU intensive, maybe that's why Google never adopted Erlang.

reply

banachtarski 21 hours ago

Yes but intentionally so. There is an intrinsic tension between latency and throughput. Erlang chooses willfully to optimize for the former rather than the latter. This works when the majority of the tasks occurring concurrently are typically small and lightweight (aka, a web server).

reply

--

"

pekk 2 days ago

Are Go and Erlang that fungible with each other? Honest question.

reply

derefr 2 days ago

Not in theory, but yes in practice.

For just one example, Go produces static native binaries, while Erlang produces bytecode for a virtual machine. But the Erlang virtual machine is tiny and it's standard practice (with tool support) to ship it with your application as a "release", so either way you get the effects of having one self-sufficient blob of code in a folder that you can "just run" without having to think about runtimes or libraries.

What I would say is that, for every IO-bound highly-concurrent C++ project Google is rewriting into Go, the same project could have been rewritten into Erlang, and they'd see most of the same advantages relative to C++: better, more transparent concurrency; "batteries included" for solving various client-server and distribution problems; being able to just drop a code package on a server and run it; etc.

reply

zaphar 2 days ago

You're assuming IO-bound highly-concurrent C++ server don't have other requirements besides those two. Maybe it's IO-bound highly concurrent text processing. Erlang will suck at this despite the two pieces it's excellent at. Go is pretty fast at processing text and google does a lot of text processing.

reply

derefr 2 days ago

What exactly does "text processing" mean, by the way? Erlang is very good at processing streams of bytes--you can pattern match on binaries to get new sub-binaries (which are basically equivalent to Go's array slices) to pass around, etc. It just gets awkward when you have to convert those streams into codepoints to do case-insensitive comparisons and such.

But to reply more directly, "IO-bound" means something specific--that the problem will be using a negligible amount of CPU, no matter what constant coefficient of overhead the language adds, and so scaling will never be a problem of "oops it's using too much CPU to do the text processing" but rather "oops the gigabit links are saturated we need to add more boxes."

reply

zaphar 2 hours ago

You answered your own question when you acknowledged the areas Erlang get awkward in. Google supports more than 50 different languages. Doing that requires performant analysis of more than just bytes but of unicode codepoints.

reply

Daishiman 1 day ago

Google has both CPU-bound and IO-bound problems at massive scales. A language that is not highly performant in either area would be insufficient.

reply "

--

" Some requirements in that set (e.g. serving data on dl.google.com) could be solved perfectly well by Erlang, and could have been for years, but can also now be solved perfectly well by Go. That they are now being solved by Go is likely an effect of the "Golang advocacy group" that has formed at Google.

Others can't be solved by Erlang, but can be solved by Go (e.g. CPU-bound matrix-multiplications for PageRank?