proj-plbook-plChGoLang

Difference between revision 119 and current revision

No diff available.

Table of Contents for Programming Languages: a survey

Go (golang)

Because it is moderately well-known and well-liked, Go (also called Golang) gets its own chapter.

Good for:

Attributes:

Pros:

Cons:

Tours and tutorials:

Feature lists and discussions and feature tutorials:

Overviews:

Best practices:

Library highlights:

Respected exemplar code:

Core types (from https://tour.golang.org/basics/11 ):

Process:

Retrospectives:

" I made a list of significant simplifications in Go over C and C++:

    regular syntax (don't need a symbol table to parse)
    garbage collection (only)
    no header files
    explicit dependencies
    no circular dependencies
    constants are just numbers
    int and int32 are distinct types
    letter case sets visibility
    methods for any type (no classes)
    no subtype inheritance (no subclasses)
    package-level initialization and well-defined order of initialization
    files compiled together in a package
    package-level globals presented in any order
    no arithmetic conversions (constants help)
    interfaces are implicit (no "implements" declaration)
    embedding (no promotion to superclass)
    methods are declared as functions (no special location)
    methods are just functions
    interfaces are just methods (no data)
    methods match by name only (not by type)
    no constructors or destructors
    postincrement and postdecrement are statements, not expressions
    no preincrement or predecrement
    assignment is not an expression
    evaluation order defined in assignment, function call (no "sequence point")
    no pointer arithmetic
    memory is always zeroed
    legal to take address of local variable
    no "this" in methods
    segmented stacks
    no const or other type annotations
    no templates
    no exceptions
    builtin string, slice, map
    array bounds checking

...

 We also added some things that were not in C or C++, like slices and maps, composite literals, expressions at the top level of the file (which is a huge thing that mostly goes unremarked), reflection, garbage collection, and so on. Concurrency, too, naturally.

One thing that is conspicuously absent is of course a type hierarchy. -- " [6]

" Doug McIlroy?, the eventual inventor of Unix pipes, wrote in 1964 (!):

    We should have some ways of coupling programs like garden hose--screw in another segment when it becomes necessary to massage data in another way. This is the way of IO also.

That is the way of Go also. Go takes that idea and pushes it very far. It is a language of composition and coupling.

The obvious example is the way interfaces give us the composition of components. It doesn't matter what that thing is, if it implements method M I can just drop it in here.

Another important example is how concurrency gives us the composition of independently executing computations.

And there's even an unusual (and very simple) form of type composition: embedding.

These compositional techniques are what give Go its flavor, which is profoundly different from the flavor of C++ or Java programs. " [7]

Spec

https://golang.org/ref/spec

Features

defer, panic, recover

defer is somewhat similar to 'finally' in languages with try/catch/finally, but one important difference is that the scope of 'defer' is a function, whereas the scope of a 'finally' is a try/catch block, and in those languages there can be many try/catch blocks in one function; so Golang is trying to force you to write small functions that only do one thing [8]. Otoh, the 'finally' in try/catch/finally must be in the same lexical scope as the 'try', whereas the 'defer' can appear in any lexical scope, but its body will not be run until the end of the current function scope. This means that multiple 'defer's may be encountered before any of their bodies is run. In this case, the bodies of the defers are placed onto a LIFO stack. An example, from [9]: this function prints "3210":

func b() {
    for i := 0; i < 4; i++ {
        defer fmt.Print(i)
    }

This enables many instances of a resource to be acquired within an inner scope, and then not disposed of until the end of the function. This is unlike try/catch/finally or Python's 'with', in which the disposal must be done in the same scope in which the resource was acquired.

Links:

Iota for enumerated constants

OOP

Golang claims not to have "classes", however you can define methods on any type declared in your package [10], including struct types [11]. You cannot declare methods on a type from another package [12] (todo: what is the rationale for that?).

" Go is “OO-ish” with its use of interfaces — interfaces are basically duck typing for your structs (as well as other types, because, well, just because). I had some trouble at first understanding how to get going with interfaces and pointers. You can write methods that act on WhateverYouWant? — and an interface is just an assertion that WhateverYouWant? has methods for X, Y, and Z. It wasn’t really clear to me whether methods should be acting on values or pointers. Go sort of leaves you to your own devices here.

At first I wrote my methods on the values, which seemed like the normal, American thing to do. The problem of course is that when passed to methods, values are copies of the original data, so you can’t do any OO-style mutations on the data. So instead methods should operate on the pointers, right?

This is where things get a little bit tricky if you’re accustomed to subclassing. If your operate on pointers, then your interface applies to the pointer, not to the struct (value). So if in Java you had a Car with RaceCar? and GetawayCar? as subclasses, in Go you’ll have an interface Car — which is implemented not by RaceCar? and GetawayCar?, but instead by their pointers RaceCar?* and GetawayCar?*.

This creates some friction when you’re trying to manage your car collection. For example, if you want an array with values of type Car, you need an array of pointers, which means you have to need separate storage for the actual RaceCar? and GetawayCar? values, either on the stack with a temporary variable or on the heap with calls to new. The design of interfaces is consistent, and I generally like it, but it had me scratching my head for a while as I got up to speed with all the pointers to my expensive and dangerous automobiles. " [13]

Structural typing / duck typing in interfaces

" jerf 4 hours ago

"Take interfaces. In Java you might start with them. In Go - in the best case - they emerge, when it's time for them."

And it's a relatively subtle language feature that does this, the way that any struct that implements a given interface automatically conforms to that interface without having to be declared. Which means you can declare an interface that foreign packages already conform to, and then freely use them. Which means that you can code freely with concrete structs to start with, and then trivially drop in interfaces to your code later, without having to go modify any other packages, which you may not even own.

I completely agree that on paper Java and Go look virtually identical. But the compounding effect of all those little differences makes them substantially different to work with. Good Go code does not look like good Java code. You'd never confuse them.

reply

kasey_junk 3 hours ago

There is a big downside to structural typing as golang implements it though. Refactoring tools. They quite simply cannot do the same kinds of safe refactoring something like a Java refactoring tool can do, because you can't be sure if the function you are trying to rename, add parameter too, etc. is actually the same function in question.

There are times when I love the structural typing aspects of golang (trivial dependency inversion) and there are times when I hate it (nontrivial renames), its one of many trade-offs you have to be prepared for in golang.

reply " [14]

" pcwalton 3 hours ago

> Which means you can declare an interface that foreign packages already conform to, and then freely use them.

But you cannot do the reverse: you cannot make a type from a foreign package conform to your interface by adding new methods to it. This is because, with structural typing, it's not safe to add methods to types in other packages, since if packages B and C both were to add a conflicting method Foo to a type from package A, B and C could not be linked together. This is a major downside of structural typing for interfaces. Swift, for example, has the "extension" feature, and Java's design allows for it in principle, but it's fundamentally incompatible with Go's design.

reply " [15]

"

justinsaccount 3 hours ago

> But you cannot do the reverse: you cannot make a type from a foreign package conform to your interface by adding new methods to it.

I thought you could.

You don't add the methods directly to it, but you can easily embed the foreign type into a new type that confirms to the interface you want.

  type FooWrapper struct {
    Foo
  }
  func (fw FooWrapper) SomeFunc() {
  ...
  }

reply

pcwalton 2 hours ago

That's making a new type, which causes a lot of friction. For example, a []Foo array is not castable to a []FooWrapper? array without recreating the entire thing.

reply "

[16]

Opinions

Opinionated comparisons / pros and cons

c]oroutines.
    The standard library and package ecosystem contains most libraries a developer could ask for.
    It’s “fast enough” for almost all usecases. Seemingly hitting a sweet spot between easy-to-use single threaded language like Python and Node and blow-your-brains-out ancient but fast behemoths like C++ and C.Or, to put it plainly. Go is a language designed for the age of open source libraries, large-scale parallelism and networking. ... All the remaining issues with Go stem from three design choices: It’s garbage collected, rather than having compile time defined lifetimes for all its resources. This harms performance, removes useful concepts (like move semantics and destructors) and makes compile-time error checking less powerful. It lacks immutability for all but a few (native) types. It lacks generics " -- [153] 

Gotchas

" simiones 8 hours ago [–]

Of course Go has hidden behaviors and overly complex syntax, like any programming language before or after.

For example:

  xrefs := []*struct{field1 int}{}
  for i := range longArrayName {
    longArrayName[i].field = value
    xrefs = append(xrefs, &longArrayName[i])
  }

Perfectly fine code. Now after a simple refactor:

  xrefs := []*struct{field1 int}{}
  for _,x := range longArrayName {
    x.field = value
    xrefs = append(xrefs, &x)
  }

Much better looking! But also completely wrong now, unfortunately. `defer` is also likely to cause similar problems with refactoring, given its scope-breaking function border.

And Go also has several ways of doing most things. Sure, C# has more, but as long as there is more than one the problems are similar.

And I'm sure in time Go will develop more ways of doing things, because the advantage of having a clean way to put your intent into code usually trumps the disadvantage of having to learn another abstraction. This is still part of Go's philosophy, even though Go's designers have valued it slightly less than others. If it weren't, they wouldn't have added Channels to the language, as they are trivial to implement using mutexes, which are strictly more powerful.

reply

feffe 7 hours ago [–]

I don't think this is a good example of proving the point that Go is difficult to read. Isn't it expected to internalize the semantics of basic language primitives when learning a new language, or does people just jump in guessing what different language constructs do? IMHO you learn this the first week when reading up on the language features.

range returns index and elements by value. The last example does what you asks of it, it's like complaining something is not returned by reference when it's not. Your mistake. Perhaps some linter could give warnings for it.

reply

pizza234 5 hours ago [–]

I think this is actually a very good example of the inverse relationship between logical complexity and language complexity.

A language that has implicit copy/move semantics is easier to write (since it's less constrained), and more difficult to read (since in order to understand the code, one needs to know the rules).

A language that has explicit copy/move semantics, is more difficult to write (since the rules will need to be adhered to), but easier to read (because the constraints are explicit).

Although I don't program in Golang, another example that pops into my mind is that slices may refer to old versions of an array. This makes working with arrays easier when writing (as in "typing"), but more difficult when reading (as in understanding/designing), because one needs to track an array references (slices). (correct me if I'm wrong on this).

In this perspective, I do think that a language that is simpler to write can be more difficult to read, and this is one (two) case where this principle applies.

(note that I don't imply with that one philosophy is inherently better than the other)

EDIT: added slices case.

reply

simiones 6 hours ago [–]

I don't think you can truly internalize the varying semantics of which operations return lvalues and which return rvalues. At least, I haven't yet been able to in ~2 years of Go programming, and it's a constant source of bugs when it comes up.

The lack of any kind of syntactic difference between vastly different semantic operations is at the very least a major impediment to readability. After all, lvalues vs rvalues are one of the biggest complexities of C's semantics, and they have been transplanted as-is into Go.

As a much more minor gripe, I'd also argue that the choice of making the iteration variable a copy of the array/slice value instead of a reference to it is the least useful choice. I expect it has been done because of the choice to make map access be an rvalue unlike array access which is an lvalue, which in turn would have given different semantics to slice iteration vs map iteration. Why they chose to have different semantics for map access vs array access, but to have the same semantics for map iteration vs array iteration is also a question I have no answer to.

reply

marcosdumay 26 minutes ago [–]

> and they have been transplanted as-is into Go

Hum... I don't think anything can return an lvalue in C.

Your first paragraph is not a huge concern when programming in C, declarations create lvalues, and that's it. I imagine you are thinking about C++, and yes, it's a constant source of problems there... So, Go made the concept much more complex than on the source.

reply

simiones 8 minutes ago [–]

I think Go has exactly C's semantics here. The following is valid syntax with the same semantics in both:

  array[i] = 9
  *pointer = 9
  array[i].fieldName = 9
  (*pointer).fieldName = 9
  structValue.fieldName = 9  
  pointer->fieldName = 9 // Go equivalent: pointer.fieldName = 9
  // you can also create a pointer to any of these lvalues
  // in either language with &

Go has some additional syntax in map[key], but that behaves more strangely (it's an lvalue in that you can assign to it - map[key] = value - but you can't create a pointer to it - &(map[key]) is invalid syntax).

reply

Measter 3 hours ago [–]

What exactly is the difference here? Is the `x` variable being updated in the second sample so that the stored references all point to the same item?

reply

simiones 1 minute ago [–]

Yes, that code is equivalent to

  xrefs := []*struct{field1 int}{}
  var x struct{field1 int}
  for i := range longArrayName {
    x = longArrayName[i] //overwrite x with the copy
    x.field = value //modify the copy in x
    xrefs = append(xrefs, &x) //&x has the same value regardless of i
  }

The desired refactoring would have been to this:

  xrefs := []*struct{field1 int}{}
  for i := range longArrayName {
    x := &longArrayName[i] //this also declares x to be a *struct{field1 int}
    x.field = value
    xrefs = append(xrefs, x)
  }" -- [229]

" Go has plenty of annoyances too though. Not having any dev vs prod build distinction is annoying. Giving maps a runtime penalty of random iteration in order just to punish devs who don't read docs is annoying. It's annoying to have crappy implementations of things like html templating in the stdlib which steal thunder from better 3rd party libs. Not being able to shadow vars is annoying. `val, err :=` vs `val, err =` is annoying when it swivels on whether `err` has been assigned yet or not, a footgun related to the inability to shadow vars. etc. etc. " [230]

" Here's an example of why Go's simplicity is complicated:

Say I want to take a uuid.UUID [1] and use it as my id type for some database structs.

At first, I just use naked UUIDs as the struct field types, but as my project grows, I find that it would be nice to give them all unique types to both avoid mixups and to make all my query functions clearer as to which id they are using.

    type DogId uuid.UUID
    type CatId uuid.UUID

I go to run my tests (thank goodness I have tests for my queries) and everything breaks! Postgres is complaining that I'm trying to use bytes as a UUID. What gives? When I remove the type definition and use naked UUIDs, it works fine!

The issue is Go encourages reflection for this use-case. The Scan() and Value() methods of a type tell the sql driver how to (de)serialize the type. uuid.UUID has those methods, but when I use a type definition around UUID, it loses those methods.

So the correct way to wrap a UUID to use in your DB is this:

    type DogId struct { uuid.UUID }
    type CatId struct { uuid.UUID }

Go promised me that I wouldn't have to deal with such weird specific knowledge of its semantics. But alas I always do.

[1] https://github.com/google/uuid

EDIT: This issue also affects encoding/json. You can see it in this playground for yourself! https://play.golang.org/p/erfcSIe-Z7b

EDIT: I wrongly used type aliases in the original example, but my issue is with type definitions (`type X Y` instead of `type X = Y`). So all you commenters saying that I did the wrong thing, have another look! " -- [231]

Best practices

'Go' is hard to search for on the web so the tag 'golang' is often used.

"APIs should be designed synchronously, and the callers should orchestrate concurrency if they choose....Channels are useful in some circumstances, but if you just want to synchronize access to shared memory...then you should just use a mutex...Novices to the language have a tendency to overuse channels...One does have to think carefully about ownership hierarchies: only one goroutine gets to close the channel. And if it's in a hot loop, a channel will always perform worse than a mutex: channels use mutexes internally. But plenty of problems are solved very elegantly with channel-based CSP-style message passing." [232]

"in Go is that it is an antipattern for your library to provide something like "a method that makes an HTTP request in a goroutine". In Go, you should simply provide code that "makes an HTTP request", and it's up to the user to decide whether they want to run that in a goroutine....Channels are smelly in an API. IIRC in the entire standard library there's less than 10 functions/methods that return a channel. But the use case does occasionally arise." [233]

Popularity

Misc

Internals and implementations

Core data structures: todo

note: it would be nice if this book had a nice diagrams, such as is found on the page [240], for the in-memory representations of each core data type

Number representations

Integers

Floating points todo

[241]

array representation

'arrays' are of constant length (the length is in the type) (like Python tuples) 'slices' are pointers to arrays. However, practically, slices operate as variable-sized arrays.

multidimensional arrays: todo

limits on sizes of the above

string representation

Golang strings are immutable [242]. They are "a 2-word structure containing a pointer to the string data and a length." [243].

memory model

"Programs that modify data being simultaneously accessed by multiple goroutines must serialize such access. To serialize access, protect the data with channel operations or other synchronization primitives such as those in the sync and sync/atomic packages. If you must read the rest of this document to understand the behavior of your program, you are being too clever. Don't be clever." -- https://golang.org/ref/mem

Links:

Goroutines

Links:

Closures

"...represent a closure as a pair of pointers, one to static code and one to a dynamic context pointer giving access to the captured variables" -- [244]

Links:

Garbage Collection and memory allocation

"Go’s garbage collector is a concurrent mark-sweep with very short stop-the-world pauses" [247]

Links:

libc

https://github.com/golang/go/blob/c95464f0ea3f87232b1f3937d1b37da6f335f336/src/runtime/stack.go#L899

This is a different approach from dedicating a big chunk of address space to the stack, and then lazily mapping & allocating the pages. In this latter approach, stack is never moved and is always continuous, it requires neither scanning the stack for pointers nor support for segmented stacks.

    1
    orib 3 months ago | link | 

Go defaults to well below a single page of memory for the thread stack.

    1
    matklad 3 months ago | link | 

“Well-bellow” is 2kb of memory, half a page. " -- [249]

Toolchain

Go has it's own toolchain, it's own ABI, it's own file formats. Here's a link explaining why. They are written in Go.

Included is an assembly IR, sort of [250] [251].

Go has it's own ABI calling convention:

"

(note: i don't quite understand this; if registers are caller-saved, then why not pass arguments and return values in registers? Is it for platform-independence of a sort?)

See also:

Compiler and toolchain in Go presentation

motivations:

" All made easier by owning the tools and/or moving to Go:

    linker rearchitecture
    new garbage collector
    stack maps
    contiguous stacks
    write barriers

The last three are all but impossible in C:

    C is not type safe; don't always know what's a pointer
    aliasing of stack slots caused by optimization" -- http://talks.golang.org/2015/gogo.slide

" Goroutine stacks

    Until 1.2: Stacks were segmented.
    1.3: Stacks were contiguous unless executing C code (runtime).
    1.4: Stacks made contiguous by restricting C to system stack.
    1.5: Stacks made contiguous by eliminating C." -- http://talks.golang.org/2015/gogo.slide

initial performance problems (were solved): "

Performance problems

Output from translator was poor Go, and ran about 10X slower. Most of that slowdown has been recovered.

Problems with C to Go:

    C patterns can be poor Go; e.g.: complex for loops
    C stack variables never escape; Go compiler isn't as sure
    interfaces such as fmt.Stringer vs. C's varargs
    no unions in Go, so use structs instead: bloat
    variable declarations in wrong place

C compiler didn't free much memory, but Go has a GC. Adds CPU and memory overhead. "

and fixes:

"

Performance fixes

Profile! (Never done before!)

    move vars closer to first use
    split vars into multiple
    replace code in the compiler with code in the library: e.g. math/big
    use interface or other tricks to combine struct fields
    better escape analysis (drchase@).
    hand tuning code and data layout

Use tools like grind, gofmt -r and eg for much of this.

Removing interface argument from a debugging print library got 15% overall!

More remains to be done.

"

Compiler: About 50K lines of portable code + about 10% architecture-specific code

Assembler: Less than 4000 lines, <10% machine-dependent

Linker: 27000 lines summed across 4 architectures, mostly tables (plus some ugliness).

Links:

Generics

Extensions

Variant implementations

ARM implementation

GoLite subset (pedagogical)

"...I poured in more than 35 per week to design, document, and implement a subset of Go...t was a subset, and much of what makes Go interesting was removed (concurrency, interfaces, GC)..."

emgo

https://github.com/ziutek/emgo

" Emgo follows Go specification with exception for memory allocation.

Think about Emgo as C with: Go syntax, Go packages, Go building philosophy.

In Go, variable declared in function can be allocated on the stack or on the heap - escaping analisis is used for decision. In Emgo (like in C), all local variables are stack allocated. Dynamic allocation and garbage collection can occurs only when:

    new or make builtin function is used.
    Non-empty strings are concatanated.
    Builtin append function is called and there is not enough space in destination.
    An element to map is added or removed.

Current allocator is trivial and doesn't contain GC. This isn't big disadventage for many embeded applications that need allocation only during startup. They often run on MCU that has only few kilobytes of SRAM and in many cases they must respond in realtime. The simple "stop the world GC" can't be used and much sophisticated one consumes to much Flash/SRAM.

The target is to allow chose between multiple allocators (simply import it as package) and use one that best fits application needs.

Examples:

The following function is correct Go function:

func F() ([]byte, *int) { i := 4 return []byte{1, 2, 3}, &i }

but it isn't correct Emgo function - you need to rewrite it this way:

func F() ([]byte, *int) { i := new(int) *i = 4 b := append([]byte{}, []byte{1, 2, 3}...) return b, i }

...

Standard library.

Emgo standard library doesn't follow Go standard library....

Not yet implemented:

Maps. Defer. String concatanation. Append. Unnamed structs. Closures. " -- [253]

tinygo

https://tinygo.org/ https://github.com/aykevl/tinygo https://aykevl.nl/2019/02/tinygo-goroutines

"similar to emgo but a major difference is that I want to keep the Go memory model (which implies garbage collection of some sort). Another difference is that TinyGo? uses LLVM internally instead of emitting C, which hopefully leads to smaller and more efficient code and certainly leads to more flexibility."

" Currently supported features:

    control flow
    many (but not all) basic types: most ints, strings, structs
    function calling
    interfaces for basic types (with type switches and asserts)
    goroutines (very initial support)
    function pointers (non-blocking)
    interface methods
    standard library (but most packages won't work due to missing language features)
    slices (partially)

Not yet supported:

    float, complex, etc.
    maps
    garbage collection
    defer
    closures
    channels
    introspection (if it ever gets implemented)
    ..."

" aykevl 2 days ago [-]

Yeah the heavyweight standard library is a big problem. I'm trying to solve it (in part) with smart analysis that tries much harder to remove dead code and dead global data. Maybe even try to run some initializers at compile time so the result can be put in flash instead of RAM.

In my experience, it's not the unicode package that's big but the run-time typing information and the reflect package used by packages like fmt. The fmt package doesn't know what types it will need to accept so it literally supports everything (including unused types).

And as you mention WASM: I've been thinking about supporting that too, exactly because WASM and microcontrollers have some surprising similarities (small code size, fast startup, little OS-support).

reply "

" There are 3 LLVM-based Go compilers that I'm aware of: gollvm, llgo, and tinygo. Of those, only TinyGo? reimplements the runtime that causes lots of (code size) overhead in the other compilers.

There is more to a toolchain than translating source code to machine code. I'm sure the others do that job just as well, but only TinyGo? combines that with a reimplemented runtime that optimizes size over speed and allows it to be used directly on bare metal hardware: binaries of just a few kB are not uncommon. " -- aykevl

" Go uses goroutines for mostly-cooperative multitasking. In general, each goroutine has a separate stack where it can store things like temporary values and return addresses. At the moment the main Go compiler starts out with a 2kB stack for each goroutine and grows it as needed.

TinyGo? is different. The system it uses for goroutines is based on the async/await model like in C#, JavaScript? and now also C++. In fact, we're borrowing the C++ implementation that's used in Clang/LLVM. The big difference here is that TinyGo? inserts async/await keywords automatically. ... Luckily, you don't have to worry about the color of your function in Go, but I've chosen to use this as an implementation strategy for TinyGo?. The reason is that TinyGo? also wants to support WebAssembly? and WebAssembly? does not support efficient stack switching like basically every other ISA does: the stack has been hidden entirely for security reasons. So I've decided to use coroutines1 instead of actually separate stacks. " [254]

golang on wasm

---

---

llgo

An LLVM Golang implementation

GoLLVM

https://go.googlesource.com/gollvm/

Links