proj-plbook-plChGoLang

Table of Contents for Programming Languages: a survey

Go (golang)

Because it is moderately well-known and well-liked, Go (also called Golang) gets its own chapter.

Good for:

Attributes:

Pros:

Cons:

Tours and tutorials:

Best practices:

Library highlights:

Respected exemplar code:

Core types (from https://tour.golang.org/basics/11 ):

Process:

Retrospectives:

" I made a list of significant simplifications in Go over C and C++:

    regular syntax (don't need a symbol table to parse)
    garbage collection (only)
    no header files
    explicit dependencies
    no circular dependencies
    constants are just numbers
    int and int32 are distinct types
    letter case sets visibility
    methods for any type (no classes)
    no subtype inheritance (no subclasses)
    package-level initialization and well-defined order of initialization
    files compiled together in a package
    package-level globals presented in any order
    no arithmetic conversions (constants help)
    interfaces are implicit (no "implements" declaration)
    embedding (no promotion to superclass)
    methods are declared as functions (no special location)
    methods are just functions
    interfaces are just methods (no data)
    methods match by name only (not by type)
    no constructors or destructors
    postincrement and postdecrement are statements, not expressions
    no preincrement or predecrement
    assignment is not an expression
    evaluation order defined in assignment, function call (no "sequence point")
    no pointer arithmetic
    memory is always zeroed
    legal to take address of local variable
    no "this" in methods
    segmented stacks
    no const or other type annotations
    no templates
    no exceptions
    builtin string, slice, map
    array bounds checking

...

 We also added some things that were not in C or C++, like slices and maps, composite literals, expressions at the top level of the file (which is a huge thing that mostly goes unremarked), reflection, garbage collection, and so on. Concurrency, too, naturally.

One thing that is conspicuously absent is of course a type hierarchy. -- " [3]

" Doug McIlroy?, the eventual inventor of Unix pipes, wrote in 1964 (!):

    We should have some ways of coupling programs like garden hose--screw in another segment when it becomes necessary to massage data in another way. This is the way of IO also.

That is the way of Go also. Go takes that idea and pushes it very far. It is a language of composition and coupling.

The obvious example is the way interfaces give us the composition of components. It doesn't matter what that thing is, if it implements method M I can just drop it in here.

Another important example is how concurrency gives us the composition of independently executing computations.

And there's even an unusual (and very simple) form of type composition: embedding.

These compositional techniques are what give Go its flavor, which is profoundly different from the flavor of C++ or Java programs. " [4]

Features

defer, panic, recover

defer is somewhat similar to 'finally' in languages with try/catch/finally, but one important difference is that the scope of 'defer' is a function, whereas the scope of a 'finally' is a try/catch block, and in those languages there can be many try/catch blocks in one function; so Golang is trying to force you to write small functions that only do one thing [5]. Otoh, the 'finally' in try/catch/finally must be in the same lexical scope as the 'try', whereas the 'defer' can appear in any lexical scope, but its body will not be run until the end of the current function scope. This means that multiple 'defer's may be encountered before any of their bodies is run. In this case, the bodies of the defers are placed onto a LIFO stack. An example, from [6]: this function prints "3210":

func b() {
    for i := 0; i < 4; i++ {
        defer fmt.Print(i)
    }

This enables many instances of a resource to be acquired within an inner scope, and then not disposed of until the end of the function. This is unlike try/catch/finally or Python's 'with', in which the disposal must be done in the same scope in which the resource was acquired.

Links:

Iota for enumerated constants

OOP

Golang claims not to have "classes", however you can define methods on any type declared in your package [7], including struct types [8]. You cannot declare methods on a type from another package [9] (todo: what is the rationale for that?).

" Go is “OO-ish” with its use of interfaces — interfaces are basically duck typing for your structs (as well as other types, because, well, just because). I had some trouble at first understanding how to get going with interfaces and pointers. You can write methods that act on WhateverYouWant? — and an interface is just an assertion that WhateverYouWant? has methods for X, Y, and Z. It wasn’t really clear to me whether methods should be acting on values or pointers. Go sort of leaves you to your own devices here.

At first I wrote my methods on the values, which seemed like the normal, American thing to do. The problem of course is that when passed to methods, values are copies of the original data, so you can’t do any OO-style mutations on the data. So instead methods should operate on the pointers, right?

This is where things get a little bit tricky if you’re accustomed to subclassing. If your operate on pointers, then your interface applies to the pointer, not to the struct (value). So if in Java you had a Car with RaceCar? and GetawayCar? as subclasses, in Go you’ll have an interface Car — which is implemented not by RaceCar? and GetawayCar?, but instead by their pointers RaceCar?* and GetawayCar?*.

This creates some friction when you’re trying to manage your car collection. For example, if you want an array with values of type Car, you need an array of pointers, which means you have to need separate storage for the actual RaceCar? and GetawayCar? values, either on the stack with a temporary variable or on the heap with calls to new. The design of interfaces is consistent, and I generally like it, but it had me scratching my head for a while as I got up to speed with all the pointers to my expensive and dangerous automobiles. " [10]

Structural typing / duck typing in interfaces

" jerf 4 hours ago

"Take interfaces. In Java you might start with them. In Go - in the best case - they emerge, when it's time for them."

And it's a relatively subtle language feature that does this, the way that any struct that implements a given interface automatically conforms to that interface without having to be declared. Which means you can declare an interface that foreign packages already conform to, and then freely use them. Which means that you can code freely with concrete structs to start with, and then trivially drop in interfaces to your code later, without having to go modify any other packages, which you may not even own.

I completely agree that on paper Java and Go look virtually identical. But the compounding effect of all those little differences makes them substantially different to work with. Good Go code does not look like good Java code. You'd never confuse them.

reply

kasey_junk 3 hours ago

There is a big downside to structural typing as golang implements it though. Refactoring tools. They quite simply cannot do the same kinds of safe refactoring something like a Java refactoring tool can do, because you can't be sure if the function you are trying to rename, add parameter too, etc. is actually the same function in question.

There are times when I love the structural typing aspects of golang (trivial dependency inversion) and there are times when I hate it (nontrivial renames), its one of many trade-offs you have to be prepared for in golang.

reply " [11]

" pcwalton 3 hours ago

> Which means you can declare an interface that foreign packages already conform to, and then freely use them.

But you cannot do the reverse: you cannot make a type from a foreign package conform to your interface by adding new methods to it. This is because, with structural typing, it's not safe to add methods to types in other packages, since if packages B and C both were to add a conflicting method Foo to a type from package A, B and C could not be linked together. This is a major downside of structural typing for interfaces. Swift, for example, has the "extension" feature, and Java's design allows for it in principle, but it's fundamentally incompatible with Go's design.

reply " [12]

"

justinsaccount 3 hours ago

> But you cannot do the reverse: you cannot make a type from a foreign package conform to your interface by adding new methods to it.

I thought you could.

You don't add the methods directly to it, but you can easily embed the foreign type into a new type that confirms to the interface you want.

  type FooWrapper struct {
    Foo
  }
  func (fw FooWrapper) SomeFunc() {
  ...
  }

reply

pcwalton 2 hours ago

That's making a new type, which causes a lot of friction. For example, a []Foo array is not castable to a []FooWrapper? array without recreating the entire thing.

reply "

[13]

Opinions

Opinionated comparisons / pros and cons

  emails := make([]string, 0)
  for _, u := range users {
    if u.active {
      emails = append(emails, u.email)
    }
  }
  return email" [99] 
    Go utilities allow you to easily download and use packages.
    Static compilation makes porting code between various environments and setting up a development environment really easy.
    Native asynchronous I/O mechanisms allow you to easily write high performance networking code.
    Built-in channels allow for easy to implement and relatively safe data transfer between [g|c]oroutines.
    The standard library and package ecosystem contains most libraries a developer could ask for.
    It’s “fast enough” for almost all usecases. Seemingly hitting a sweet spot between easy-to-use single threaded language like Python and Node and blow-your-brains-out ancient but fast behemoths like C++ and C.

Or, to put it plainly. Go is a language designed for the age of open source libraries, large-scale parallelism and networking.

...

All the remaining issues with Go stem from three design choices:

    It’s garbage collected, rather than having compile time defined lifetimes for all its resources. This harms performance, removes useful concepts (like move semantics and destructors) and makes compile-time error checking less powerful.
    It lacks immutability for all but a few (native) types.
    It lacks generics" -- [113] 

Gotchas

for x := range v { wg.Add(1) go func() { defer wg.Done() foo(x) }() }

x would be scoped to the block, rather than to the whole for statement, so it would Just Work. This is the most common mistake I still regularly make in quick one-off programs." [143]

Best practices

'Go' is hard to search for on the web so the tag 'golang' is often used.

"APIs should be designed synchronously, and the callers should orchestrate concurrency if they choose....Channels are useful in some circumstances, but if you just want to synchronize access to shared memory...then you should just use a mutex...Novices to the language have a tendency to overuse channels...One does have to think carefully about ownership hierarchies: only one goroutine gets to close the channel. And if it's in a hot loop, a channel will always perform worse than a mutex: channels use mutexes internally. But plenty of problems are solved very elegantly with channel-based CSP-style message passing." [161]

"in Go is that it is an antipattern for your library to provide something like "a method that makes an HTTP request in a goroutine". In Go, you should simply provide code that "makes an HTTP request", and it's up to the user to decide whether they want to run that in a goroutine....Channels are smelly in an API. IIRC in the entire standard library there's less than 10 functions/methods that return a channel. But the use case does occasionally arise." [162]

Popularity

Misc

Internals and implementations

Core data structures: todo

note: it would be nice if this book had a nice diagrams, such as is found on the page [163], for the in-memory representations of each core data type

Number representations

Integers

Floating points todo

[164]

array representation

'arrays' are of constant length (the length is in the type) (like Python tuples) 'slices' are pointers to arrays. However, practically, slices operate as variable-sized arrays.

multidimensional arrays: todo

limits on sizes of the above

string representation

Golang strings are immutable [165]. They are "a 2-word structure containing a pointer to the string data and a length." [166].

memory model

"Programs that modify data being simultaneously accessed by multiple goroutines must serialize such access. To serialize access, protect the data with channel operations or other synchronization primitives such as those in the sync and sync/atomic packages. If you must read the rest of this document to understand the behavior of your program, you are being too clever. Don't be clever." -- https://golang.org/ref/mem

Links:

Goroutines

Links:

Closures

"...represent a closure as a pair of pointers, one to static code and one to a dynamic context pointer giving access to the captured variables" -- [167]

Links:

Garbage Collection

"Go’s garbage collector is a concurrent mark-sweep with very short stop-the-world pauses" [170]

Links:

Toolchain

Go has it's own toolchain, it's own ABI, it's own file formats. Here's a link explaining why. They are written in Go.

Included is an assembly IR, sort of [171] [172].

Go has it's own ABI calling convention:

"

(note: i don't quite understand this; if registers are caller-saved, then why not pass arguments and return values in registers? Is it for platform-independence of a sort?)

Compiler and toolchain in Go presentation

motivations:

" All made easier by owning the tools and/or moving to Go:

    linker rearchitecture
    new garbage collector
    stack maps
    contiguous stacks
    write barriers

The last three are all but impossible in C:

    C is not type safe; don't always know what's a pointer
    aliasing of stack slots caused by optimization" -- http://talks.golang.org/2015/gogo.slide

" Goroutine stacks

    Until 1.2: Stacks were segmented.
    1.3: Stacks were contiguous unless executing C code (runtime).
    1.4: Stacks made contiguous by restricting C to system stack.
    1.5: Stacks made contiguous by eliminating C." -- http://talks.golang.org/2015/gogo.slide

initial performance problems (were solved): "

Performance problems

Output from translator was poor Go, and ran about 10X slower. Most of that slowdown has been recovered.

Problems with C to Go:

    C patterns can be poor Go; e.g.: complex for loops
    C stack variables never escape; Go compiler isn't as sure
    interfaces such as fmt.Stringer vs. C's varargs
    no unions in Go, so use structs instead: bloat
    variable declarations in wrong place

C compiler didn't free much memory, but Go has a GC. Adds CPU and memory overhead. "

and fixes:

"

Performance fixes

Profile! (Never done before!)

    move vars closer to first use
    split vars into multiple
    replace code in the compiler with code in the library: e.g. math/big
    use interface or other tricks to combine struct fields
    better escape analysis (drchase@).
    hand tuning code and data layout

Use tools like grind, gofmt -r and eg for much of this.

Removing interface argument from a debugging print library got 15% overall!

More remains to be done.

"

Compiler: About 50K lines of portable code + about 10% architecture-specific code

Assembler: Less than 4000 lines, <10% machine-dependent

Linker: 27000 lines summed across 4 architectures, mostly tables (plus some ugliness).

Links:

Variant implementations

ARM implementation

GoLite subset (pedagogical)

"...I poured in more than 35 per week to design, document, and implement a subset of Go...t was a subset, and much of what makes Go interesting was removed (concurrency, interfaces, GC)..."

emgo

https://github.com/ziutek/emgo

" Emgo follows Go specification with exception for memory allocation.

Think about Emgo as C with: Go syntax, Go packages, Go building philosophy.

In Go, variable declared in function can be allocated on the stack or on the heap - escaping analisis is used for decision. In Emgo (like in C), all local variables are stack allocated. Dynamic allocation and garbage collection can occurs only when:

    new or make builtin function is used.
    Non-empty strings are concatanated.
    Builtin append function is called and there is not enough space in destination.
    An element to map is added or removed.

Current allocator is trivial and doesn't contain GC. This isn't big disadventage for many embeded applications that need allocation only during startup. They often run on MCU that has only few kilobytes of SRAM and in many cases they must respond in realtime. The simple "stop the world GC" can't be used and much sophisticated one consumes to much Flash/SRAM.

The target is to allow chose between multiple allocators (simply import it as package) and use one that best fits application needs.

Examples:

The following function is correct Go function:

func F() ([]byte, *int) { i := 4 return []byte{1, 2, 3}, &i }

but it isn't correct Emgo function - you need to rewrite it this way:

func F() ([]byte, *int) { i := new(int) *i = 4 b := append([]byte{}, []byte{1, 2, 3}...) return b, i }

...

Standard library.

Emgo standard library doesn't follow Go standard library....

Not yet implemented:

Maps. Defer. String concatanation. Append. Unnamed structs. Closures. " -- [174]

tinygo

https://tinygo.org/ https://github.com/aykevl/tinygo

"similar to emgo but a major difference is that I want to keep the Go memory model (which implies garbage collection of some sort). Another difference is that TinyGo? uses LLVM internally instead of emitting C, which hopefully leads to smaller and more efficient code and certainly leads to more flexibility."

" Currently supported features:

    control flow
    many (but not all) basic types: most ints, strings, structs
    function calling
    interfaces for basic types (with type switches and asserts)
    goroutines (very initial support)
    function pointers (non-blocking)
    interface methods
    standard library (but most packages won't work due to missing language features)
    slices (partially)

Not yet supported:

    float, complex, etc.
    maps
    garbage collection
    defer
    closures
    channels
    introspection (if it ever gets implemented)
    ..."

" aykevl 2 days ago [-]

Yeah the heavyweight standard library is a big problem. I'm trying to solve it (in part) with smart analysis that tries much harder to remove dead code and dead global data. Maybe even try to run some initializers at compile time so the result can be put in flash instead of RAM.

In my experience, it's not the unicode package that's big but the run-time typing information and the reflect package used by packages like fmt. The fmt package doesn't know what types it will need to accept so it literally supports everything (including unused types).

And as you mention WASM: I've been thinking about supporting that too, exactly because WASM and microcontrollers have some surprising similarities (small code size, fast startup, little OS-support).

reply "

" There are 3 LLVM-based Go compilers that I'm aware of: gollvm, llgo, and tinygo. Of those, only TinyGo? reimplements the runtime that causes lots of (code size) overhead in the other compilers.

There is more to a toolchain than translating source code to machine code. I'm sure the others do that job just as well, but only TinyGo? combines that with a reimplemented runtime that optimizes size over speed and allows it to be used directly on bare metal hardware: binaries of just a few kB are not uncommon. " -- aykevl

llgo

An LLVM Golang implementation

GoLLVM

https://go.googlesource.com/gollvm/

Links