![[Home]](http://www.bayleshanks.com/cartoonbayle.png)
The following, although written as present-tense facts, are in fact future hypothetical wishlist items! None of this has been implemented yet, and even the design is still in flux! These notes about 'Oot' contain many conflicting proposals. Oot has neither been designed nor implemented. Oot is not even vaporware. No one is seriously planning to create Oot within any reasonable timeframe.
Oot is a readable, programmable programming language with facilities for massive concurrency. It has a single fundamental data structure, graphs. It is built on a small core language.
Oot is a high-level multi-paradigm language supporting optional static typing, automatic memory management, and attention to safety and error handling. It has great libraries geared towards immutable data structures, and extensive tooling. We value communal gravity, and an understandable, portable implementation.
Oot is a general-purpose language that particularly targets the application domains of expression of computation, exploration of brain-like computational architectures, glue language, exploratory numerical computing, command-line one-liners, prototyping (cross-platform client-side applications and also server-side), live coding, learning to program, and writing new programming languages.
Some downsides of Oot are: slow speed; high memory usage; optional type safety (can lead to a reduction of communal standards of safety); metaprogrammability (can lead to difficult-to-read code); its goals are ambitous, but its creator is a hack.
Oot aims to combine the readability and ease of Python, the functional laziness of Haskell, the commandline suitability of Perl, the straighforwardness of C, and the simplicity of BASIC (well, almost), with metaprogramming and massive concurrency.
In a nutshell, Oot is: simple, metaprogrammable, expressive, readable, concise, concurrent, graph-based. Note that the first five of these are general abstract design goals of (many) programming languages, and the last two of them are specific approaches or features.
Next, we're going to explain each part of the above. Then, we'll go back and give lists of features associated with each goal.
Oot prioritizes readability. We define readable to mean that a moderately experienced Oot programmer can quickly understand someone else's source code.
Some languages can encourage newcomers to write 'ugly' and hence hard-to-read code. Other languages can encourage experts to write 'fancy' and hence hard-to-read code. An example of a language which does neither is Python. That is the sort of thing we strive for.
One important kind of readability is 'skimmability', the ability to navigate through and get a general sense of code without understanding all of it. Skimmability can be opposed to succinctness (a succinct language is one in which programs can be relatively short); if you want to understand the gist of something without reading the definition of every terms it contains, then there must be some redundancy. For Oot, we want succinctness only to the extent that it aids readability and skimmability.
As a means to the end of readability, we pursue various other design criteria, including minimization of 'gotchas' (also known as the principal of least surprise); regularity, minimal language size; maximal simplicity, expressiveness; and succinctness.
We want our language to be all of:
We define power as maximizing the range of ideas that can be succinctly and naturally expresssed with the language, while minimizing complexity (complexity is the opposite of simplicity). Each feature must increase expressivity or succintness enough to justify its complexity.
Oot provides very expressive metaprogramming constructs. Oot itself is implemented as a small 'core' language, plus other constructs writing in Oot using metaprogramming (plus a standard library).
Meta-programming can make source code hard for others to read. Oot provides a variety of meta-programming mechanisms, and encourages the use of the less powerful mechanisms when possible.
Oot attempts to provide the power of meta-programming while avoiding the curse of Lisp (namely that libraries in a very expressive language have trouble creating enough 'gravity' to attract contributors).
Note that one part of making Oot 'programmable' is keeping its syntax simple, in order to minimize the complexity of writing functions that transform code (and also in order to make it easier to write tooling).
By 'massive' we mean situations where the number of concurrently executing threads may be comparable to the number of elements of data being operated upon (for example, one thread for each pixel in an image). (this scale is clearly ill-matched to the number of CPUs in present-day computers, so why would you want this? See 'exploration of brain-like computational architectures', below).
When many operations are provided for a few pervasive data structures, it becomes easy to compose functionality. Lisp and Haskell have lists. Python has lists and dicts. Can we do better? Oot has one fundamental data structure, the Oot graph. Lists, associative arrays, matrices, dataframes, relations, partial orders, and more are all special cases of Oot graphs.
High-level: Oot works at a high level of abstraction. This means that you can write abstract code which is flexible enough to be adapted to various uses. It also means that you can express what you want a program to do, and Oot will handle some implementation details for you.
Multi-paradigm: Oot combines elements of the lazy functional, imperative, logic, object-oriented, and 'neolanguage' paradigms. 'Neo-language' is a term i made up for certain programming language ideas imported from human languages.
Portable: Oot's semantics are relatively platform-independent, and its implementation is relatively easy to port to new platforms
Optional static typing: Oot has an extensible type system with built-in support for higher-order polymorphic gradual typing.
Safety and error handling: Mature code contains a lot of error handling, and programmers spend a lot of time debugging. Therefore, a primary concern of Oot is to reduce bugs (safety) and to provide convenient, readable mechanisms for handling errors.
communal gravity: any force which opposes fragmentation (which could lead to library incompatibilities). We are aware of the Curse of Lisp and actively trying to oppose it in the design of the language and of the community.
canonical implementation: Oot is focused around a canonical implementation rather than a spec (although there is a spec)
open source: todo: which license? Probably Apache? BSD/Apache/MIT? triple disjunctive license? or, aren't those already covertible?
expression of computation: For example, a description of an algorithm in a textbook.
brain-like computational architectures: It is unclear exactly how the brain actually works, but in order to gain intuition in this area, one idea is to try to create programming languages that seem like they may be suited to 'brain-like' algorithms, ones which utilize massive concurrency, non-uniform memory access (NUMA), and that are robust.
glue language: There are so many libraries already available in other languages, and so many projects already started in other languages, that Oot is expected to often be used to write new code to extend an existing project in another language, or which uses libraries written in other languages.
live coding: Applications such as simulations, performances, servers, and learning to program often benefit from the ability to change the program code while the program is still running (see live coding).
the rest of this document is under heavy construction
python haskell basic note: http://www.c2.com/cgi/wiki?FirstLanguageLearned hypercard lisp perl assembly nock
todo
team programming
sandboxing
modular
todo http://en.wikipedia.org/wiki/Scala_%28programming_language%29 has a nice list of individual features, most of which we have. also, add these to PL book constructs list.
Let us discuss some ways in which the goal of readability is achieved.
When syntax is 'scannable', your eye can quickly scan through code looking for where a certain thing occurs.
Declarative code is easier to understand than code which mixes the specificiation of what is true with how it is to be achieved. For example, code which explicitly states invariants in the definition of data types is easier to read than code which provides only the procedure for transforming data and leaves the programmer to infer invariants. todo
Mechanistic concepts are those which consist of a set of parts, each of which has simple behavior and few connections, which can be imagined as interacting in linear chains of causation over time as a program runs. Although theorems are easier to analyze than procedural code with many hidden special cases, mechanistic concepts can be easier to understand than theorems. One reason is that theorems are atemporal in the sense that a premise about later events might imply a conclusion about earlier ones. For example, unification-style Hindley–Milner? type inference can be hard to reason about when code which will execute later contains type constraints that affects the typing of code that will execute earlier. todo
Although Oot is very metaprogrammable, it's almost always semantics which is altered, not syntax. A reader should be able to tell how almost all Oot code parses without looking up the definitions of metaprogramming code. For example, there is no custom operator precedence; although you can define custom operators, their precedence is determined by the puncuation symbols used. Similarly, a wide variety of grouping constructs are built-in, but there is no metaprogramming for new grouping constructs (actually you could abuse macros, source filtering, or cross-language code members, in conjunction with 'eval', to achieve custom syntax, but this is strongly discouraged). Oot achieves this by offering a very powerful set of core syntactic constructs whose semantics can be altered by context. The goal is to provide a single syntax which can support almost any programming paradigm, and then to allow metaprogramming by defining custom semantics for this syntax. Metaprogrammable syntactic constructs provided by Oot include:
oot tries to make it easy for a casual reader to first understand the 'happy path' and only then to understand error-checking complexities, by trying to provide some spatial separation between happy path code and error handling code; and by trying to provide some separation between the calculation being carried out and performance optimizations in the way it is carried out. Similarly, Oot tries to support separation of distributed programming, concurrency, and scalability concerns from the 'happy path'. Ideally, rather than changing the expression of the 'happy path' computation to enable scalability, the happy path computation is always expressed in the same way as it could be for a non-concurrent, non-distributed, non-robust, in-memory prototype; and the other complexity is spatially separated in the codebase.
todo
todo ez comments
todo footnotes
(from concise)
lenient evaluation
first-class functions
In pursuit of as much generality as possible, Oot's graphs are directed, labeled, reified, multi, hyper graphs with ports. Directed, meaning that there may be an arcs from one node to another one, without an arc from the second to the first. Labeled, meaning that nodes and arcs may have data attached to them. Reified, meaning that the target of an arc may be a node, or another arc, or it might be the graph as a whole. Multi, meaning that there may be multiple arcs going from one node to another node. Hyper, meaning that we generalize binary arcs, which have a source and a target, to n-ary arcs, which involve a number of nodes in a number of roles. With ports, meaning that we can assign labels to the 'connection' between a particular node and a particular edge, in such a way so that within any given node, each edge has a unique label; and we can group these ports into 'port types'.
List is a homoiconic language based on lists; function calls in the source code are written as lists of arguments appended to the function identifier. Oot is 'almost homoiconic' but is based on keyword arguments rather than positional arguments; therefore, an associative array is the fundamental structure, rather than a list like in Lisp.
So why do we say graphs, instead of associative arrays? Because we often want to reason about the nodes underneath the first level of the array (eg the node 'female' in the associative array: {Alice: {id: 1, gender: female}, Bob: {id: 2, gender, male}}).
Also, note that (a) graphs can embed almost all of the popular computer science data structures, and (b) Oot is good at manipulating ASTs, which assists metaprogramming.
A value may have different representations in memory. For example, the integer '-1' might be represented as an 8-bit integer with a sign bit, as an 8-bit integer with twos complement, as a 16-bit integer with a sign bit, etc.
Often we find that one piece of data must be represented in multiple forms because the different functions that need to operate on it expect different data types. For example, a persistence framework may wish to attach metadata to persisted arrays, but this metadata must be hidden when the arrays are to be passed to library that expects plain arrays, yet still present after the mutated arrays are returned.
In Oot, one can easily pass between (a) a 'non-meta' graph, and (b) that same graph, but with additional annotation nodes, in the form of specially-typed metanotes which have "annotates" arcs which point to nodes and arcs in the 'non-meta' part of the graph. In Oot, one can use views to 'hide' the annotations, pass the non-meta graph to some function that traverses all of the nodes and mutates them (or rather, since we like immutable data, creates a 'mutated' copy of the graph as a new value), and then to recover the annotation nodes from the new value, because the runtime secretly copied the hidden annotation nodes when the relevant underlying nodes were copied into the new value.
Another example of when views are useful is when a tree data structure must appear as a linear array of nodes for the purpose of passing to a filtering function, but in actuality must retain its tree structure after the undesirable nodes have been filtered out.
Popular constructs for this purpose include type classes and OOP ad-hoc polymorphism; both of which work via the creation of an interface in the form of a set of functions supported by a type (or type class). To borrow the language of RESTful web architecture, this solution amounts to the creation of a new set of verbs for each interface.
Oot discourages types defined by the unnecessary creation of verbs in cases where they can instead be defined more simply as graphs conforming to a certain shape. This allows the pervasive graph operations to be applied to more types. For example, trees and arrays are both graphs with different shape constraints. If data in the shape of a tree needs to be filtered by a function that expects a linear list, instead of creating an adaptor that exposes a set of functions such as next(), the Oot way is to specify the shape transformation that says, for each node in the tree, which item in the list it corresponds to.
To assist in this, Oot provides the concept of 'views'. Oot views allow the same piece of data to be viewed in different ways. They are defined by a specification of the shape transformations (mathematically, homomorphisms) which define the various representations of the same piece of data. A mutation of the data in any of its views affects it in all views. When this sort of thing is implemented in other languages it tends to involve things such as pointers linking together data fields within distinct variables, but by contrast, Oot presents the abstraction of a single variable that holds a single value which can be seen from different views. Views are a type of ad-hoc polymorphism, a way for a piece of data to expose a different 'interface' to various consumers, but these interfaces differ via shape rather than via a distinct function signature.
todo
like spreadsheets
binding is tightly restricted between different state domains (or just 'domains')
todo
Nodes in a graph might correspond to values of a given data type; for example, you might have a graph whose nodes have integer values. But what happens when the type of the nodes itself has a graph structure? For example, if you have a data type A, where A is a list whose nodes are type B, where B is the type of arrays of integers? You might conceptualize an instance of A in two ways; first, as a simple list-shaped graph with opaque nodes of type A; or second, as one big graph whose nodes are integers. The question at hand is: when do we stop recursing into node values and decide to just consider something an opaque 'node'? The difference is not merely academic; for example, when a value of type A is operated upon by a generic library function that visits each node in a graph and applies some transformation, different results will obtain depending whether the graph is composed of arrays of integers, or integers.
Oot allows the programmer to fluidly move between these conceptions via the construct of 'boundaries'. A boundary can be visualized as a wall surrouding nodes in a graph; any remaining structure inside the wall is considered opaque (until such time as the boundary is moved or deactivated). A single graph can be marked with multiple boundaries of different 'color', only one of which is considered the 'active boundary color' at any one time.
Boundaries seem even more important when one contemplates the binding together of different variables in a Oot state domain. In essence, each Oot state domain could be considered as one big graph. The question of where one variable stops and another begins becomes merely a question of boundaries. Taking this viewpoint even further, the different state domains within a program can themselves be thought of components of a larger graph, a single graph containing the contents of all of the variables of the program as subgraphs, separated by various boundaries.
In other words, Oot data values (what Oot variables hold) are graphs; where one graph references another, one can consider these two graphs as two separate values, or one can consider them as one graph value with two entry points. Oot allows the programmer to flexibly shift between these conceptions.
Oot's focus on graphs fits together well with its pattern matching, and both assist in its extensible type system and its metaprogramming.
Oot is good at having one graph provide an overlay or annotation for another. it is also good at representing advanced versions of overlays such as homomorphisms.
The emphasis on annotation makes it good for representing code and for defining type systems and for attaching invariants, theorems, etc, to parts of code (to points and to regions; and also preconditions and postconditions of regions)
The extensible syntax is good for representing modal logic, which fits in well with pre- and post-conditions.
A programming language based on graph data structures is well-suited for talking about code, which eases metaprogramming.
For example, graphs with ports can represent the axis of potential control flow through an instruction sequence by using one or more 'next'-type ports to indicate potential subsequent instructions; branch instructions are represented with multiple 'next'-type ports. We could also use 'previous'-type ports to indicate from which other instructions control could pass to the current instruction. We can also represent the concept of an 'execution trace' in this fashion using the same abstraction, by addition a flattening/linearization constraint: each instruction may have only one 'previous' and only one 'next' (of course, in an execution trace, the same instruction may appear multiple times, which is another difference from the potential control flow abstraction).
For example, Oot favors the ML function calling syntax (eg 'f x') instead of the traditional mathematical one ('f(x)'), because it is more succinct.
One strategy that Oot uses to achieve power is to try to unify what in other languages would be many concepts, into one general concept, allowing us to pay the complexity cost once and reap many applications. Examples of this strategy:
(from flexibility)
(from modularity)
Oot attempts to make it easy to use standard Oot constructs to interact with external resources and APIs, for example CRUD and query operations on datastores and REST APIs.
oot understands the distinction between an ideal data type, and a represeentation data type; has conventions for errors that signal when something should be allowed by the ideal, but isnt due to the represeentation, has conventions for requesting or requiring speecific representations, or requesting representations with sprecific performance characteristics
Oot data types deal with the idealized structure of data and with what operations can be performed on it; they do not mandate a particular format to store the data in. The format is what we call a 'representation'.
For example, an integer can be any size. Some representations of integers, however (such as 32-bit "integers") have size limits; others (so-called "BigNums?") don't.
For example, an Oot list is a data type meaning an Oot graph in which the nodes are labeled with a prefix of the natural numbers; various operations can be performed on it including getting the length, appending a member, inserting a member, deleting a member, concatenating two lists, etc. This one data type has many representations; the list could be stored as an array with a length header, as a null-terminated array, as a linked list, etc.
For example, a 'mutable' list could be implemented as a pointer to an immutable list that is switched to point to a different immutable list upon every insertion or deletion.
For example, some representations of a multidimensional array are a a row major contiguous block, a column major contiguous block, a row each of whose elements is a pointer to a column, a column each of whose elements is a pointer to a row, etc.
For example, a Unicode string could be stored in various encodings, could be stored in a struct with an extra field for a memoized hash value, etc.
By default, the Oot toolchain chooses a representation for you. Oot has conventions for providing hints (and also demands) to the toolchain about what performance or other characteristics of the representation would be desirable for a given value, or for requesting or demanding specific representations.
Oot has standard conventions for expressing the exceptional situation in which a value that is a valid member of the ideal type is attempted to be represented in a format that doesn't support it.
Graphs with ports allow us to consider a representation as a 'shape' of graphs; for example, a linear array in memory can be thought of as a graph such that each element except for the first and last ones have a 'predecessor' link and a 'successor' link. This allows us to tell Oot how to map a general graph onto a particular representation.
Oot's so-called representation-agnosticism is similar to one of the key parts of the OOP (object-oriented programming) paradigm, the encapsulation of data along with the methods used to operate upon that data into an 'object', thereby hiding the information of how exactly the data is represented. However, the type system of many popular OOP languages is tied into the idea of inheritance, which often relates to code reuse (an implementation detail) and which encourages programmers to sometimes specify signatures in terms of classes which have many methods, which often forces an object to implement more methods than are really needed for the application at hand. In Java terms, we are talking about the difference between classes and interfaces. In Oot, type signatures only refer to what in Java would be called interfaces.
Oot's so-called representation-agnosticism is to Haskell's typeclasses, which permit the type system to understand a demand for a value that supports operations of a certain signature, without demanding a particular implementation of those operations. However, Haskell also permits types that are not typeclasses, leading to issues such as libraries with signatures that demand linked lists of characters, when you want to pass around ByteStrings?; in addition, Haskell's syntax makes dealing with non-typeclasses more concise than dealing with typeclasses. In Oot, type signatures only refer to what in Haskell would be called typeclasses, and the syntax for these is concise.
Oot provides a system to create patterns that match against graphs. This is used for dispatch (across case statements, and to choose polymorphic function implementations), for structural typing (Oot's typing is 'mostly structural'; it is possible for types with identical signature and graph shape to differ if they have different semantics/behavior), for message handling, and is accessable to the programmer as a 'regex for graphs'.
Oot supports pattern matching over binary data.
Some things that are unified into first-class 'queries':
Also related:
Oot can also express equations on queries. So, for example, you can express that the row accessed by this primary key in database A, and the row accessed by a different primary key in database B, are referring to the same object.
Graphs with ports can be used to tell Oot how a data structure can be queried. For example, a 2-D array can be represented by a graph pattern with ports like 'up', 'down', 'left', and 'right', and Oot can be told that up/down and left/right are inverses, and that up/down and left/right form the vertical and horizontal axes. Oot can then infer that a 2-D array can be queried by giving a position along one or both axes relative to some fixed node, for example, node 0.
todo
type system permits a function to say something like 'pure except for interactions (side effects to / nondeterminism from / reading from / writing to) with the caching_subsystem', where 'caching_subsystem' is a program-defined 'region', or grouping of variables.
regions are also used for concurrency, to say which variables support various types of synchronization/isolation/consistency, or various implicit memory consistency models.
regions are also used for versioning, to say that the values of various variables form a 'possible world', which can be swapped out for other sets of 'possible worlds' (values of these variables) as a unit, or swapped out with other versions (eg undo/redo)
regions are also used as in OOP, to package together a number of variables into a structure that can be conveniently passed around
todo
todo lazy
Key tenants of functional programming supported by Oot include:
todo
Key tenants of structured programming supported by Oot include:
todo
Key tenants of object-oriented programming (OOP) supported by Oot include:
'Neo-language' is a term i made up for certain programming language ideas imported from human languages. Todo
(see https://en.wikipedia.org/wiki/Inferential_programming )
Oot has a smattering of this, for example in the way that one can request a list to be sorted, without expressly indicating which sort operation should be applied, or even whether the function generating the list should be asked to sort it too (if it's capable of this). Oot's distinction between interfaces and implementations might also be considered to be "intentional programming" in that the compiler is permitted to choose the representation.
Some features Oot provides for massive concurrency are:
Oot supports tooling by providing APIs allowing tools to ask the language implementation to do things such as incrementally parse source code, locate which function or scope encloses a given position in source code, find potential auto-completions, etc.
As a text-based language, Oot interoperates well with existing tools such as text editors, build systems, cross-language IDEs, and version control systems.
Oot also strives to provide good tooling including a fast compiler, a canonical source code formatter (ootfmt), good compiler/interpreter error messages, and infrastructure for packaging and deployment.
In addition, Oot libraries and language facilities aim for 'debuggability'.
To this end, Oot is very concerned with interoperability. Oot supports calling to and from, embedding and being embedded, and compiling to and from a handful of popular platforms and languages, with attention paid to preserving idiomaticity, even in round-trip compilations.
Oot supports bit- and byte-level manipulation of memory and has facilities for passing along debugging information such as source code line number. Oot programs can easily run other Oot code in a secure sandbox (although i am not very confident that this is actually very secure yet, as it hasn't yet been analyzed by many people), and can easily be stepped through one 'operation' at a time.
In addition, the very structure of Oot, such as the flexible abstractions of views and boundaries, the ease of attaching 'hidden' meta-data to data using labels and views, the representation-agnostic nature of the type system is designed to support the convenient adaptation of foreign data structures and interfaces; other parts of Oot, such as the focus on graphs, and the powerful meta-programming facilities, support compilation and embedding from or to foreign languages.
This affects the design of Oot, in that we want to provide facilities that make it easy to 'map' Oot programming constructs to code in other languages, and vice versa.
Oot can be used to try out the application of various algorithms, transformation, and visualizations to a data set. It can be run interactively or in notebook mode, linear algebra and array operations can be expressed concisely, and there are libraries supporting common numerical and array operations as well as some plotting.
Oot is a convenient language for writing one-liners from the commandline, and also for writing utility scripts. The Oot interpreter starts up quickly, does not require a lot of imports or introductory boilerplate at the beginning of each program, and can be passed flags for convenient pre- and post-processing steps.
Features to help it load quickly:
The ability to change the program code while the program is still running (see live coding). In addition, to make this work well, Oot tries to make it easy to build robust applications.
The language is not too complex nor too large, there is not a lot of boilerplate to get started, the basic operation of the language is easy to simulate in one's head, it is easy to integrate with GUIs, it runs on cheap client-side platforms that kids are likely to own, and advanced constructs can be ignored until they are wanted.
However, Oot is emphatically a TEXTUAL language, in that the primary representation of program source code is a plaintext file that is intended to be edited with a text editor, in contrast to some other 'learn to program' languages which intend for the programmer to edit their source code via a visualization within a GUI.
As a 'programmable programming language', Oot should be a great vehicle for prototyping and perhaps even writing new programming languages.
Many of the same facilities that make Oot interoperable (graph-based data structures, ease of attaching metadata to data, metaprogramming) also make it a good target for prototyping new programming languages.
In addition, its emphasis on being a good 'glue language', able to use other languages' libraries, and to serve as a bridge between libraries in two different other languages, should make it possible for new languages built with Oot to have immediate compatibility with a large set of libraries from popular languages, easing adoption.
todo: cp everything above '== Syntax ==', above, into this table
Oot is built upon a single powerful data structure, the Oot network (net).
Lisp is a homeoiconic language built upon lists. Oot is a homeoiconic language build upon networks (labeled directed graphs).
Semantically, a linked list, a doubly-linked list, an unboxed list (vector), and a hash table with natural number keys are all similar; they are all lists of things indexed by natural numbers, all of which support sequential access, random access, appending, insertion, and deletion. The differences are the space and time complexity of these various operations. In Oot, these things all look the same; if you want to choose one or the other for efficiency reasons, you use type attributes (a kind of annotation) to specify.
In Oot, the following two operations look the same: (a) looking up the third element of the array in variable "x", and (b) applying the function in the variable "x" to the number 3 and getting the result. This allows library functions to present a simple interface that can be thought of in terms of arrays, while allowing the caller to use arbitrary objects in place of those arrays.
Furthermore, fields on structures are treated the same way. This makes it easy to replace a structure that formerly used simple fields with one that uses intelligent getters and setters under the hood (this supports the so-called http://en.wikipedia.org/wiki/Uniform_access_principle ).
Data operations in Oot are, for the most part, expressed as functions on immutable values, rather than as changes to mutable data. Those that are not are clearly demarcated.
However, there is an exception. We freely allow 'local mutation', that is, mutation that takes the form of assigning a value to a symbol within a scope and then later reassigning a different value to that same symbol within the same scope. This kind of mutation could in theory be eliminated by renaming every mention of the variable after the reassignment (and doing some copying in the case of multiple branches, e.g. if the reassignment occurred in only one branch of an 'if' statement). We feel that this is useful to avoid having to create throwaway names for intermediate variables.
We feel that local mutation isn't bad; the bad (confusing) stuff is when you have aliasing of references, that is, when a change to the value of one variable 'magically' causes a change to a different variable (especially when the two variables are located in two different scopes, neither of which is an ancestor of the other).
Oot has a system to allow various frameworks to attach their metadata to values.
The Oot nets are labeled, directed, multi, hyper, reified graphs. Oot networks are:
Nodes in Oot nets can be arbitrary functions.
Each node in a net can used as a simple array or lookup table. However, Oot supports more powerful idioms that regard the network in its totality, rather than each node individually. For example:
Nets can be used to represent regular multidimensional labeled arrays, similar to R dataframes.
A net boundary is a border that cuts through a set of edges in the graph.
Boundaries in code are used to represent atomic operations, exception catching domains, etc.
Net patterns are a matching language that generalize regular expressions and database queries to nets. Net patterns are first-class objects.
A view is a different set of labels and edges associated with the same nodes. Views allow the same data to be presented in different ways to different functions operating upon it. For example, a dictionary could be presented as a list of key, value pairs, and mutations on this list could alter the dictionary.
A mutation to data in one view can affect the same data in other views. When one view is a projection of another, syntax is provided to specify the 'lineage' of data so that metadata attached to a node N1 in another view remains attached to the correct node(s) if N1 is mutated by way of replacing it with a new node.
Since network nodes can be functions as well as static lookup tables, they can be passed around as data.
todo
Oot has a lazy evaluation strategy; values aren't evaluated until they are needed. This allows for separation between definition of data structures in the abstract, and the control flow used to traverse them. For example, if you are writing a chess-playing program, you can represent the entire tree of possible chess board states in a data structure, and worry about how and how far you traverse this unrealized data structure elsewhere.
A useful special case is infinite data structures; for example, you can have an array which contains all prime numbers; unless you try to traverse this array arbitrarily far, this does not cause your program to hang because thhe unaccess members of the infinite array are never computed.
In other lazy languages (e.g. Haskell), it can be hard to reason about and memory leaks (e.g. "space leaks"), or to discover them via profiling. Oot makes this easier.
In addition to providing operators like Haskell's seq and deepseq, which forces strictness in a top-down manner, Oot allows you to mark data values as eager; a function fed eager inputs will eagerly evaluate them. This forces strictness in a bottom-up manner.
Oot presents the programmer with abstractions like 'fold'. The choice between functions corresponding to Haskell's foldl, foldl', foldr, foldr' is made by the compiler based on strictness analysis of the arguments. The sort of analysis discussion on this page ( http://www.haskell.org/haskellwiki/Stack_overflow ) is done by the compiler.
Oot automatically converts 'almost tail recursive' functions with only associative operations preventing them from being tail recursive, to tail recursive accumulating ones.
TODO
Oot can eliminate recursive tail calls, allowing efficient deeply nested recursion.
Oot allows functions to have multiple entry points (this is sort of like a object with multiple methods that is only executed once). The function can synchronize on conditions between these entry points before executing. The notation is uniform with message handling (see below).
For the most part, Oot "long-distance" operations are referentially transparent. Those that are not are clearly demarcated, allowing the programmer to steer clear of unwanted side effects and unexpected shared state.
You can ask the compiler whether any function is referentially transparent. This is accomplished in a similar way to Haskell's use of monads to type non-referentially transparent functions as 'tainted', but it is done implicitly without the need for the programmer to worry about those types or to manually change a bunch of type annotations if you decide to make something non-referentially transparent later.
You can do non-referentially transparent operations without tainting for the purpose of debugging and profiling, in a manner similar to Haskell's unsafeIO.
You can do seemingly non-refentially transparent things without tainting if you promise the compiler that their behavior is actually referentially transparent.
Variable scope is simply defined by the block of code that they are present in, except that there is also a way to refer to the "object" of which a function is a part (a "method" in other languages).
Closures and References are supported but any variable containing a reference or a structure containing a reference are clearly distinguished from values with sigils.
Although we use lexical scoping by default, we also have dynamically scoped variables.
Oot provides a powerful, uniform, concurrency-friendly mechanism for messages to propagate and be handled.
This is used for event-driven programming, exception handling, dispatch, and parallelization.
Oot has a zoo of powerful constructs for correct, concurrent programming.
todo (see zoo in [1])
Error handling for pure functions is through Maybe (option) types. Maybe types can be subclassed to give more information on the type of error when a Nothing is returned.
Error handling for mutation is thru exceptions, however Oot provides a concise operator to convert a return value from a function that might throw an exception into one that returns a Maybe, and vice versa.
Like D's, these allow you to place code that undoes side effects in the event of an error right after the code creating the side effect.
Oot provides a way for side-effectful code to specify how these side-effects can be reversed, if they can be at all. Oot can use this to allow the programmer to create transactions.
Like C++ RAII or Python 'with' statements, you can ensure that resources are freed by use of an object protocol.
If the caller uses a 'with' (todo find a different name), this occurs as the stack is unwound, otherwise it occurs when the object is garbage collected (unless there is a reference cycle of objects which reference each other in their cleanup routine, in which case a runtime warning is generated).
(todo: if the protocol mixes stuff like Python __del__ and Python context mananger __exit__, then what to do if an object in a 'with' goes out of scope but participates in a reference cycle of other objects with finalizers?)
Oot has a powerful module system and a built-in runtime component system with facilities for directories of components, dependencies and capabilities, configuration, parameterization of types, declaration and binding to extension points, typed messaging, and event handling.
Oot allows you to flexibly inherit from or to compose new code with old.
Oot allows you to encapsulate data structure definitions with operations to act on those structures.
In Oot, the philosophy is that the type system is your servant, not your master. We see static typing as a way to allow the compiler to prove theorems about your program. These theorems are useful for (a) autocompletion in IDEs, (b) preventing bugs, and (c) optimizing execution.
Because there is a cost (the programmer must do some extra work to help the compiler prove the theorems), static typing is optional; by default, all variables are dynamically typed, and you can enable static typing for everything or for any subset of variables that you choose.
To save you time writing type annotations, Oot can infer some of them.
The type system is powerful and extensible. We support generics. We support type attributes such as 'immutable' and 'unique' that constrain, not the structure of data, but rather how it can be accessed. We allow the user to define custom attributes of this sort themselves. The type system is extensible enough that you can use it to ensure the correctness of all sorts of things, using techniques such as loop invariants and programming-by-contract.
However, the type system's power is limited in order to make code easier to understand.
One important principal is that, when reading code, complex reasoning about static type inference should not be necessary to figure out what the code does.
For example, in many ways Oot's type system is less powerful than Haskell's; we limit the power of the type system to (combine type inference with polymorphism in order to do compile-type computations that alter or determine the semantics of code), e.g. stuff like http://stackoverflow.com/questions/3467279/how-to-create-a-polyvariadic-haskell-function .
Although the type system allows you to express complex statically-checked constraints on semantics, for the most part this power can only be used to doublecheck the semantics that are explicitly expressed in code, rather than to resolve semantics that the rest of the code leaves ambiguous.
Although Oot has inheritance, the type system will never require you to inherit from anything or to use a certain implementation of a data type; everything is an interface.
In Oot, you don't think that a value IS a type, for example "3 is an Int". Rather, you think that value X has a type attribute, for example, "3 has the property of conforming to the Int interface". "3" might have other properties as well, for example, it might conform to the Num interface, and it might conform to the Serializable interface. The Serializable type attribute is no more fundamental to 3 than the Int type attribute (at least as far as the type system is concerned).
You don't have to inherit from another implementation of something in order to make a replacement for it; you just have to implement the appropriate interface.
If you want to make something to replace a duck, you don't have to convince the compiler that it IS a duck; you just have to make something that walks like a duck and quacks like a duck.
For example, you can look at the type List of homogeneous lists to be a function that takes a type value (the type of the elements in the list) and returns a type value (the type of a list of those elements). The notation for this sort of thing is the same as for ordinary functions.
The type checker is your servant, not your master. If you want some amount of typechecking or static assertion verification on a variable or module, but there is some piece of code for which you don't want to bother right now to write a proof that it does what it should, you can instruct the type checker to simply assume the result that you want.
Oot provides a Coq-ish decidable theorem prover with customizable tactics to allow you to express and prove difficult assertions about your code.
I am NOT suggesting that you will have to write a proof in order to get your program to type check; the proof checker is provided so that you can (a) opt to encode complex invariants that the compiler verifies, with your help; (b) extend and customize the type system.
Oot has facilities to permit convenient serialization of any type.
Oot has a simple, near-homoiconic syntax with an emphasis on conciseness and readability.
Oot's lexical syntax is based on regular experssions, and its parse syntax is LL(1) and does not require a symbol table, making it easy to parse.
Most of Oot's syntax is for grouping (and quotation, which is a form of grouping).
todo
Oot was designed to minimize the necessity for delimiters. For example, a function call that in many language would be written "f(x)" is just "f x" in Oot.
Oot has syntax for assignment to an element of an array, a common operation that some languages lack concise, readable syntax for.
Oot does not permit custom operator precedence because that would require you to to look up operator definitions before you can parse code that you are reading.
Footnotes are a convention that allow you to present "the main part" of a function uninterrupted, and then to handle error checking, logging, variadic argument checking, parallelism annotations, other annotations, etc in the footnotes.
You can pass in arguments to a function by keyword instead of by position. Keyword arguments can be optional.
But not significant indentation.
In Oot, the presence or absence of (a) space in between words, (b) newlines, (c) an empty line (a line with no non-whitespace characters) are significant, but the amount of these (indentation level, etc) is not. Any code can also be written in a "one-liner" without whitespace.
If a function takes two arguments and you give it one argument, the result is a partially applied function.
Oot return values mirror function arguments; they can be many of them, they can be labeled by keywords, and they can be optional.
You can create vector variables such that when you apply ordinary scalar operations to them, they are applied to each item in the vector, without having to extend the definitions of each operation to be able to handle vectors. There is a generic way of extending operators to act in ways such as this.
You can represent mathematical 'Del' in Oot, and Del(f) = gradient of f, dot(Del, v) = divergence of v, cross(Del, v) = curl of v.
In statistics, some operations apply to values (and expressions act as if each random variable in the expression had been instantiated to a particular value), and others apply to expressions and care about the random variables in them. For example, if X is a random variable, "X + 3" is the former, but "var[X]" is the latter; if y is a 'normal' (non-random) variable, and Y is a random variable, then when simplifying, var[X + y] is treated differently from var[X + Y], because in the latter case we have to know if Y is correlated to X.
Oot uses ?x to mark x as a 'metavariable', which means different things in different contexts; in the context of statistics, a metavariable is a random variable. Functions which act differently when given expressions with metavariables ar distinguished with a ? suffix; so we have mean? and var?.
However, every operator must also have an alphanumeric function name, so you can use those if you don't like line noise.
Oot's approach to metaprogramming is to provide a ladder of less- to more- powerful metaprogramming constructs. Programmers are encouraged to stick to the less powerful constructs when possible to promote readability, but the more powerful ones are there in case you need them.
Oot supports first-class delimited continuations, which means essentially a first-class abstraction of the state of part of the stack.
todo
Hygenic macros are applied to source code after Oot parsing but before anything else.
Optionally, the hygenicity can be weakened with with inner keyword/keypatterns, that allow the macro to access the actual string naming a symbol, allowing the creation of frameworks similar to Ruby on Rails, which associates symbols to database tables based on their name. Such symbols have sigils and/or capitalization to allow readers to recognize that their name is important.
These allow you to write custom handlers for strings within the source code that will be exempted from Oot parsing. For example, regexes are implemented using these. These can also be used to intermix foreign language source code with a Oot program.
The Oot executable has an API that provides services for IDEs, for example, online parsing, online type inference, querying the type attributes of an instance of a token, querying for symbols meeting certain criteria, autocompletion, querying potential exceptions thrown by a function, and more.
Oot has a standard notation for documentation markup.
Out of the box, Oot can call, be called by, and pass data to and from various popular languages. Oot semantics and data structures are available from other languages via a standalone library. Oot is available within various host platforms, such as JVM and .Net.
Like C and Rust, Oot can express low-level concepts, allowing it to interoperate with other languages.
Oot can read and write to memory locations, and Oot programmers can create data structures which specify their exact layout in memory. Oot programmers can manually allocate and deallocate memory, and can perform pointer arithmetic.
An illegal memory access produces a Nothing or an exception, not a segfault (unless the programmer turns off this checking for the sake of speed).
In the tradition of Erlang and Go, Oot manages lightweight threads and prevents starvation.
Erlang-ish interthread robustness facilities are provided.
By default, Oot takes care of memory management, but you can opt to take control yourself if you need to.
ootfmt
Copy-on-write (COW) makes immutable value semantics efficient.
Oot is written in Oot.
The Oot implementation uses its own metaprogramming facilities to bootstrap itself off of a small subset of the language, Oot Core. This means that in order to port Oot, all you have to do is to implement Oot Core on your new platform. (as a side effect, Oot Core's spec is short)
Oot supports execution of untrusted code with limited capabilities. (note: while this is a design goal of Oot, it is likely that the initial Oot implmentations will have unintentional flaws, and so at least initially the sandboxing should not be trusted for high-value use cases).
You can alter and extend Oot by writing your own modules to be used in the compiler.
To allow you to experiment with a running program in the interpreter without stopping, recompiling, rerunning, and getting back to the state you were in, the Oot interpreter allows you to dynamically inspect and alter the functions in memory during execution.
To allow you to edit source code files and re-run the program without manually giving the compile command, the Oot interpreter will by default recompile changed source code dependencies when told to execute Oot source.
A Oot program can optionally be compiled into a static binary that includes all dependencies, making it easier to distribute single binary to production.
Package manager that knowns about dependencies, signatures, volume of documentation, and canonicalness. The package mananger will almost never get 'stuck' in a state where it cannot install required dependencies. The package manager can get packages from the local file system, from a remote git project, or from a Oot package website.
A governance process that selects a subset of libraries as more or less canonical, yet not so canonical that they are included in the distribution.
Oot is licensed under the permissive MIT/X11 license.
One motivation for Oot is simply to make a useful programming language. Another motivation is to create a programming language in which it is easy to program in a paradigm which is my best guess for how the brain works. This should be useful because we know that the brain is capable of useful computation, and also in order to assist the cognitive studies students among us in gaining hands-on intuition about how the brain might compute.
Some tentative properties of this 'Oot/brain' paradigm are:
By default, Oot does not do anything that could cause a crash (SEGFAULT), such as dereferencing arbitrary pointers, or accessing an array without bounds checking. However, the confident programmer can turn this off for speed.
Oot contains a way for you to differentiate between a public interface and private implementation details, so that you will not break downstream programs when you change private implementation details.
todo rly?
Oot has a great standard library.
Oot has a well-designed library of collections.
They include:
Many of these can be treated as simply graphs; their implementation is just an implementation detail.
Oot can operate on relations.
Oot can operate on partial orders.
Oot's library has many of the functions in the Haskell prelude.
The operations of getting the length of a sequence and of generating a range of integers (or floating point or other numbers) has quite concise syntax.
The mathematical integers, and multiple/arbitrary-precision arithmetic, are primary in Oot; fixed-width number representations are secondary.
---
strong at lazy purely functional like Haskell, but easy to read like Python
conducive to macros like Lisp, but communal gravity like Python (and many other languages)
like Lisp, but based on labeled graphs (keyword arguments) rather than lists (positional arguments) (and more focus on readability; and slightly higher-level)
shell scripting: good for one-liners like Perl, but easy to read like Python
good for numerical computation like Python (and maybe but with less impedence mismatch between lists/numpy arrays and between method chaining and function call chaining) and Octave (but general-purpose language)
small-ish like (roots of) Lisp, Scheme, Nock, C, Lua
massively concurrent
robust concurrency like erlang
well-designed like C, Python
encourages readable code like Python
good programming language for prototyping/writing programming languages in, like Racket
like squeak, but based on plaintext source code files, and tooling that interoperates more with the outside world (e.g. conventional text-based source code editing, IDEs) (see also http://stackoverflow.com/questions/5833619/how-can-i-dump-all-the-source-code-from-a-squeak-smalltalk-image )
reference implementations and prototyping of networking protocols and components (since performance is not a goal it may not be the best language to write productions versions in, though)
performance is not a goal (although we would like to architect the language so that in theory it could be made performant later; however, without actually trying to make it performant we'll probably miss lots of stuff)
like perl, good for writing one-liners from the commandline (including syntactic sugar for regexes), but readable like python
a 'glue language': have an existing application written in Python, and want to keep developing it using Oot? That's easy. Have an application in Haskell, want to keep developing it using Oot, while making use of some libraries in Python and Java? That's easy.
Why not Python? Python is my favorite language but i'd like more support for: concurrency (no GIL!), immutable data structures, a single data structure instead of lists/dicts/numpy arrays, concise partial function application and libraries based on higher-order functions, metaprogrammability, optional static typing, laziness, commandline oneliners. Why not Haskell? I'd like more support for: readability, typeclasses everywhere, a simpler, optional type system, keyword arguments, simpler reasoning about memory usage, standard error handling, uniform collections libraries, mutability, conciseness, simpler I/O, simpler monad composition, less visible monads. Why not C or C++? C is too close to the metal. Why not Perl? Perl is too hard to read. Why not Java? Many of the same reasons as Python; also, it's too verbose. Why not Lisp/Scheme/Clojure? Not enough syntax (specifically: (1) I think being able to say something like v[a] = 10 is easier to read than (assoc v a 10), and M[b][c] = 10 is easier to read than (mset! M b c 10) (see http://stackoverflow.com/questions/16007062/ways-to-quickly-update-an-element-of-a-matrix-in-incanter-clojure ); (2) too many parens), and the JVM startup time is too slow for commandline oneliners.
---
todo not in prioritized order, not in correct major/minor/supp lists
---
some old versions :
Simplicity means minimization of the number and difficulty of concepts and details. (todo do better here; it's really something having to do with the difficulty of thinking in the language, and of remembering the language)
simple, small
todo
layered: core, language, standard libraries
expressive
todo
relation to powerful: if by 'powerful', it is meant, 'enough features so that i can do what i want without writing much new code', then we are focused not so much on providing every desirable feature within the language, but rather upon making the language expressive enough so that it's possible to write a library for that feature that allows source code using it to read naturally.
todo mechanistic concepts are simpler than others
'expressing computation in a natural way rather than bending it into a non-intuitive form to fit it into the language's paradigm'
by readability we mean that the time necessary for an advanced Oot programmer to understand someone else's source code is minimal.
todo
Oot strives to have 'scannable' syntax; meaning that your eye can quickly scan through code looking for where a certain thing occurs.
todo
Also, a word about simplicity, size, and learnability. Oot is a simple, small language which is easy to remember. However, although the language is small, it has a large standard library, so in order to become fluent in Oot idioms and read others' code there is a bit to learn. Oot is focused on being easy to remember, but not necessarily easy to learn; what I mean is that it may take you more than a few days to grok it, but once groked, you should be able to work on it on weekends every now and then without having to spend a lot of time getting back up to speed.
todo
relation to concise: we value conciseness but we see this as only a means to the ends of expressivity and readability. Insofar as 'concise' just means 'i want to write it the way i think of it', that's what we're calling expressiveness. Insofar as lack of boilerplate makes it easier to see what code is doing, we value reduction of boilerplate. Insofar as making programs shorter makes them easier to quickly understand, we value making them shorter. But insofar as 'concise' means 'Making programs really short by inventing non-intuitive abstract operators that must be 'unpacked' by the reader to understand the program', we disagree. We would rather have you write more code if that will help someone else understand it quicker.
(note: non-intuitive abstract operators are fine with us if their usefulness is broad enough to make it worthwhile for every advanced user of the language to learn them; but not if they are application domain specific. But we still don't want to add too many non-intuitive abstract operators to the language, because that impairs our other goal of simplicity.).
Our brain's neurons are individually slower than computer CPUs, but it makes up for this by having a lot of them. Moore's Law isn't increasing CPU clock speeds at an exponential rate anymore, but instead it is giving us the possibility of many processors working in parallel. Together, these show that it is possible to do interesting things with massive concurrency, if only we knew how to write software that could use it. Oot is a platform for experimenting with ways of writing massively concurrent software.
todo
---
To put it another way, cryptic code becomes necessary only when a language does not provide the means to express the program in the same shape that the writer thinks of it.
--
numbers of threads in the tens of thousands or more. At this scale it should be possible to have one thread for each data element
todo
Our brain's neurons are individually slower than computer CPUs, but it makes up for this by having a lot of them. Moore's Law isn't increasing CPU clock speeds at an exponential rate anymore, but instead it is giving us the possibility of many processors working in parallel. Together, these show that it is possible to do interesting things with massive concurrency, if only we knew how to write software that could use it. Oot is a platform for experimenting with ways of writing massively concurrent software. todo shorten this or throw this out.
---
note: in addition to this one composite structuring element, there there are still multiple atomic data types, such as integers, etc).
---
We're not going to give examples of Oot's readability here, because (a) readable-looking examples can be contrived even for unreadable languages, and (b) Oot may look alien at first; it becomes readable only after you learn the language.
---
high-level: opinionated, for less-leakly abstractions in which the tradeoff is performance (rather than flexibility)
synonyms that mean 'to have something as a goal': focus, concentrate, attention paid to, oriented towards, prioritize, emphasize, geared towards, feature, stress, support
should make sure to include the features in the feature list in the 2009 Steering Committee Position Statement, see http://bayleshanks.com/proj-plbook-plChLispLangs :
exceptions
modules
concurrency
unicode text
union types
record types
abstract data typesand probably the listed Scheme features too:
lexical scope
dynamic typing
list structure
higher-order functions
proper tail recursion
garbage collection
macros
s-expression based lexical syntax. ---
