proj-jasper-whyJasper

note that none of this has been implemented yet, and the design is still in flux! All of these items, although written as present-tense facts, are in fact future hypothetical wishlist items! I am just writing down ideas. I have many pages of notes on things i may or may not want to put into Jasper. At this point, the design is unfinished; there is no coherent concept of a language called 'Jasper', just a bunch of ideas towards such a proposal. There is no timeframe to actually implement this, and it most likely will never get finished. To be clear, i repeat myself: there is no such thing as Jasper. Jasper is not even vaporware. Jasper has neither been designed nor implemented. No one is seriously planning to create Jasper within any reasonable timeframe. These notes about 'Jasper' contain many conflicting proposals.

Why Jasper

Intro

Jasper is a readable, powerful programming language focused on massive concurrency. It has a single fundamental data structure, graphs. It is a high-level, programmable programming language.

Jasper is a multi-paradigm language with elements of the lazy functional, imperative, logic, and object-oriented paradigms. Jasper supports optional static typing (higher-order polymorphic gradual typing), memory management, and advanced error handling. It is a practical language with tooling, packaging, and deployment infrastructure, and great libraries which are geared towards immutable data structures. It is open source with a permissive license.

Jasper is a general-purpose language that particularly targets the application domains of expression of computation, exploration of brain-like computational architectures, glue language, exploratory numerical computing, command-line one-liners, client-side application programs (web, desktop, mobile), live coding, learning to program, and writing new programming languages.

Some downsides of Jasper are: (a) lack of speed, (b) memory usage, (c) its goals are insanely ambitous, but its creator is an talentless hack.

Jasper aims to combine the readability and ease of Python, the functional laziness of Haskell, the commandline suitability of Perl, the straighforwardness of C, and the simplicity of BASIC (almost), with metaprogramming and massive concurrency.

Next, we're going to expand on each part of the above. Then, we'll go back and give lists of features associated with each goal.

Readable

Jasper's number one concern is to be readable. We define readable to mean that a moderately experienced Jasper programmer can quickly understand someone else's source code.

Some languages (for example, Perl) tend to encourage newcomers to write 'ugly' and hence hard-to-read code. Other languages (for example, Haskell) tend to encourage experts to write 'fancy' and hence hard-to-read code. An example of a language which does neither is Python. That is the sort of thing we strive for.

As a means to the end of readability, we pursue various other design criteria. Readability implies that code does what you think it will do, so Jasper must minimize 'gotchas' (this criteria is also known as the principal of least surprise or least astonishment). This is related to regularity, because special cases are easily forgotten (causing gotchas); and also to language size and simplicity, because in order to understand source code you must recall and understand the language features it uses, and it's hard to remember a large number of of rarely used features, or to understand complex ones. Concise code is often more readable, in that a small program is easier to hold in one's head (and hence to quickly understand).

Powerful

We want our language to be all of:

We define power as maximizing the range of ideas that can be succinctly and naturally expresssed with the language, while minimizing complexity (complexity is the opposite of simplicity) (the use of the word 'power' to denote this concept is inspired by the physics word 'power', meaning energy per unit time). This is a tradeoff. For any language, there are ideas that are difficult to express, or to express concisely, with that language. Any particular idea can be added to the language in the form of a new feature. But each time this is done, the cost is that the language becomes more difficult to understand and to remember. So, maximizing power in a language is like maximizing return-on-investment or profit in a business; add ideas to your language only when you get a lot of "bang for your buck". Each feature must increase expressivity or succintness enough to justify its complexity.

An expressive language is one in which programs can be written the way that you think of them. Expressiveness also aids readability, because if the writer can write a program the way they think of it, the reader does not have to think as hard to figure out the motivation behind it.

todo examples

A succinct language is one in which programs can be relatively short. When the goals of succinctness and readability conflict, Jasper favors readability.

For example, Jasper favors the ML function calling syntax (eg 'f x') instead of the traditional mathematical one ('f(x)'), because it is more concise.

todo examples

One strategy that Jasper uses to achieve power is to try to unify what in other languages would be many concepts, into one general concept, allowing us to pay the complexity cost once and reap many applications. Examples of this strategy:

Massive concurrency

By 'massive' we mean situations where the number of concurrently executing threads may be comparable to the number of elements of data being operated upon (for example, one thread for each pixel in an image). (this scale is clearly ill-matched to the number of CPUs in present-day computers, so why would you want this? See 'exploration of brain-like computational architectures', below).

Some features Jasper provides for massive concurrency are:

One powerful data structure

When many operations are provided for a few pervasive data structures, it becomes easy to compose functionality. Lisp and Haskell have lists. Python has lists and dicts. Can we do better? Jasper has one fundamental data structure, the Jasper graph. Lists, associative arrays, matrices, dataframes, relations, partial orders, and more are all special cases of Jasper graphs.

Jasper graphs

In pursuit of as much generality as possible, Jasper's graphs are directed, labeled, reified, multi, hyper graphs. Directed, meaning that there may be an arcs from one node to another one, without an arc from the second to the first. Labeled, meaning that nodes and arcs may have data attached to them. Reified, meaning that the target of an arc may be a node, or another arc, or it might be the graph as a whole. Multi, meaning that there may be multiple arcs going from one node to another node. Hyper, meaning that we generalize binary arcs, which have a source and a target, to n-ary arcs, which involve a number of nodes in a number of roles.

Views

A value may have different representations in memory. For example, the integer '-1' might be represented as an 8-bit integer with a sign bit, as an 8-bit integer with twos complement, as a 16-bit integer with a sign bit, etc.

Often we find that one piece of data must be represented in multiple forms because the different functions that need to operate on it expect different data types. For example, a persistence framework may wish to attach metadata to persisted arrays, but this metadata must be hidden when the arrays are to be passed to library that expects plain arrays, yet still present after the mutated arrays are returned. Another example is when a tree data structure must appear as a linear array of nodes for the purpose of passing to a filtering function, but in actuality must retain its tree structure after the undesirable nodes have been filtered out.

Popular constructs for this purpose include type classes and OOP ad-hoc polymorphism; both of which work via the creation of an interface in the form of a set of functions supported by a type (or type class). To borrow the language of RESTful web architecture, this solution amounts to the creation of a new set of verbs for each interface.

Jasper discourages types defined by the unnecessary creation of verbs in cases where they can instead be defined more simply as graphs conforming to a certain shape. This allows the pervasive graph operations to be applied to more types. For example, trees and arrays are both graphs with different shape constraints. If data in the shape of a tree needs to be filtered by a function that expects a linear list, instead of creating an adaptor that exposes a set of functions such as next(), the Jasper way is to specify the shape transformation that says, for each node in the tree, which item in the list it corresponds to.

To assist in this, Jasper provides the concept of 'views'. Jasper views allow the same piece of data to be viewed in different ways. They are defined by a specification of the shape transformations (mathematically, homomorphisms) which define the various representations of the same piece of data. A mutation of the data in any of its views affects it in all views. When this sort of thing is implemented in other languages it tends to involve things such as pointers linking together data fields within distinct variables, but by contrast, Jasper presents the abstraction of a single variable that holds a single value which can be seen from different views. Views are a type of ad-hoc polymorphism, a way for a piece of data to expose a different 'interface' to various consumers, but these interfaces differ via shape rather than via a distinct function signature.

Binding

todo

like spreadsheets

Domains

binding is tightly restricted between different state domains (or just 'domains')

todo

Boundaries

Nodes in a graph might correspond to values of a given data type; for example, you might have a graph whose nodes have integer values. But what happens when the type of the nodes itself has a graph structure? For example, if you have a data type A, where A is a list whose nodes are type B, where B is the type of arrays of integers? You might conceptualize an instance of A in two ways; first, as a simple list-shaped graph with opaque nodes of type A; or second, as one big graph whose nodes are integers. The question at hand is: when do we stop recursing into node values and decide to just consider something an opaque 'node'? The difference is not merely academic; for example, when a value of type A is operated upon by a generic library function that visits each node in a graph and applies some transformation, different results will obtain depending whether the graph is composed of arrays of integers, or integers.

Jasper allows the programmer to fluidly move between these conceptions via the construct of 'boundaries'. A boundary can be visualized as a wall surrouding nodes in a graph; any remaining structure inside the wall is considered opaque (until such time as the boundary is moved or deactivated). A single graph can be marked with multiple boundaries of different 'color', only one of which is considered the 'active boundary color' at any one time.

Boundaries seem even more important when one contemplates the binding together of different variables in a Jasper state domain. In essence, each Jasper state domain could be considered as one big graph. The question of where one variable stops and another begins becomes merely a question of boundaries. Taking this viewpoint even further, the different state domains within a program can themselves be thought of components of a larger graph, a single graph containing the contents of all of the variables of the program as subgraphs, separated by various boundaries.

High-level

Jasper works at a high level of abstraction. This means that you can write abstract code which is flexible enough to be adapted to various uses. It also means that you can express what you want a program to do, and Jasper will handle some implementation details for you.

todo

flexible

safe (no segfaults, unless of course you call out to an extension/library written in another language)

todo

The goal of 'handling implementation details for you' can add complexity. Under what circumstances is it worth adding this sort of thing to the language, rather than leaving it to be built in a framework?

Jasper's approach to this tradeoff is:

A programmable programming language

It is inevitable that new programming language constructs will be discovered, and also that there will be application domains for which Jasper is almost enough, except for the omission of a few constructs. todo

A language built upon a small core language is also more portable, as only the core really needs to be implemented on the underlying platform, although further optimization can be had by providing platform-optimized versions of other elements of the language.

What's all this you hear from those Lisp guys about a style of programming where first you augment the programming language to suit your domain, then you write a program in your new language? We'd like to do that too.

And what's all this about a small language, a gem-like thing of perfect beauty, in which can be written a (meta-circular) interpreter for itself in only a few pages? Around which a larger, more practical language can be built? That's pretty cool, also.

As a bonus, the effort to find a set of primitives that are good for building a larger language helps us find powerful primitives, and a self-hosting language built on a small core is easier to port. Furthermore, if the language is small enough, and the abstractions are powerful enough to allow it to be implemented concisely, then the language implementation will be readable by ordinary mortals (sans a million optimizations, but we can keep them in a separate module).

todo list meta-programming features

However, we are aware that use of meta-programming features can make source code hard for others to read. Jasper provides an (approximate) ladder of meta-programming mechanisms, and encourages the use of the less powerful mechanisms when possible (temperance will also allow us to optimize your code more). The most powerful metaprogramming mechanisms are only for use when all else fails.

todo self-hosting

Jasper itself is made of three parts; Jasper Core; a language written in Jasper Core using metaprogramming; and a large standard library of language-like constructs.

Jasper strives to be small, but not minimalistic. For example, 'nand' gates are universal for Boolean functions, but instead Jasper bases its Boolean functions on 'and', 'or', and 'not'. Furthermore, we do not try to be minimalistic in the sense that we aren't trying to provide an orthogonal basis set of core primitives. Rather, we strive to be 'multi-paradigm'; when there are multiple paradigms, each with their own characteristic primitives (and when each set of primitives cannot be expressed in terms of another paradigm's primitives except in a clunky or non-local fashion), we try to provide some close to the union of these primitives (or at least all of the 'popular' ones), so that the language can express any of these paradigms.

Used well, metaprogramming can make programs more readable, by making the language more expressive in exactly the ways that are needed for a particular program; the time cost of understanding the black magic is repaid in making it quicker to read the rest of the program. However, a downside of metaprogramming is that it can also be (ab)used to make code that is concise but cryptic.

multi-paradigm

todo

Laziness

Jasper has a lazy evaluation strategy; values aren't evaluated until they are needed. This allows for separation between definition of data structures in the abstract, and the control flow used to traverse them. For example, if you are writing a chess-playing program, you can represent the entire tree of possible chess board states in a data structure, and worry about how and how far you traverse this unrealized data structure elsewhere.

A useful special case is infinite data structures; for example, you can have an array which contains all prime numbers; unless you try to traverse this array arbitrarily far, this does not cause your program to hang because thhe unaccessed members of the infinite array are never computed.

Lazy evaluation isn't always efficient, and the programmer can request strictness where desired.

(todo: actually, we're lenient; non-strict, but laziness isn't guaranteed either)

type system

todo

attributes

higher-order polymorphic

supports both parametric polymorphism (generics) and ad-hoc polymorphism

Operator overloading and polymorphism is useful for to allow the names of library functions which perform analogous operations on different types of data to be made more succinct and memorable by allowing the same function name to indicate different (but, by convention, closely related) functions depending on the type of the arguments. For example, '+' can refer to both integer addition and floating point addition depending on the types of its arguments; this is easier to remember than having a separate 'iadd' and 'fadd', and also more succinct (since the need to distingush between fewer function names means that the length of these names can be shorter).

Gradual typing

success typing

can use assertions to override/escape type system

"everything is an interface"

memory management

todo

Safety and error handling

Mature code contains a lot of error handling, and programmers spend a lot of time debugging. Therefore, a primary concern of Jasper is to reduce bugs (safety) and to provide convenient, readable mechanisms for handling errors.

Jasper's approach to safety is to provide strong but optional safety constructs, and to provide an API to allow IDEs to determine which parts of code are safe. Two examples of this are its approach to static vs. dynamic typing, and to checked vs. unchecked exceptions.

Jasper has gradual typing, meaning that you can write code without type annotations (and Jasper will use dynamic typing to execute this code), or you can annotate some or all of your variables (allowing the compiler to typecheck your code, and also to achieve better performance by omitting dynamic typechecks). Similarly, by default Jasper doesn't force you to handle or declare every type of exception that could be thrown by your code (that is, we don't have checked exceptions). Instead, Jasper provides an API that allows IDEs to query Jasper and ask which unhandled exceptions could be thrown by a given piece of code, and which variables are dynamically typed.

For larger projects which need more safety, Jasper offers a 'strict' mode which requires all variables to be statically typed, and all exceptions to be handled.

todo

Tooling

Jasper supports tooling by providing APIs allowing tools to ask the language implementation to do things such as incrementally parse source code, locate which function or scope encloses a given position in source code, find potential auto-completions, etc.

As a text-based language, Jasper interoperates well with existing tools such as text editors, build systems, cross-language IDEs, and version control systems.

Jasper also strives to provide good tooling including a fast compiler, a canonical source code formatter (jasperfmt), and good compiler/interpreter error messages.

Libraries

Much of the practical utility of a language comes from its libraries. We value great libraries, and both the language and the community process are designed to encourage this.

Jasper attempts to provide the power of meta-programming while avoiding the curse of Lisp (namely that libraries in a very powerful language have trouble creating enough 'gravity' to attract contributors).

todo

open source

A language is most useful when it is used by many people, so Jasper's source code is made available under a permissive open-source license.

todo: which license? Probably Apache? BSD/Apache/MIT? triple disjunctive license? or, aren't those already covertible?

Application Domains

expression of computation

What is a programming language for? Jasper is for two things. First, communication between humans, for example an algorithm description in a textbook. Second, software development.

To be good at the first thing, Jasper must allow a writer to clearly communicate exactly what they are trying to say (expressivity), and must minimize the labor required for the reader to understand it (readability). To be good at the second thing, Jasper must minimize the amount of labor necessary to design, develop, and maintain correct software.

exploration of brain-like computational architectures

Neurons have an individual refractory period between spikes on the order of milliseconds, yet the brain can respond to a stimulus on the order of a hundred milliseconds. This suggests that the number of serial computation steps involved in recognizing a stimulus, deciding what to do, and orchestrating the motor response are on the order of hundreds of steps (Jerome A. Feldman, http://dl.acm.org/citation.cfm?id=42378 ?). This is many fewer serial steps than a typical computer must execute to accomplish a similar task. But the brain has billions of neurons running in paradigm, whereas a typical computer only has a handful of CPUs. This suggests that in order to explore or simulate brain-like computation, a very different programming paradigm is needed, one which relies on massive concurrency in place of long serial executions.

Another seeming property of the brain's computation architecture is that its 'CPUs' are mixed in with its memory. One consequence is NUMA

Yet another is that the brain is robust to errors (both incorrect or unavilable input, and also internal errors) and never 'crashes'. These sort of ideas are related to 'connectionism', a programme to model cognitive processes with interconnected networks of simple units. It is unclear exactly how the brain actually works, but in order to gain intuition in this area, one idea is to try to create programming languages that seem like they may be suited to this sort of thing. Jasper is one such attempt.

Possibly the intuitions programmers develop while using such a language will also be of use for writing practical concurrent software on today's computers, a problem which is attracting the attention of programming language designers due to the the inability of Moore's Law to continue to exponentially increase the serial speed of CPUs, while still permitting a rapid increase in the number of CPUs running in parallel.

Another approach to modeling cognitive processess is symbolist A.I., with methods such as the representation of knowledge as a graph of 'nodes' representing various concepts. This is less alien to traditional programming practices than massive concurrency, and hence less in need of language support, yet Jasper also intends to serve this application.

Conventional embedded systems with just one or a few processors is not a target application of Jasper, however, if there were a machine that you could buy for around $100 with around 64K very slow, cheap, error-prone processors each with somewhere between 128 bytes to 32K of memory, i hope that Jasper would be an okay language with which to program that machine. In fact, Jasper seeks to be a general-purpose glue language in another sense; Jasper seeks to be an okay language for programs that run on systems with a MIXTURE of: traditional systems (one or a few high-speed CPUs with a large shared memory); GPUs (massively SIMD); massively parallel MIMD systems (many slow CPUs each with a little bit of attached memory).

glue language

There are so many libraries already available in other languages, and so many projects already started in other languages, that Jasper is expected to often be used to write new code to extend an existing project in another language, or which uses libraries written in other languages.

To this end, Jasper is very concerned with interoperability. Jasper supports calling to and from, embedding and being embedded, and compiling to and from a handful of popular platforms and languages, with attention paid to preserving idiomaticity, even in round-trip compilations. Jasper supports bit- and byte-level manipulation of memory and has facilities for passing along debugging information such as source code line number. Jasper programs can easily run other Jasper code in a secure sandbox (although i am not very confident that this is actually very secure yet, as it hasn't yet been analyzed by many people), and can easily be stepped through one 'operation' at a time.

In addition, the very structure of Jasper, such as the flexible abstractions of views and boundaries, the ease of attaching 'hidden' meta-data to data using labels and views, the representation-agnostic nature of the type system is designed to support the convenient adaptation of foreign data structures and interfaces; other parts of Jasper, such as the focus on graphs, and the powerful meta-programming facilities, support compilation and embedding from or to foreign languages.

This affects the design of Jasper, in that we want to provide facilities that make it easy to 'map' Jasper programming constructs to code in other languages, and vice versa.

exploratory numerical computing

Jasper can be used to try out the application of various algorithms, transformation, and visualizations to a data set. It can be run interactively or in notebook mode, linear algebra and array operations can be expressed concisely, and there are libraries supporting common numerical and array operations as well as some plotting.

command-line one-liners

Jasper is a convenient language for writing one-liners from the commandline, and also for writing utility scripts. The Jasper interpreter starts up quickly, does not require a lot of imports or introductory boilerplate at the beginning of each program, and can be passed flags for convenient pre- and post-processing steps.

client-side application programs (web, desktop, mobile)

Jasper is suitable for writing programs which respond interactively to the user via a variety of UI paradigms. The implementation conveniently runs on various desktop, server, and mobile platforms, and the data model, binding facilities, event system, support for reactive functional programming, component architecture system, object oriented and metaprogramming facilities attenuate the boilerplate that traditionally clutters the connection between programs and UI frameworks.

live coding

Applications such as simulations, performances, servers, and learning to program often benefit from the ability to change the program code while the program is still running (see live coding). Jasper has this; in addition, to make this work well, Jasper tries to make it easy to build robust applications.

learning to program

Jasper aspires to be a good first programming language for someone who has never programmed before. The language is not too complex nor too large, there is not a lot of boilerplate to get started, the basic operation of the language is easy to simulate in one's head, it is easy to integrate with GUIs, it runs on cheap client-side platforms that kids are likely to own, and advanced constructs can be ignored until they are wanted.

However, Jasper is emphatically a TEXTUAL language, in that the primary representation of program source code is a plaintext file that is intended to be edited with a text editor, in contrast to some other 'learn to program' languages which intend for the programmer to edit their source code via a visualization within a GUI.

writing new programming languages

As a 'programmable programming language', Jasper should be a great vehicle for prototyping and perhaps even writing new programming languages.

Many of the same facilities that make Jasper interoperable (graph-based data structures, ease of attaching metadata to data, metaprogramming) also make it a good target for prototyping new programming languages.

In addition, its emphasis on being a good 'glue language', able to use other languages' libraries, and to serve as a bridge between libraries in two different other languages, should make it possible for new languages built with Jasper to have immediate compatibility with a large set of libraries from popular languages, easing adoption.

other properties

todo

team programming

sandboxing

modular

todo http://en.wikipedia.org/wiki/Scala_%28programming_language%29 has a nice list of individual features, most of which we have. also, add these to PL book constructs list.

Features that achieve the above goals

Readability

Features that make Jasper read

Let us discuss some ways in which the goal of readability is achieved.

When syntax is 'scannable', your eye can quickly scan through code looking for where a certain thing occurs.

Declarative code is easier to understand than code which mixes the specificiation of what is true with how it is to be achieved. For example, code which explicitly states invariants in the definition of data types is easier to read than code which provides only the procedure for transforming data and leaves the programmer to infer invariants. todo

Mechanistic concepts are those which consist of a set of parts, each of which has simple behavior and few connections, which can be imagined as interacting in linear chains of causation over time as a program runs. Although theorems are easier to analyze than procedural code with many hidden special cases, mechanistic concepts can be easier to understand than theorems. One reason is that theorems are atemporal in the sense that a premise about later events might imply a conclusion about earlier ones. For example, unification-style Hindley–Milner? type inference can be hard to reason about when code which will execute later contains type constraints that affects the typing of code that will execute earlier. todo

todo

todo ez comments

todo footnotes

(from concise)

One data structure

simple, programmable programming language

High-level, flexible

(from flexibility)

(from modularity)

Safety and error handling

Libraries and Communal Gravity

Massive concurrency

good tooling, implementation, ecosystem, interop

todo: cp everything above '== Syntax ==', above, into this table

Data

Jasper is built upon a single powerful data structure, the Jasper network (net).

A single powerful data structure

Lisp is a homeoiconic language built upon lists. Jasper is a homeoiconic language build upon networks (labeled directed graphs).

Type attributes, not different data types, for efficiency

Semantically, a linked list, a doubly-linked list, an unboxed list (vector), and a hash table with natural number keys are all similar; they are all lists of things indexed by natural numbers, all of which support sequential access, random access, appending, insertion, and deletion. The differences are the space and time complexity of these various operations. In Jasper, these things all look the same; if you want to choose one or the other for efficiency reasons, you use type attributes (a kind of annotation) to specify.

Uniform treatment of array lookup, function application, and structure lookup

In Jasper, the following two operations look the same: (a) looking up the third element of the array in variable "x", and (b) applying the function in the variable "x" to the number 3 and getting the result. This allows library functions to present a simple interface that can be thought of in terms of arrays, while allowing the caller to use arbitrary objects in place of those arrays.

Furthermore, fields on structures are treated the same way. This makes it easy to replace a structure that formerly used simple fields with one that uses intelligent getters and setters under the hood (this supports the so-called http://en.wikipedia.org/wiki/Uniform_access_principle ).

Bias towards immutability

Data operations in Jasper are, for the most part, expressed as functions on immutable values, rather than as changes to mutable data. Those that are not are clearly demarcated.

However, there is an exception. We freely allow 'local mutation', that is, mutation that takes the form of assigning a value to a symbol within a scope and then later reassigning a different value to that same symbol within the same scope. This kind of mutation could in theory be eliminated by renaming every mention of the variable after the reassignment (and doing some copying in the case of multiple branches, e.g. if the reassignment occurred in only one branch of an 'if' statement). We feel that this is useful to avoid having to create throwaway names for intermediate variables.

We feel that local mutation isn't bad; the bad (confusing) stuff is when you have aliasing of references, that is, when a change to the value of one variable 'magically' causes a change to a different variable (especially when the two variables are located in two different scopes, neither of which is an ancestor of the other).

Uniform treatment of metadata

Jasper has a system to allow various frameworks to attach their metadata to values.

Powerful graph abstractions

The Jasper nets are labeled, directed, multi, hyper, reified graphs. Jasper networks are:

Nodes in Jasper nets can be arbitrary functions.

Each node in a net can used as a simple array or lookup table. However, Jasper supports more powerful idioms that regard the network in its totality, rather than each node individually. For example:

Nets can be used to represent regular multidimensional labeled arrays, similar to R dataframes.

Boundaries

A net boundary is a border that cuts through a set of edges in the graph.

Boundaries in code are used to represent atomic operations, exception catching domains, etc.

Patterns

Net patterns are a matching language that generalize regular expressions and database queries to nets. Net patterns are first-class objects.

Views

A view is a different set of labels and edges associated with the same nodes. Views allow the same data to be presented in different ways to different functions operating upon it. For example, a dictionary could be presented as a list of key, value pairs, and mutations on this list could alter the dictionary.

A mutation to data in one view can affect the same data in other views. When one view is a projection of another, syntax is provided to specify the 'lineage' of data so that metadata attached to a node N1 in another view remains attached to the correct node(s) if N1 is mutated by way of replacing it with a new node.

Functions are first-class objects

Since network nodes can be functions as well as static lookup tables, they can be passed around as data.

.set protocol

.apply protocol

Typed streams/pipes/message queues

todo

Control flow

Laziness

Jasper has a lazy evaluation strategy; values aren't evaluated until they are needed. This allows for separation between definition of data structures in the abstract, and the control flow used to traverse them. For example, if you are writing a chess-playing program, you can represent the entire tree of possible chess board states in a data structure, and worry about how and how far you traverse this unrealized data structure elsewhere.

A useful special case is infinite data structures; for example, you can have an array which contains all prime numbers; unless you try to traverse this array arbitrarily far, this does not cause your program to hang because thhe unaccess members of the infinite array are never computed.

Reasoning about memory

In other lazy languages (e.g. Haskell), it can be hard to reason about and memory leaks (e.g. "space leaks"), or to discover them via profiling. Jasper makes this easier.

In addition to providing operators like Haskell's seq and deepseq, which forces strictness in a top-down manner, Jasper allows you to mark data values as eager; a function fed eager inputs will eagerly evaluate them. This forces strictness in a bottom-up manner.

Jasper presents the programmer with abstractions like 'fold'. The choice between functions corresponding to Haskell's foldl, foldl', foldr, foldr' is made by the compiler based on strictness analysis of the arguments. The sort of analysis discussion on this page ( http://www.haskell.org/haskellwiki/Stack_overflow ) is done by the compiler.

Jasper automatically converts 'almost tail recursive' functions with only associative operations preventing them from being tail recursive, to tail recursive accumulating ones.

TODO

Tail call optimization

Jasper can eliminate recursive tail calls, allowing efficient deeply nested recursion.

Multiple entry, exit points, multiple stacks

Jasper allows functions to have multiple entry points (this is sort of like a object with multiple methods that is only executed once). The function can synchronize on conditions between these entry points before executing. The notation is uniform with message handling (see below).

State management

Referential transparency by default

For the most part, Jasper "long-distance" operations are referentially transparent. Those that are not are clearly demarcated, allowing the programmer to steer clear of unwanted side effects and unexpected shared state.

Compiler inference of referential transparency

You can ask the compiler whether any function is referentially transparent. This is accomplished in a similar way to Haskell's use of monads to type non-referentially transparent functions as 'tainted', but it is done implicitly without the need for the programmer to worry about those types or to manually change a bunch of type annotations if you decide to make something non-referentially transparent later.

Escape hatch for debugging

You can do non-referentially transparent operations without tainting for the purpose of debugging and profiling, in a manner similar to Haskell's unsafeIO.

Escape hatch for caching

You can do seemingly non-refentially transparent things without tainting if you promise the compiler that their behavior is actually referentially transparent.

Lexical block scoping + object scope

Variable scope is simply defined by the block of code that they are present in, except that there is also a way to refer to the "object" of which a function is a part (a "method" in other languages).

Closures and References

Closures and References are supported but any variable containing a reference or a structure containing a reference are clearly distinguished from values with sigils.

Semiglobals

Syntactic sugar for passing objects that mimic dynamically scoped variables.

Message handling

Jasper provides a powerful, uniform, concurrency-friendly mechanism for messages to propagate and be handled.

This is used for event-driven programming, exception handling, dispatch, and parallelization.

Concurrency

Jasper has a zoo of powerful constructs for correct, concurrent programming.

todo (see zoo in [1])

Error handling

Error handling for pure functions is through Maybe (option) types. Maybe types can be subclassed to give more information on the type of error when a Nothing is returned.

Error handling for mutation is thru exceptions, however Jasper provides a concise operator to convert a return value from a function that might throw an exception into one that returns a Maybe, and vice versa.

scope(fail) and scope(end)

Like D's, these allow you to place code that undoes side effects in the event of an error right after the code creating the side effect.

transactions and .__undo attribute

Jasper provides a way for side-effectful code to specify how these side-effects can be reversed, if they can be at all. Jasper can use this to allow the programmer to create transactions.

RAII/context managers

Like C++ RAII or Python 'with' statements, you can ensure that resources are freed by use of an object protocol.

If the caller uses a 'with' (todo find a different name), this occurs as the stack is unwound, otherwise it occurs when the object is garbage collected (unless there is a reference cycle of objects which reference each other in their cleanup routine, in which case a runtime warning is generated).

(todo: if the protocol mixes stuff like Python __del__ and Python context mananger __exit__, then what to do if an object in a 'with' goes out of scope but participates in a reference cycle of other objects with finalizers?)

Modularization

Component programming system

Jasper has a powerful module system and a built-in runtime component system with facilities for directories of components, dependencies and capabilities, configuration, parameterization of types, declaration and binding to extension points, typed messaging, and event handling.

Inheritance and composition

Jasper allows you to flexibly inherit from or to compose new code with old.

Encapsulation

Jasper allows you to encapsulate data structure definitions with operations to act on those structures.

Type system

In Jasper, the philosophy is that the type system is your servant, not your master. We see static typing as a way to allow the compiler to prove theorems about your program. These theorems are useful for (a) autocompletion in IDEs, (b) preventing bugs, and (c) optimizing execution.

Optional static typing

Because there is a cost (the programmer must do some extra work to help the compiler prove the theorems), static typing is optional; by default, all variables are dynamically typed, and you can enable static typing for everything or for any subset of variables that you choose.

Type inference

To save you time writing type annotations, Jasper can infer some of them.

Powerful yet comprehensible

The type system is powerful and extensible. We support generics. We support type attributes such as 'immutable' and 'unique' that constrain, not the structure of data, but rather how it can be accessed. We allow the user to define custom attributes of this sort themselves. The type system is extensible enough that you can use it to ensure the correctness of all sorts of things, using techniques such as loop invariants and programming-by-contract.

However, the type system's power is limited in order to make code easier to understand.

One important principal is that, when reading code, complex reasoning about static type inference should not be necessary to figure out what the code does.

For example, in many ways Jasper's type system is less powerful than Haskell's; we limit the power of the type system to (combine type inference with polymorphism in order to do compile-type computations that alter or determine the semantics of code), e.g. stuff like http://stackoverflow.com/questions/3467279/how-to-create-a-polyvariadic-haskell-function .

Although the type system allows you to express complex statically-checked constraints on semantics, for the most part this power can only be used to doublecheck the semantics that are explicitly expressed in code, rather than to resolve semantics that the rest of the code leaves ambiguous.

Everything is an interface

Although Jasper has inheritance, the type system will never require you to inherit from anything or to use a certain implementation of a data type; everything is an interface.

Interfaces/typeclasses/attributes/structural typing, not principal types

In Jasper, you don't think that a value IS a type, for example "3 is an Int". Rather, you think that value X has a type attribute, for example, "3 has the property of conforming to the Int interface". "3" might have other properties as well, for example, it might conform to the Num interface, and it might conform to the Serializable interface. The Serializable type attribute is no more fundamental to 3 than the Int type attribute (at least as far as the type system is concerned).

You don't have to inherit from another implementation of something in order to make a replacement for it; you just have to implement the appropriate interface.

If you want to make something to replace a duck, you don't have to convince the compiler that it IS a duck; you just have to make something that walks like a duck and quacks like a duck.

Intersection types

Uniform notation for typing, net patterns, constraint satisfaction, and logic programming

Uniform notation for operations on types and operations on values

For example, you can look at the type List of homogeneous lists to be a function that takes a type value (the type of the elements in the list) and returns a type value (the type of a list of those elements). The notation for this sort of thing is the same as for ordinary functions.

Assumptions

The type checker is your servant, not your master. If you want some amount of typechecking or static assertion verification on a variable or module, but there is some piece of code for which you don't want to bother right now to write a proof that it does what it should, you can instruct the type checker to simply assume the result that you want.

Embedded proof checker

Jasper provides a Coq-ish decidable theorem prover with customizable tactics to allow you to express and prove difficult assertions about your code.

I am NOT suggesting that you will have to write a proof in order to get your program to type check; the proof checker is provided so that you can (a) opt to encode complex invariants that the compiler verifies, with your help; (b) extend and customize the type system.

Generics

Union types

Serialization

Jasper has facilities to permit convenient serialization of any type.

Syntax

Jasper has a simple, near-homoiconic syntax with an emphasis on conciseness and readability.

Jasper's lexical syntax is based on regular experssions, and its parse syntax is LL(1) and does not require a symbol table, making it easy to parse.

Simple, near-homoiconic syntax

Most of Jasper's syntax is for grouping (and quotation, which is a form of grouping).

Concise

todo

Jasper was designed to minimize the necessity for delimiters. For example, a function call that in many language would be written "f(x)" is just "f x" in Jasper.

Readable

Jasper has syntax for assignment to an element of an array, a common operation that some languages lack concise, readable syntax for.

Jasper does not permit custom operator precedence because that would require you to to look up operator definitions before you can parse code that you are reading.

Footnotes

Footnotes are a convention that allow you to present "the main part" of a function uninterrupted, and then to handle error checking, logging, variadic argument checking, parallelism annotations, other annotations, etc in the footnotes.

Keyword and optional function arguments

You can pass in arguments to a function by keyword instead of by position. Keyword arguments can be optional.

Optional significant whitespace

But not significant indentation.

In Jasper, the presence or absence of (a) space in between words, (b) newlines, (c) an empty line (a line with no non-whitespace characters) are significant, but the amount of these (indentation level, etc) is not. Any code can also be written in a "one-liner" without whitespace.

Easy partial application

If a function takes two arguments and you give it one argument, the result is a partially applied function.

Multiple, labeled, optional return values

Jasper return values mirror function arguments; they can be many of them, they can be labeled by keywords, and they can be optional.

Anonymous functions

Flexible operator application

You can create vector variables such that when you apply ordinary scalar operations to them, they are applied to each item in the vector, without having to extend the definitions of each operation to be able to handle vectors. There is a generic way of extending operators to act in ways such as this.

You can represent mathematical 'Del' in Jasper, and Del(f) = gradient of f, dot(Del, v) = divergence of v, cross(Del, v) = curl of v.

Notation for 'metavariable'

In statistics, some operations apply to values (and expressions act as if each random variable in the expression had been instantiated to a particular value), and others apply to expressions and care about the random variables in them. For example, if X is a random variable, "X + 3" is the former, but "var[X]" is the latter; if y is a 'normal' (non-random) variable, and Y is a random variable, then when simplifying, var[X + y] is treated differently from var[X + Y], because in the latter case we have to know if Y is correlated to X.

Jasper uses ?x to mark x as a 'metavariable', which means different things in different contexts; in the context of statistics, a metavariable is a random variable. Functions which act differently when given expressions with metavariables ar distinguished with a ? suffix; so we have mean? and var?.

Operations

Operator overloading

However, every operator must also have an alphanumeric function name, so you can use those if you don't like line noise.

Metaprogramming

Jasper's approach to metaprogramming is to provide a ladder of less- to more- powerful metaprogramming constructs. Programmers are encouraged to stick to the less powerful constructs when possible to promote readability, but the more powerful ones are there in case you need them.

Annotations

Source filters

Syntax rules

Delimited continuations

Jasper supports first-class delimited continuations, which means essentially a first-class abstraction of the state of part of the stack.

First-class stacks

Monads

todo

Macros

Hygenic macros are applied to source code after Jasper parsing but before anything else.

Optionally, the hygenicity can be weakened with with inner keyword/keypatterns, that allow the macro to access the actual string naming a symbol, allowing the creation of frameworks similar to Ruby on Rails, which associates symbols to database tables based on their name. Such symbols have sigils and/or capitalization to allow readers to recognize that their name is important.

Character-level macros

These allow you to write custom handlers for strings within the source code that will be exempted from Jasper parsing. For example, regexes are implemented using these. These can also be used to intermix foreign language source code with a Jasper program.

todo: list the other metaprogramming facilities

Tools and interoperability

Standardized, language-level support for advanced IDEs

The Jasper executable has an API that provides services for IDEs, for example, online parsing, online type inference, querying the type attributes of an instance of a token, querying for symbols meeting certain criteria, autocompletion, querying potential exceptions thrown by a function, and more.

Standardized, language-level support for documentation markup

Jasper has a standard notation for documentation markup.

Interoperability with popular languages and platforms

Out of the box, Jasper can call, be called by, and pass data to and from various popular languages. Jasper semantics and data structures are available from other languages via a standalone library. Jasper is available within various host platforms, such as JVM and .Net.

Low-level operations

Like C and Rust, Jasper can express low-level concepts, allowing it to interoperate with other languages.

Jasper can read and write to memory locations, and Jasper programmers can create data structures which specify their exact layout in memory. Jasper programmers can manually allocate and deallocate memory, and can perform pointer arithmetic.

Safe memory access

An illegal memory access produces a Nothing or an exception, not a segfault (unless the programmer turns off this checking for the sake of speed).

Lightweight threads

In the tradition of Erlang and Go, Jasper manages lightweight threads and prevents starvation.

Robustness

Erlang-ish interthread robustness facilities are provided.

Memory managed by default

By default, Jasper takes care of memory management, but you can opt to take control yourself if you need to.

Standardized tool for formatting convention

jasperfmt

Copy-on-write

Copy-on-write (COW) makes immutable value semantics efficient.

Self-hosting canonical implementation

Jasper is written in Jasper.

Portable

The Jasper implementation uses its own metaprogramming facilities to bootstrap itself off of a subset of the language, Jasper Core. This means that in order to port Jasper, all you have to do is to implement Jasper Core on your new platform.

Sandboxing

Jasper supports execution of untrusted code with limited capabilities. (note: while this is a design goal of Jasper, it is likely that the initial Jasper implmentations will have unintentional flaws, and so at least initially the sandboxing should not be trusted for high-value use cases).

Modular compilation toolchain

You can alter and extend Jasper by writing your own modules to be used in the compiler.

Dynamic changing of program; also, code auto-reloading

To allow you to experiment with a running program in the interpreter without stopping, recompiling, rerunning, and getting back to the state you were in, the Jasper interpreter allows you to dynamically inspect and alter the functions in memory during execution.

To allow you to edit source code files and re-run the program without manually giving the compile command, the Jasper interpreter will by default recompile changed source code dependencies when told to execute Jasper source.

Optional static binaries

A Jasper program can optionally be compiled into a static binary that includes all dependencies, making it easier to distribute single binary to production.

Package manager

Package manager that knowns about dependencies, signatures, volume of documentation, and canonicalness. The package mananger will almost never get 'stuck' in a state where it cannot install required dependencies. The package manager can get packages from the local file system, from a remote git project, or from a Jasper package website.

Governance process for canonicalization

A governance process that selects a subset of libraries as more or less canonical, yet not so canonical that they are included in the distribution.

Permissive license

Jasper is licensed under the permissive MIT/X11 license.

Brain-like paradigm

One motivation for Jasper is simply to make a useful programming language. Another motivation is to create a programming language in which it is easy to program in a paradigm which is my best guess for how the brain works. This should be useful because we know that the brain is capable of useful computation, and also in order to assist the cognitive studies students among us in gaining hands-on intuition about how the brain might compute.

Some tentative properties of this 'Jasper/brain' paradigm are:

Safety

No-crash by default

By default, Jasper does not do anything that could cause a crash (SEGFAULT), such as dereferencing arbitrary pointers, or accessing an array without bounds checking. However, the confident programmer can turn this off for speed.

Programming in the large

Access modifiers

Jasper contains a way for you to differentiate between a public interface and private implementation details, so that you will not break downstream programs when you change private implementation details.

todo rly?

Standard library

Jasper has a great standard library.

Collections

Jasper has a well-designed library of collections.

They include:

Many of these can be treated as simply graphs; their implementation is just an implementation detail.

Relations

Jasper can operate on relations.

Partial orders

Jasper can operate on partial orders.

Haskell prelude

Jasper's library has many of the functions in the Haskell prelude.

Integers and sequences

The operations of getting the length of a sequence and of generating a range of integers (or floating point or other numbers) has quite concise syntax.

Mathematical numbers

The mathematical integers, and multiple/arbitrary-precision arithmetic, are primary in Jasper; fixed-width number representations are secondary.

---

By niche / comparison to other languages

strong at lazy purely functional like Haskell, but easy to read like Python

conducive to macros like Lisp, but communal gravity like Python (and many other languages)

like Lisp, but based on labeled graphs (keyword arguments) rather than lists (positional arguments) (and more focus on readability; and slightly higher-level)

shell scripting: good for one-liners like Perl, but easy to read like Python

good for numerical computation like Python (and maybe but with less impedence mismatch between lists/numpy arrays and between method chaining and function call chaining) and Octave (but general-purpose language)

small-ish like (roots of) Lisp, Scheme, Nock, C, Lua

massively concurrent

robust concurrency like erlang

well-designed like C, Python

encourages readable code like Python

good programming language for prototyping/writing programming languages in, like Racket

like squeak, but based on plaintext source code files, and tooling that interoperates more with the outside world (e.g. conventional text-based source code editing, IDEs) (see also http://stackoverflow.com/questions/5833619/how-can-i-dump-all-the-source-code-from-a-squeak-smalltalk-image )

reference implementations and prototyping of networking protocols and components (since performance is not a goal it may not be the best language to write productions versions in, though)

performance is not a goal (although we would like to architect the language so that in theory it could be made performant later; however, without actually trying to make it performant we'll probably miss lots of stuff)

like perl, good for writing one-liners from the commandline (including syntactic sugar for regexes), but readable like python

a 'glue language': have an existing application written in Python, and want to keep developing it using Jasper? That's easy. Have an application in Haskell, want to keep developing it using Jasper, while making use of some libraries in Python and Java? That's easy.

Why not Python? Python is my favorite language but i'd like more support for: concurrency (no GIL!), immutable data structures, a single data structure instead of lists/dicts/numpy arrays, concise partial function application and libraries based on higher-order functions, metaprogrammability, optional static typing, laziness, commandline oneliners. Why not Haskell? I'd like more support for: readability, typeclasses everywhere, a simpler, optional type system, keyword arguments, simpler reasoning about memory usage, standard error handling, uniform collections libraries, mutability, conciseness, simpler I/O, simpler monad composition, less visible monads. Why not C or C++? C is too close to the metal. Why not Perl? Perl is too hard to read. Why not Java? Many of the same reasons as Python; also, it's too verbose. Why not Lisp/Scheme/Clojure? Not enough syntax (specifically: (1) I think being able to say something like v[a] = 10 is easier to read than (assoc v a 10), and M[b][c] = 10 is easier to read than (mset! M b c 10) (see http://stackoverflow.com/questions/16007062/ways-to-quickly-update-an-element-of-a-matrix-in-incanter-clojure ); (2) too many parens), and the JVM startup time is too slow for commandline oneliners.

---

Prime-priority list of features/design goals

todo not in prioritized order, not in correct major/minor/supp lists

Major features / high-level design goals

Minor features / mid-level design goals

Supplementary features / low-level design goals

Prime-priority list of application domain design goals

Why NOT Jasper? Prime-priority list of disadvantages/design anti-goals

Major disadvantages / high-level design antigoals

Minor disadvantages / high-level design antigoals

Details on things in the prime-priority lists

---

some old versions :

Simple, expressive

Simplicity means minimization of the number and difficulty of concepts and details. (todo do better here; it's really something having to do with the difficulty of thinking in the language, and of remembering the language)

simple, small

todo

layered: core, language, standard libraries

expressive

todo

relation to powerful: if by 'powerful', it is meant, 'enough features so that i can do what i want without writing much new code', then we are focused not so much on providing every desirable feature within the language, but rather upon making the language expressive enough so that it's possible to write a library for that feature that allows source code using it to read naturally.

todo mechanistic concepts are simpler than others

'expressing computation in a natural way rather than bending it into a non-intuitive form to fit it into the language's paradigm'

Readability

by readability we mean that the time necessary for an advanced Jasper programmer to understand someone else's source code is minimal.

todo

Jasper strives to have 'scannable' syntax; meaning that your eye can quickly scan through code looking for where a certain thing occurs.

todo

Also, a word about simplicity, size, and learnability. Jasper is a simple, small language which is easy to remember. However, although the language is small, it has a large standard library, so in order to become fluent in Jasper idioms and read others' code there is a bit to learn. Jasper is focused on being easy to remember, but not necessarily easy to learn; what I mean is that it may take you more than a few days to grok it, but once groked, you should be able to work on it on weekends every now and then without having to spend a lot of time getting back up to speed.

todo

relation to concise: we value conciseness but we see this as only a means to the ends of expressivity and readability. Insofar as 'concise' just means 'i want to write it the way i think of it', that's what we're calling expressiveness. Insofar as lack of boilerplate makes it easier to see what code is doing, we value reduction of boilerplate. Insofar as making programs shorter makes them easier to quickly understand, we value making them shorter. But insofar as 'concise' means 'Making programs really short by inventing non-intuitive abstract operators that must be 'unpacked' by the reader to understand the program', we disagree. We would rather have you write more code if that will help someone else understand it quicker.

(note: non-intuitive abstract operators are fine with us if their usefulness is broad enough to make it worthwhile for every advanced user of the language to learn them; but not if they are application domain specific. But we still don't want to add too many non-intuitive abstract operators to the language, because that impairs our other goal of simplicity.).

Massive concurrency

Our brain's neurons are individually slower than computer CPUs, but it makes up for this by having a lot of them. Moore's Law isn't increasing CPU clock speeds at an exponential rate anymore, but instead it is giving us the possibility of many processors working in parallel. Together, these show that it is possible to do interesting things with massive concurrency, if only we knew how to write software that could use it. Jasper is a platform for experimenting with ways of writing massively concurrent software.

todo

---

To put it another way, cryptic code becomes necessary only when a language does not provide the means to express the program in the same shape that the writer thinks of it.

--

numbers of threads in the tens of thousands or more. At this scale it should be possible to have one thread for each data element

todo

Our brain's neurons are individually slower than computer CPUs, but it makes up for this by having a lot of them. Moore's Law isn't increasing CPU clock speeds at an exponential rate anymore, but instead it is giving us the possibility of many processors working in parallel. Together, these show that it is possible to do interesting things with massive concurrency, if only we knew how to write software that could use it. Jasper is a platform for experimenting with ways of writing massively concurrent software. todo shorten this or throw this out.

---

note: in addition to this one composite structuring element, there there are still multiple atomic data types, such as integers, etc).

---

We're not going to give examples of Jasper's readability here, because (a) readable-looking examples can be contrived even for unreadable languages, and (b) Jasper may look alien at first; it becomes readable only after you learn the language.

---

high-level: opinionated, for less-leakly abstractions in which the tradeoff is performance (rather than flexibility)


synonyms that mean 'to have something as a goal': focus, concentrate, attention paid to, oriented towards, prioritize, emphasize, geared towards, feature, stress, support


should make sure to include the features in the feature list in the 2009 Steering Committee Position Statement, see http://bayleshanks.com/proj-plbook-plChLispLangs :

    exceptions
    modules
    concurrency
    unicode text
    union types
    record types
    abstract data types

and probably the listed Scheme features too:

    lexical scope
    dynamic typing
    list structure
    higher-order functions
    proper tail recursion
    garbage collection
    macros
    s-expression based lexical syntax. 

---