proj-oot-whyOotDesignIdeas

stuff moved from whyOot

---

An important kind of readability is 'skimmability'. Skimming can mean the ability to quickly, incrementally gaining a partial understanding of code without fully understanding the definitions of the terms that occur in that code; to glance through some code and figure out most of the general structure without understanding the particulars; to guess what 'the gist' of some code is without thinking it through; or to traverse a code path or chain to some desired end without having to understand exactly what the other things along the way mean.

As a means to the end of readability, we pursue various other design criteria. Readability implies that code does what you think it will do, so Oot must minimize 'gotchas' (this criteria is also known as the principal of least surprise or least astonishment). This is related to regularity, because special cases are easily forgotten (causing gotchas); and also to language size and simplicity, because in order to understand source code you must recall and understand the language features it uses, and it's hard to remember a large number of of rarely used features, or to understand complex ones. Concise code is often more readable, in that a small program is easier to hold in one's head (and hence to quickly understand). But skimmability can be opposed to conciseness; if you want to understand the gist of something without reading the definition of every terms it contains, then there must be some redundancy to tell you something about what kind of thing the unknown terms are or what role they serve, information that could already be had if you just read every term's definition.

---

The goal of 'handling implementation details for you' can add complexity. Under what circumstances is it worth adding this sort of thing to the language, rather than leaving it to be built in a framework?

Oot's approach to this tradeoff is:

---

It is inevitable that new programming language constructs will be discovered, and also that there will be application domains for which Oot is almost enough, except for the omission of a few constructs. todo

A language built upon a small core language is also more portable, as only the core really needs to be implemented on the underlying platform, although further optimization can be had by providing platform-optimized versions of other elements of the language.

What's all this you hear from those Lisp guys about a style of programming where first you augment the programming language to suit your domain, then you write a program in your new language? We'd like to do that too.

And what's all this about a small language, a gem-like thing of perfect beauty, in which can be written a (meta-circular) interpreter for itself in only a few pages? Around which a larger, more practical language can be built? That's pretty cool, also.

As a bonus, the effort to find a set of primitives that are good for building a larger language helps us find powerful primitives, and a self-hosting language built on a small core is easier to port. Furthermore, if the language is small enough, and the abstractions are powerful enough to allow it to be implemented concisely, then the language implementation will be readable by ordinary mortals (sans a million optimizations, but we can keep them in a separate module).

todo list meta-programming features

However, we are aware that use of meta-programming features can make source code hard for others to read. Oot provides an (approximate) ladder of meta-programming mechanisms, and encourages the use of the less powerful mechanisms when possible (temperance will also allow us to optimize your code more). The most powerful metaprogramming mechanisms are only for use when all else fails.

todo self-hosting

Oot itself is made of three parts; Oot Core; a language written in Oot Core using metaprogramming; and a large standard library of language-like constructs.

Oot strives to be small, but not minimalistic. For example, 'nand' gates are universal for Boolean functions, but instead Oot bases its Boolean functions on 'and', 'or', and 'not'. Furthermore, we do not try to be minimalistic in the sense that we aren't trying to provide an orthogonal basis set of core primitives. Rather, we strive to be 'multi-paradigm'; when there are multiple paradigms, each with their own characteristic primitives (and when each set of primitives cannot be expressed in terms of another paradigm's primitives except in a clunky or non-local fashion), we try to provide some close to the union of these primitives (or at least all of the 'popular' ones), so that the language can express any of these paradigms.

Used well, metaprogramming can make programs more readable, by making the language more expressive in exactly the ways that are needed for a particular program; the time cost of understanding the black magic is repaid in making it quicker to read the rest of the program. However, a downside of metaprogramming is that it can also be (ab)used to make code that is concise but cryptic.

---

safe (no segfaults, unless of course you call out to an extension/library written in another language)

---

Laziness

Oot has a lazy evaluation strategy; values aren't evaluated until they are needed. This allows for separation between definition of data structures in the abstract, and the control flow used to traverse them. For example, if you are writing a chess-playing program, you can represent the entire tree of possible chess board states in a data structure, and worry about how and how far you traverse this unrealized data structure elsewhere.

A useful special case is infinite data structures; for example, you can have an array which contains all prime numbers; unless you try to traverse this array arbitrarily far, this does not cause your program to hang because thhe unaccessed members of the infinite array are never computed.

Lazy evaluation isn't always efficient, and the programmer can request strictness where desired.

(todo: actually, we're lenient; non-strict, but laziness isn't guaranteed either)

type system

todo

attributes

higher-order polymorphic

supports both parametric polymorphism (generics) and ad-hoc polymorphism

Operator overloading and polymorphism is useful for to allow the names of library functions which perform analogous operations on different types of data to be made more succinct and memorable by allowing the same function name to indicate different (but, by convention, closely related) functions depending on the type of the arguments. For example, '+' can refer to both integer addition and floating point addition depending on the types of its arguments; this is easier to remember than having a separate 'iadd' and 'fadd', and also more succinct (since the need to distingush between fewer function names means that the length of these names can be shorter).

Gradual typing

success typing

can use assertions to override/escape type system

"everything is an interface"

memory management

todo

Safety and error handling

Oot's approach to safety is to provide strong but optional safety constructs, and to provide an API to allow IDEs to determine which parts of code are safe. Two examples of this are its approach to static vs. dynamic typing, and to checked vs. unchecked exceptions.

Oot has gradual typing, meaning that you can write code without type annotations (and Oot will use dynamic typing to execute this code), or you can annotate some or all of your variables (allowing the compiler to typecheck your code, and also to achieve better performance by omitting dynamic typechecks). Similarly, by default Oot doesn't force you to handle or declare every type of exception that could be thrown by your code (that is, we don't have checked exceptions). Instead, Oot provides an API that allows IDEs to query Oot and ask which unhandled exceptions could be thrown by a given piece of code, and which variables are dynamically typed.

For larger projects which need more safety, Oot offers a 'strict' mode which requires all variables to be statically typed, and all exceptions to be handled.

todo

Libraries

Much of the practical utility of a language comes from its libraries. We value great libraries, and both the language and the community process are designed to encourage this.

---

A language is most useful when it is used by many people, so Oot's source code is made available under a permissive open-source license.

---

expression of computation

What is a programming language for? Oot is for two things. First, communication between humans, for example an algorithm description in a textbook. Second, software development.

To be good at the first thing, Oot must allow a writer to clearly communicate exactly what they are trying to say (expressivity), and must minimize the labor required for the reader to understand it (readability). To be good at the second thing, Oot must minimize the amount of labor necessary to design, develop, and maintain correct software.

---

exploration of brain-like computational architectures

Neurons have an individual refractory period between spikes on the order of milliseconds, yet the brain can respond to a stimulus on the order of a hundred milliseconds. This suggests that the number of serial computation steps involved in recognizing a stimulus, deciding what to do, and orchestrating the motor response are on the order of hundreds of steps (Jerome A. Feldman, http://dl.acm.org/citation.cfm?id=42378 ?). This is many fewer serial steps than a typical computer must execute to accomplish a similar task. But the brain has billions of neurons running in paradigm, whereas a typical computer only has a handful of CPUs. This suggests that in order to explore or simulate brain-like computation, a very different programming paradigm is needed, one which relies on massive concurrency in place of long serial executions.

Another seeming property of the brain's computation architecture is that its 'CPUs' are mixed in with its memory. One consequence is NUMA

Yet another is that the brain is robust to errors (both incorrect or unavilable input, and also internal errors) and never 'crashes'. These sort of ideas are related to 'connectionism', a programme to model cognitive processes with interconnected networks of simple units. It is unclear exactly how the brain actually works, but in order to gain intuition in this area, one idea is to try to create programming languages that seem like they may be suited to this sort of thing. Oot is one such attempt.

Possibly the intuitions programmers develop while using such a language will also be of use for writing practical concurrent software on today's computers, a problem which is attracting the attention of programming language designers due to the the inability of Moore's Law to continue to exponentially increase the serial speed of CPUs, while still permitting a rapid increase in the number of CPUs running in parallel.

Another approach to modeling cognitive processess is symbolist A.I., with methods such as the representation of knowledge as a graph of 'nodes' representing various concepts. This is less alien to traditional programming practices than massive concurrency, and hence less in need of language support, yet Oot also intends to serve this application.

Conventional embedded systems with just one or a few processors is not a target application of Oot, however, if there were a machine that you could buy for around $100 with around 64K very slow, cheap, error-prone processors each with somewhere between 128 bytes to 32K of memory, i hope that Oot would be an okay language with which to program that machine. In fact, Oot seeks to be a general-purpose glue language in another sense; Oot seeks to be an okay language for programs that run on systems with a MIXTURE of: traditional systems (one or a few high-speed CPUs with a large shared memory); GPUs (massively SIMD); massively parallel MIMD systems (many slow CPUs each with a little bit of attached memory).

---

prototyping

Both client-side application programs (web, desktop, mobile) and server-side backends.

Oot is suitable for writing programs which respond interactively to the user via a variety of UI paradigms. The implementation conveniently runs on various desktop, server, and mobile platforms, and the data model, binding facilities, event system, support for reactive functional programming, component architecture system, object oriented and metaprogramming facilities attenuate the boilerplate that traditionally clutters the connection between programs and UI frameworks.

Oot is designed so that a prototype can be extended (rather than rewritten), until such time as either it needs to be rearchitected, or rewritten in a more efficient language.

---

we want Oot to be simple, but complexity in the form of very general/abstract/powerful operators is better than complexity in the form of many special cases. For example, Python's 'and' and 'or' are short-circuiting and also return the last argument they evaluate; this allows them to be used for things such as "(propertyThatXMustHave(x) and x) or default". This is complex, but we prefer this type of complexity to the complexity of having a large number of fundamental constructs.

In addition, we hate having many things in Oot Core much more than we hate having many things in Oot. That is to say, adding an additional construct to Oot via metaprogramming is not nearly as disliked as adding an addition to Oot by extending Oot Core.

---

in fact, we favor general primitive abstractions over simple ones; eg we favor 'switch' over 'if'; this fits our (subjective) criterion of 'power', because the expressivity is worth the complexity here

yet, we still try to preserve 'if's in translations to the core language and to bytecode, so that we can use platform-dependent optimizations for it. This goes with our design goal of preserving intent.

---

The five general abstract design goals listed in whyOot can be mapped to the Wu Xing five elements:

Oot prioritizes them in the order above. However, readability is emphasized because although in theory i would not greatly restrict metaprogrammability or expressiveness for a gain in readability, in practice i feel that most other contemporary programming languages don't think enough about readability, so by comparison to the mainstream, readability will appear to be a high priority in Oot.

---

i guess really we may as well list 7, in declining order of priority for oot:

simplicity, metaprogrammability, expressiveness, readability, conciseness, safety, performance

---

CGI:

https://www.devever.net/~hl/mildlydynamic

    13
    dvogel edited 26 hours ago | link | flag | 

As mentioned in the article, there were other Apache modules providing similar functionality, such as mod_python. There were also CGI approaches to the same template-forward bent, such as Mason (which was perl). If there was anyone saying “why support another since we already have PHP?” it was admins on shared hosting services. Each additional module was yet another security threat vector and a customer service training.

    5
    ewintr 13 hours ago | link | flag | 

I was at a talk given by Rasmus Lerdorf (creator of PHP) once and he claimed it was because the PHP implementation was the most basic, limited version possible and it therefore it was very simple to isolate different users from each other. This made PHP very popular with cheap shared hosters. Whereas the Perl implementation was much more thorough and hooked (not sure what the correct terms are) into the whole of Apache and therefore it needed a dedicated server. Much more expensive.

    ~
    hlandau 11 hours ago | link | flag | 

Yeah. Even though mod_php is a single module loaded into a single Apache instance, it was designed with some sandboxing options like safe_mode. Or you could use PHP CGI and isolate things even better (running as the user’s UID).

Other language hosting modules for Apache like mod_perl didn’t offer the same semantics. I also recall mod_perl being pretty oriented towards having access to the web server’s configuration file to set it up. People did use Perl before the rise of PHP, but most often via CGI (remember iKonboard?)

 hoistbypetard 5 hours ago | link | flag |

mod_perl was more oriented toward exposing the apache extension API so that you could build apache modules in perl, as I remember it. It got used to write some cool web applications (slashcode springs to mind) that’d have been hard to write (at that scale) any other way at the time. But mod_php was a very different beast, just aiming to be a quick way to get PHP to work well without the overhead of CGI.

I agree with the article… there’s nothing now (other than PHP, which I still use now for the kind of pages you mention, the same way I did in the early ‘00s) that’s nearly as low-friction as PHP was back then to just add a couple of dynamic elements to your static pages.

~ gcupc 5 hours ago

link flag

Yeah, I was at a small web hosting company in the late ’90s, early 2000s, and we used PHP CGI with our shared hosting.

pm 25 hours ago

link flag
    It’s not like you couldn’t build a similar system based on Python or Ruby or JS.

Not quite. The article touches this, although not explicitly, you have to read a bit between the lines.

PHP allowed for easy jump in and out static and dynamic context like no other alternative. It still does this better than anything else. This was in the core of the language no need to install third party libraries. It also included a MySQL? client library in its core with work out if the box. Essentially, it shipped with everything necessary in the typical setup. No need to fiddle with server set up.

The language was also arguably more approachable for beginners than perl with a multitude of simple data structures easily accessible through the infamous array() constructor. It also retained familiarity for C programmers, which were a big audience back then. While python for example, didn’t.

One thing I don’t agree with is the simplicity nor the deployment model. It’s only simple in the context of the old shared hosting reality. If you include setting up the server yourself like we do nowadays, it is actually more cumbersome than a language that just allows you to fire up a socket listening on port 80 and serve text responses.

It.s.how it was marketed and packages that made all the difference.

    9
    hobbified 23 hours ago | link | flag | 

Yes, but it was “better” in the sense of “making it easy to do things that are ultimately a lousy idea”. It’s a bit better now, but I used it back then and I remember what it was like.

Convenience feature: register_globals was on by default. No thinking about nasty arrays, your query params are just variables. Too bad it let anyone destroy the security of all but the most defensively coded apps using nothing more than the address bar.

Convenience feature: MySQL? client out of the box. Arguably the biggest contributor to MySQL’s? success. Too bad it was a clumsy direct port of the C API that made it far easier to write insecure code than secure. A halfway decent DB layer came much, much later.

Convenience feature: fopen makes URLs look just like files. Free DoS?