proj-jasper-jasperLibrariesNotes2

i'm not quite sure how to divide up the core/bundled/recommended libraries.

At the one extreme, we have the core language within Jasper. (this defines the standard AST that macros operate upon, i guess)

Then we have the minimal profile for Jasper. Eg a very small version of Jasper without many libraries, that defines the language itself. This is what you'd run on massively parallel machine where each CPU has little more than 32k (or whatever) local memory (although couldn't we put the other libraries in shared global memory here?).

Then we have the standard profile, ie everything that is imported by default on PCs.

Then we have the distribution, ie everything that is shipped/distributed. All of this is released and versioned together.

Then we have the 'blessed' libraries, ie things that are screened for quality and usefulness by the Jasper team, but not as carefully as the distributed libraries; and which are released and versioned separately from the distribution.

Then we have the 'canonicalized' library set, that is the 'canonical' libraries for doing various tasks, according to the wider community.

And then we have everything else that is being tracked by Jasper's official packaging system/CPAN analog (is this confined to open source?).

Quesions: Should all of these things be considered separately?

which set of library functions should a 'typical Jasper user' be assumed to be familiar with (i'm guessing, either the minimal profile, the standard profile, the distribution?) I'm leaning towards saying they should know the minimal profile cold, they should be familiar with every function in the standard profile, and they should be aware of the general capabilities of every library in the distribution.

if a blessed library becomes orphaned, should the core team try to adopt it? (i'm guessing, yes, but they should feel free to cut it if they don't have the manpower)

should we even have blessed library? (the motivation for this is so as to not bloat the distribution, but also possibly to provide a way for a library to become 'standard' even if an outsider wants to use their own release system for it)

what are the counterparts of these things in other languages? haskell: i feel like the 'minimal profile' is like the Haskell Prelude; what does GHC import by default (corresponding to our 'standard profile'; here's hawk's: https://github.com/ssadler/hawk/issues/35 ). is haskell.base part of the GHC distribution? actually Haskell 2010 specifies a set of libraries, so i guess these would be the std. profile: http://www.haskell.org/onlinereport/haskell2010/haskellpa2.html#x20-192000II

---

http://www.haskell.org/haskellwiki/Functor-Applicative-Monad_Proposal (called the AMP in other places). related to the idea to add foldable/traverable to the Prelude.

---

"classy-prelude has had the luxury of being able to re-think every Haskell wart"

https://hackage.haskell.org/package/classy-prelude

another effort along those lines

http://www.haskell.org/haskellwiki/The_Other_Prelude

---

http://www.haskell.org/haskellwiki/Class_system_extension_proposal

https://ghc.haskell.org/trac/ghc/wiki/InstanceTemplates

---

" a design that makes it easier for new releases to make backwards incompatible changes. One approach to this could be at the package level the way that base-compat operates. Another approach that could be useful to library authors is incorporate versioning at the module level.

Something to keep in mind though is that because the new Prelude needs to try to work with the old Prelude, there are not that many options in the design space. classy-prelude has had the luxury of being able to re-think every Haskell wart. So it was able to remove all partial functions and use Text instead of String in many places. But that process is very difficult for the actual Prelude, which is severly constrained. "

---

" Wednesday, October 01, 2014 Why Traversable/Foldable should not be in the Prelude

Summary: For GHC 7.10, Traversable and Foldable are going to be in the Prelude. I missed the original discussion, but I suspect it's a bad idea.

Types are how Haskell programmers communicate their intentions to each other. Currently, the Haskell Prelude contains:

mapM :: Monad m => (a -> m b) -> [a] -> m [b]

As of GHC 7.10, as part of something known as the Burning Bridges Proposal (ticket, discussion, I can't actually find a full proposal...), that will become:

mapM :: (Traversable t, Monad m) => (a -> m b) -> t a -> m (t b)

Surely that's a good thing? Aren't more general types always better? Isn't the Prelude an archaic beast from the time before? I'd argue functions which are highly polymorphic are hard to use, and hard to think about, especially for beginners. I'd also argue the Prelude is remarkably well designed, not perfect, but quite an impressive feat.

What makes a type signature complex?

I've been thinking recently about what makes type signatures complex, both to practitioners, and to relative beginners. My rough metric is:

    Fully concrete types are usually simple, as long as they aren't too long. The longer a type gets, the more complex it gets.
    Types with functions in them aren't too bad (order-1 types), but as you go up to order-2 types things start to get more complex.
    Fully polymorphic functions can be simpler than concrete functions, since they declare what you don't need to worry about.
    Functions with type classes are more complex, since you need to read the type signature while looking at the context, and need to know each class being used.
    Simple type classes (Eq, Show) aren't too bad, but custom type classes impose more of a burden.
    As you add more type classes, the complexity grows faster than linearly. Three type classes are not three times as complex as one, but quite a bit harder.
    Higher kinded type classes are significantly more complex than kind * type classes, e.g. Monad, Functor. The reason is that instead of having a hole you fill in, you now have a hole which itself has a hole.
    The higher-kinded type classes Monad and Functor aren't as bad as the others, since Functor is really the "simplest" higher-kinded type class, and Monad is required knowledge for IO.
    As you have more higher kinded type classes, the complexity burden grows even worse than for kind * type classes. Two is significantly more complex than one.

By that metric, the old mapM isn't too bad, but the new mapM is quite complex. It has two higher-kinded type classes, and one of them is not one of the common ones. I appreciate that making Foldable and Traversable key to Haskell will probably lead to them being more used, but now all beginners are going to have to wade through the Monad tutorial, their Foldable tutorial and their Traversable tutorial before they start programming (or just give up).

Why generality hurts

There are two main reasons why generality hurts:

Reading type signatures becomes difficult/impossible. We already have that problem with the Control.Arrow module, which (as far as most people use it), is just a pile of tuple combinators. But unlike other tuple combinators, these are ones whose type signature can't be understood. When I want to use &&& or * I just pick randomly, see if it type checks, then try again. When other people I know what to use these functions they just use an explicit lambda. No one thinks of referring to the documentation, since the documentation presents a unification problem (which most of the people I know could solve), not an intuition.

Reading code becomes difficult. Haskell is brilliant for letting you write a composable pipeline of code that takes some input, does some processing, and produces some output. But that only works if you have enough concrete pieces in each function to read each piece in isolation. As an example:

test = foo . mapM baz . bar

Using the current mapM definition I can, in a fraction of a second, know the approximate shape of what foo consumes, and what bar produces. With the new mapM I don't, and have to keep more context in my head to reason about the code.

Who it hurts

Generality of this nature tends to hurt two types of people:

Beginners are hurt because they need to know more concepts just to get going. As a beginner I read through Data.List regularly to build up weapons in my arsenal to attack larger problems. The new Data.List will be generalised, and reading it won't give the insights I enjoyed. Maybe the beginner can instantiate all Foldable things to [], but that adds a mental burden to exactly those people who can bear it least.

Practitioners, those who are paid to code for a living, will have greater problems with maintenance. This isn't an unsubstantiated guess... I have taken over a project which made extensive use of the generalised traverse and sequence functions. Yes, the code was concise, but it was read-only, and even then, required me to "trust" that the compiler and libraries snapped together properly.

Who it benefits

The benefit probably comes from those who are already using the Applicative/Traversable classes regularly. For these people, they can probably avoid an import Prelude(). I am not against ever changing the Prelude, but I do think that for changes of this magnitude the ideas should probably be prototyped as a separate package, widely accepted, and only then should significant surgery be attempted on the Prelude. The classy-prelude work has gone in that direction, and I wish them luck, but the significant changes they've already iterated through suggest the design space is quite large.

Concluding remarks

I realise that I got to this discussion late, perhaps too late to expect my viewpoint to count. But I'd like to leave by reproducing Henning Thielemann's email on the subject:

        David Luposchainsky wrote:
        +1. I think the Prelude should be a general module of the most commonly
        needed functions, which (generalized) folds and traversals are certainly
        part of; right now it feels more like a beginner module at times.
    It is certainly a kind of beginner module, but that's good. Experts know
    how to import. Putting the most general functions into Prelude does not
    work because:
    1. There are often multiple sensible generalizations of a Prelude
    function.
    2. You have to add more type annotations since types cannot be infered
    from the functions.
    There is simply no need to change Prelude and all packages that rely on
    specific types. Just don't be lazy and import the stuff you need!
    I should change my vote to:
    -10"

-- http://neilmitchell.blogspot.co.uk/2014/10/why-traversablefoldable-should-not-be.html

--

[–]yitz 6 points 1 day ago

    proposal made by Simon Marlow a year and a half ago that if you import Prelude.Foo then NoImplicitPrelude would get set automatically. This would make alternate preludes easier for folks to push.

That is a really nice idea.

[–]WilliamDhalgren? 3 points 1 day ago*

right.

If I'm reading that design correctly, the leaf class with InstanceTemplates? still needs to be coded for the true hierarchy above it, with a "default instance <classname>" for each superclass template it inherits. The example given has Monad decl still conscious of the default Functor instance in Applicative.

and still gets warnings for any generated instances unless doing a "deriving <classnames>" for all actual classes on the final datatype.

IMHO not as scalable as McBride?'s proposals, where final instances apparently freely mix declaration from all intrinsicly-declared superclasses.

There you only get warnings if pre-empting with explicitly created instances, allowable for a transitional period with a PRE-EMPT pragma, or an error otherwise, without excluding these explicitly from being generated.

    permalink
    save
    parent
    report
    give gold
    reply

[–]edwardkmett 6 points 1 day ago

When I last spoke with Richard we'd talked about including such a component in the proposal. I'm unsure if its absence is an act of omission or commission.

The ability to split a class is particularly dear to me, if we ever want to have the ability to refine our class hierarchies without doing so on the back of every user.

    permalink
    save
    parent
    report
    give gold
    reply

---

http://www.reddit.com/r/haskell/comments/2if0fu/on_concerns_about_haskells_prelude_favoring/

---

"I think a more promising solution to the problem of generic type complexity is making specialization of type signatures easier in code, documentation, and compiler error messages."

---

haskell prelude highlights:

---

"libraries in haskell that stood out as being unique":

http://www.reddit.com/r/haskell/comments/1k3fq7/what_are_some_killer_libraries_and_frameworks/

summary:

the main ones:

runner-ups:

others:

---

confusing error in Python numpy: if you use an array where a scalar is expected, you get:

"TypeError?: only length-1 arrays can be converted to Python scalars"

of course, you weren't trying to convert anything

---

max and nanmax

---

need a find_first!

http://numpy-discussion.10968.n7.nabble.com/Implementing-a-quot-find-first-quot-style-function-td33085.html

---

numpy 'take' (axis-wise array indexing by array)

---

http://www.yukinishijima.net/2014/10/21/did-you-mean-experience-in-ruby.html

---