proj-oot-ootLibrariesNotes4

Difference between revision 13 and current revision

No diff available.

---

i guess we should at least have something like this:

https://github.com/zeit/micro

---

inglor 10 hours ago

> In order to maximize your usage of this feature, you'll want to use modules from the ecosystem that expose Promise instead of just a callback.

Bluebird which is one of the most popular promise libraries converts entire callback APIs to promises in one go:

Promise.promisifyAll(require("fs")); fs is now promisified and methods return promises.

(full disclosure, I'm a bluebird contributor)

reply

---

macros in js:

http://sweetjs.org/

---

some concurrency frameworks are suggested at the end of https://gist.github.com/staltz/868e7e9bc2a7b8c1f754

---

scala's pulsar concurrency framework

---

Without digging into it, the ((Elixir)) Calendar stuff looks like a strict improvement over Erlang's tuply handling of dates, times and timestamps, which are all tuples with three numbers. This can be confusing/problematic, and something that would benefit from typing to avoid mixing up Dates with Timestamps, say.

Can anyone who knows both confirm?

reply

antipax 1 hour ago

Yes, it absolutely is an improvement over erlang-style tuples IMO, and it also vastly improves the interoperability story between the stdlib, date/time/calendar libs, and database libs.

reply

"

Elixir Logo

    Home
    Install
    Getting Started
    Learning
    Docs
    Blog
    Packages

Elixir v1.3 released June 21, 2016 · by José Valim . in Releases

Elixir v1.3 brings many improvements to the language, the compiler and its tooling, specially Mix (Elixir’s build tool) and ExUnit? (Elixir’s test framework). The most notable additions are the new Calendar types, the new cross-reference checker in Mix, and the assertion diffing in ExUnit?. We will explore all of them and a couple more enhancements below.

With this release, we also welcome Andrea Leopardi to Elixir Core Team. He has contributed greatly to this release and maintains important packages in the community, like Gettext and Redix. Language improvements

The language has been improved semantically and includes new types and APIs. Let’s see the three major features. Deprecation of imperative assignment

Elixir will now warn if constructs like if, case and friends assign to a variable that is accessed in an outer scope. As an example, imagine a function called format that receives a message and some options and it must return a path alongside the message:

def format(message, opts) do path = if (file = opts) && (line = opts) do relative = Path.relative_to_cwd(file) message = Exception.format_file_line(relative, line) <> " " <> message relative end

  {path, message}end

The if block above is implicitly changing the value in message. Now imagine we want to move the if block to its own function to clean up the implementation:

def format(message, opts) do path = with_file_and_line(message, opts) {path, message} end

defp with_file_and_line(message, opts) do if (file = opts) && (line = opts) do relative = Path.relative_to_cwd(file) message = Exception.format_file_line(relative, line) <> " " <> message relative end end

The refactored version is broken because the if block was actually returning two values, the relative path and the new message. Elixir v1.3 will warn on such cases, forcing both variables to be explicitly returned from if, case and other constructs. Furthermore, this change gives us the opportunity to unify the language scoping rules in future releases. Calendar types and sigils

Elixir v1.3 introduces the Calendar module as well as 4 new calendar types:

    Date - used to store dates (year, month, day) in a given calendar
    Time - used to store time (hour, minute, second, microseconds)
    NaiveDateTime - used to store datetimes without a timezone (year, month, day, hour, minute, second, microseconds) in a given calendar. It is called naïve because without a timezone, the datetime may not actually exist. For example, when there are daylight savings changes, a whole hour may not exist (when the clock moves forward) or a particular instant may happen twice (when the clock moves backwards)
    DateTime - used to store datetimes with timezone (year, month, day, hour, minute, second, microsecond and time zone, with abbreviation, UTC and standard offset)

The current Calendar modules and its types is to provide a base for interoperatibility in the ecosystem instead of full-featured datetime API. This release includes basic functionality for building new types and converting them from and back strings.

Elixir v1.3 also introduces 3 new sigils related to the types above:

    ~D[2016-05-29] - builds a new date
    ~T[08:00:00] and ~T[08:00:00.285] - builds a new time (with different precisions)
    ~N[2016-05-29 08:00:00] - builds a naive da"

---

http://stackoverflow.com/questions/15995/useful-code-which-uses-reduce-in-python/282206#282206

---

i hear great things about the Scala collections library

also the operations in the LINQ section of Self:proj-plbook-plChData look like a good core set of collection operations, with special attention to those that they deigned to give C# LINQ syntax to.

Note that that core set is pretty much just a slightly expanded version of Python list (generator, actually) comprehensions.

---

numerical computing

you often hear about the popular fortran libraries BLAS and LAPACK.

LAPACK is a layer above BLAS (it calls BLAS). There are various implementations ("typically ATLAS + LAPACK, or MKL, or ACML" [1]). ATLAS contains BLAS and a subset [2] of LAPACK.

So maybe first provide the subset of LAPACK that ATLAS provides?

as of this writing, these are (from [3]):

" [S,D,C,Z]GESV [S,D,C,Z]GETRF [S,D,C,Z]GETRS [S,D,C,Z]GETRI [S,D,C,Z]TRTRI [S,D,C,Z]POSV [S,D,C,Z]POTRF [S,D,C,Z]POTRS [S,D,C,Z]POTRI [S,D,C,Z]LAUUM "

S,D,C,Z prefixes indicate single precision real, double precision real, single precision complex, double precision complex.

Here's what they do:

Solve AX = B:

Factorization:

Matrix determinant and inverse:

Products of special matrixes:

seems to me like we can start by just providing:

then:

then looking at BLAS, then LAPACK (the union of http://www.netlib.org/lapack/lug/node25.html and http://www.netlib.org/lapack/lug/node37.html appear to list the things that LAPACK can do, but the latter list is more numerics-y so avoid it for now), then see if MKL and ACML provide anything extra

then numpy

then scipy

---

yoavm 23 hours ago

D3 has the reputation of being super-complicated because of all the libraries that are based on it, "simplifying" it so that everyone can use it. In the past year I wanted to create pretty unique type of data visualisation, so I dived into D3 and discovered it a makes a lot more sense than I though. Of course, if you only want a regular bar chart, you'll do better with things like C3, nvd3 etc'. But if you want anything a bit special, D3 itself is very powerful and the documentation in pretty good - there's no reason to avoid using it directly.

Definitely looking forward to try the new release.

reply

minimaxir 22 hours ago

To add to that, if you are a complete newbie to any data visualization, do not start with d3. If you want to make pretty charts programatically, using R/ggplot2 or Python/Seaborn is good enough. Even Excel is fine if you tweak the defaults.

D3 is good if your visualization benefits from interactivity, either with dynamic data adjustment or rich tooltips. But static visualizations are important too. (I recently restructured my workflow so I can output static images and interactive charts with the same code, which makes it the best of both worlds.)

reply

danso 21 hours ago

What is your static+interactive workflow now, if I can ask? Also, is it fairly easy to build a workflow that generates static visualizations via D3 (i.e. making savable SVGs)?

reply

minimaxir 21 hours ago

I make charts with R/ggplot2. Standard workflow is to construct chart and save as static file. (PNG/SVG etc.) But with the plot.ly bridge, I can convert to an interactive chart w/ rich tooltips with very very little tweaking. (See this notebook: https://github.com/minimaxir/interactive-facebook-reactions/...)

To convert D3 to SVG, I believe all you need is a helper function but that is not my area of expertise.

reply

monfera 19 hours ago

There's also Shiny, Jupyter, Bokeh...

reply

minimaxir 19 hours ago

Those aren't updated clientside (have to ask the server for the new chart instead of just the data), which is a different architecture that does not scale.

reply

---

https://github.com/cht8687/You-Dont-Need-Lodash-Underscore

(Lists of JavaScript? methods which you can use natively )

"

frostik 8 hours ago

There's a huge gotcha: _.map, _.forech, ... can iterate over objects (often used as associative arrays) - Native map can't.

At some point you'll need lodash and co again - at least for convinience.

reply "

" Alternate title for the article: "Lists of lodash features replaced by ES6 if you don't mind throwing an exception when given invalid input such as null or undefined".

All kidding aside, a lot of our lodash code ends up looking something like this:

    function (xs) {
        return _(xs).pluck('foo').filter().value();
    }"

"

The need for Lodash and Underscore is diminishing, but not because of JS getting native support for a subset of utilities, but because there are already better alternatives. Utility libraries like Ramda[1] or pointfree-fantasy[2] with related modules which offer a better support for FP idioms and more useful higher-order functions are what made me drop Underscore.

I think that, with a language with such tiny stdlib, there always be a need for some kind of utility belt-style library ("if only for convenience", as another commenter write, which is actually the main point of such libraries). It's just a matter of choosing the best library for your coding style and project at hand. It may be a hassle, but an alternative is an stdlib the size of Python's - and I'm not sure if it's a good direction.

[1] http://ramdajs.com/0.21.0/index.html [2] https://github.com/DrBoolean/pointfree-fantasy

reply "

---

imatix SFL (legacy but this guy liked it [7] )

http://imatix-legacy.github.io/sfl/index.htm

---

dir(__builtins__)

['ArithmeticError?', 'AssertionError?', 'AttributeError?', 'BaseException?', 'BlockingIOError?', 'BrokenPipeError?', 'BufferError?', 'BytesWarning?', 'ChildProcessError?', 'ConnectionAbortedError?', 'ConnectionError?', 'ConnectionRefusedError?', 'ConnectionResetError?', 'DeprecationWarning?', 'EOFError', 'Ellipsis', 'EnvironmentError?', 'Exception', 'False', 'FileExistsError?', 'FileNotFoundError?', 'FloatingPointError?', 'FutureWarning?', 'GeneratorExit?', 'IOError', 'ImportError?', 'ImportWarning?', 'IndentationError?', 'IndexError?', 'InterruptedError?', 'IsADirectoryError?', 'KeyError?', 'KeyboardInterrupt?', 'LookupError?', 'MemoryError?', 'NameError?', 'None', 'NotADirectoryError?', 'NotImplemented?', 'NotImplementedError?', 'OSError', 'OverflowError?', 'PendingDeprecationWarning?', 'PermissionError?', 'ProcessLookupError?', 'RecursionError?', 'ReferenceError?', 'ResourceWarning?', 'RuntimeError?', 'RuntimeWarning?', 'StopAsyncIteration?', 'StopIteration?', 'SyntaxError?', 'SyntaxWarning?', 'SystemError?', 'SystemExit?', 'TabError?', 'TimeoutError?', 'True', 'TypeError?', 'UnboundLocalError?', 'UnicodeDecodeError?', 'UnicodeEncodeError?', 'UnicodeError?', 'UnicodeTranslateError?', 'UnicodeWarning?', 'UserWarning?', 'ValueError?', 'Warning', 'ZeroDivisionError?', '__build_class__', '__debug__', '__doc__', '__import__', '__loader__', '__name__', '__package__', '__spec__', 'abs', 'all', 'any', 'ascii', 'bin', 'bool', 'bytearray', 'bytes', 'callable', 'chr', 'classmethod', 'compile', 'complex', 'copyright', 'credits', 'delattr', 'dict', 'dir', 'divmod', 'enumerate', 'eval', 'exec', 'exit', 'filter', 'float', 'format', 'frozenset', 'getattr', 'globals', 'hasattr', 'hash', 'help', 'hex', 'id', 'input', 'int', 'isinstance', 'issubclass', 'iter', 'len', 'license', 'list', 'locals', 'map', 'max', 'memoryview', 'min', 'next', 'object', 'oct', 'open', 'ord', 'pow', 'print', 'property', 'quit', 'range', 'repr', 'reversed', 'round', 'set', 'setattr', 'slice', 'sorted', 'staticmethod', 'str', 'sum', 'super', 'tuple', 'type', 'vars', 'zip']

>>> len(dir(__builtins__)) 151 >>> len(['abs', 'all', 'any', 'ascii', 'bin', 'bool', 'bytearray', 'bytes', 'callable', 'chr', 'classmethod', 'compile', 'complex', 'copyright', 'credits', 'delattr', 'dict', 'dir', 'divmod', 'enumerate', 'eval', 'exec', 'exit', 'filter', 'float', 'format', 'frozenset', 'getattr', 'globals', 'hasattr', 'hash', 'help', 'hex', 'id', 'input', 'int', 'isinstance', 'issubclass', 'iter', 'len', 'license', 'list', 'locals', 'map', 'max', 'memoryview', 'min', 'next', 'object', 'oct', 'open', 'ord', 'pow', 'print', 'property', 'quit', 'range', 'repr', 'reversed', 'round', 'set', 'setattr', 'slice', 'sorted', 'staticmethod', 'str', 'sum', 'super', 'tuple', 'type', 'vars', 'zip']) 72

---

how many functions are in the Python standard library?

i don't know a quick way to find that out.

the official tutorial has a 'brief tour of the standard library' at https://docs.python.org/2/tutorial/stdlib.html and https://docs.python.org/2/tutorial/stdlib2.html

they list sections and modules (with approximate number of contained functions in parens):

(total of numbers in parentheses: 622)

not counting the ones listed under 'Batteries Included', that's 35 modules, which are:

os, shutil, glob, getopt, argparse, sys, re, math, random, urllib2, smtplib, datetime, zlib, gzip, bz2, zipfile, tarfile, timeit, profile, pstats, doctest, unittest, repr, pprint, textwrap, locale, string, struct, threading, logging, weakref, array, collections, heapq, decimal

(numbers in parentheses were obtained using:

def count_namespace(ns, already_seen = None): if already_seen is None: already_seen = set()

  items = [x for x in dir(ns) if [types.FunctionType, types.BuiltinFunctionType, types.UnboundMethodType].__contains__(type(ns.__getattribute__(x))) and x[0] != '_']
  items = set(items).difference(already_seen)
  already_seen = already_seen.union(items)
  count = len(items)
  for subns in [ns.__getattribute__(x) for x in dir(ns) if (type(ns.__getattribute__(x)) == type) and x[0] != '_']:
    try:
      subns2 = subns()
      countIncrement, already_seen = count_namespace(subns2, already_seen=already_seen)
      count += countIncrement
    except TypeError:  # if the constructor for subns required arguments
      pass
    except ValueError:  # no such test method in <class 'unittest.case.TestCase'>: runTest
      pass
      
  return count, already_seen

already_seen = set() for n in ['os', 'shutil', 'glob', 'getopt', 'argparse', 'sys', 're', 'math', 'random', 'urllib2', 'smtplib', 'datetime', 'zlib', 'gzip', 'bz2', 'tarfile', 'timeit', 'profile', 'pstats', 'doctest', 'repr', 'pprint', 'textwrap', 'locale', 'string', 'struct', 'threading', 'logging', 'weakref', 'array', 'collections', 'heapq', 'decimal']: from importlib import import_module; m=import_module(n); count, already_seen = count_namespace(m, already_seen = already_seen); print n, count

  1. zipfile omitted b/c 'CRC' in dir(zipfile.ZipInfo?()), but zipfile.ZipInfo?().CRC yields an AttributeError?
  2. unittest omitted b/c it did something weird when i ran this

note: this is an underestimate, because we are only counting one when multiple functions in different modules have the same name; eg there is both decimal.Decimal.log10 and math.log10 but we only count one.

)

old:

def count_namespace(ns, already_seen = None): if already_seen is None: already_seen = set() print dir(ns)[0] print ns.__getattribute__(dir(ns)[0])

  items = [x for x in dir(ns) if [types.FunctionType, types.BuiltinFunctionType, types.UnboundMethodType].__contains__(type(ns.__getattribute__(x))) and x[0] != '_']
  print 'AS: ', already_seen
  items = set(items).difference(already_seen)
  already_seen = already_seen.union(items)
  print ns, items
  count = len(items)
  for subns in [ns.__getattribute__(x) for x in dir(ns) if (type(ns.__getattribute__(x)) == type) and x[0] != '_']:
    countIncrement, already_seen = count_namespace(subns, already_seen=already_seen)
    count += countIncrement
  return count, already_seen

already_seen = set() for n in ['os', 'shutil', 'glob', 'getopt', 'argparse', 'sys', 're', 'math', 'random', 'urllib2', 'smtplib', 'datetime', 'zlib', 'gzip', 'bz2', 'zipfile', 'tarfile', 'timeit', 'profile', 'pstats', 'doctest', 'unittest', 'repr', 'pprint', 'textwrap', 'locale', 'string', 'struct', 'threading', 'logging', 'weakref', 'array', 'collections', 'heapq', 'decimal']: from importlib import import_module; m=import_module(n); count, already_seen = count_namespace(m, already_seen = already_seen); print n, count

note: this i

---

https://docs.python.org/2/library/index.html#library-index lists about 25 main library sections, here they are with the number of modules in each section:

7. String Services: 11 8. Data Types: 19 9. Numeric and Mathematical Modules: 9 10. File and Directory Access: 12 11. Data Persistence: 13 12. Data Compression and Archiving: 5 13. File Formats: 6 14. Cryptographic Services: 4 15. Generic Operating System Services: 17 16. Optional Operating System Services: 9 17. Interprocess Communication and Networking: 7 18. Internet Data Handling: 16 19. Structured Markup Processing Tools: 15 20. Internet Protocols and Support: 25 21. Multimedia Services: 10 22. Internationalization: 2 23. Program Frameworks: 2 24. Graphical User Interfaces with Tk: 7 25. Development Tools: 6 26. Debugging and Profiling: 7 27. Software Packaging and Distribution: 2 28. Python Runtime Services: 16 29. Custom Python Interpreters: 2 30. Restricted Execution: 2 31. Importing Modules: 7 32. Python Language Services: 13

---

nostrademons 3 hours ago [-]

It's because languages are communication tools as well as compilers, and the stdlib is part of the common vocabulary that the rest of the language ecosystem shares.

Without a decent stdlib, you end up with the pre-STL C++ situation, where every library & framework declares its own string, vector, and hashmap classes and you need slow, verbose, and error-prone routines to convert between them. The main benefit of having these in the stdlib isn't that you don't have to re-implement them, it's that every third-party lib knows exactly which class represents the concepts of text, sequences, and dictionaries.

The same goes for many other types, eg. Promises, Paths, URIs, Dates, Input/OutputStreams?, Iterators, etc. Indeed, when one of the stdlib types is misdesigned (eg. java.util.Date), you get an utter mess as the community converges on an alternative (eg. JodaTime?) and then the alternative API is folded back into the stdlib (Java 8).

Luckily, we're learning just which types need to be in the stdlib and which can be outsourced to a third-party package manager. In particular, a lot of serialization/parsing formats are better off in a package manager, as long as the language defines an annotation system that can be used to define which fields must be saved. And giant systems like webservers, webframeworks, or RPC formats are often better off as a third-party library.

reply

jackmott 10 hours ago [-]

F# isn't much like C#, and in fact it does suffer a bit for it, as some of the back end causes compromises in the F# language. Mutable fields in records have secret baggage, functions with arithmetic operators can't be automatically genericized unless you mark them inline, etc

reply

__s 6 hours ago [-]

My biggest let down with F# is that NullReferenceException? is still a thing. Also a C# function is called as if they're typed to take tuples but then you can't create a tuple & pass it along

reply

the HLA Standard Library

http://www.plantation-productions.com/Webster/HighLevelAsm/HLADoc/HLAStdlib/1_HTML/HLAStdlib.htm (see http://www.plantation-productions.com/Webster/HighLevelAsm/HLADoc/index.html for other formats)

---

mb the YASM 'libyasm' module (YASM is a ground-up rewrite of NASM), although that seems to really be just for assembler (assembly language compiler)-related stuff:

https://github.com/yasm/yasm/wiki/Libyasm http://yasm.tortall.net/ (section Key Internal Features )

---

https://github.com/fantasyland/fantasy-land https://github.com/hemanth/functional-programming-jargon

---

https://www.haskell.org/platform/contents.html

---

tibbe 14 hours ago [-]

I wouldn't recommend following the Haskell approach ((small stdlib core with a 'platform' of recommended other libs; https://www.haskell.org/platform/ )). It hasn't worked well for us. (I took part in creating the Haskell Platform and the process used to add packages to it. I also used to maintain a few of our core libraries, like our containers packages and networking).

Small vs large standard library:

A small standard library with most functionality in independent, community-maintained packages has given us API friction as types, traits (type classes), etc are hard to coordinate across maintainers and separate package release cycles. We ended up with lots of uncomfortable conversions at API boundaries.

Here's a number of examples of problems we currently have:

Empirically, languages that have large standard libraries (e.g. Java, Python, Go) seem to do better than their competitors.

reply

---

"who doesn't have `mtl` in their applications at this point" -- Haskeller, in a comment (yes, mtl is in the Haskell Platform)

---

pcwalton 2 hours ago [-]

> But if you have a large standard library and want to break the API, you can.

We have a policy of no breakage for stable libraries post 1.0 (as does Python, and Go, etc.). So no, we can't.

The size of the standard library has nothing to do with it.

reply

((this was in reply to a comment proposing a larger stdlib, rather than a small core stdlib, so that the language could atomically make changes in various libs without breaking other libs; pcwalton is saying that regardless of if the atomic change would fit inside the stdlib, a no-breaking-changes-to-stdlib policy prevents this anyways))

---

rtpg 16 hours ago

parent flag favorite on: The Rust Platform

...I think the failure of the Haskell Platform has a lot more to do with how Haskell deals with dependencies, and the difficulties it entails, than with the "batteries included" approach itself.

((this guy is on a thread arguing about whether a small core stdlib with a larger recommended stdlib is a good idea; e is arguing that it is; the thread had previous cited Haskell Platform as a failure of this sort of thing, and e is rebutting this by saying the failure was actually with Haskell dependency management))

hyperhopper 14 hours ago

parent flag favorite on: The Rust Platform

But even then, languages that have great, thriving easy to use dependency systems and package managers with small standard libraries still run into problems

see: javascript

---

"Python codebases that are resilient don't use much of "core", arrow for time, requests for http, simplejson, etc. "

---

pjmlp 11 hours ago [-]

I disagree.

In the enterprise space it is quite common that we only get to use what it is in the computer and access to anything else is strictly controlled by IT.

So if it isn't in the standard library or some internal library mirror, we don't get to use it, as simple as that.

reply

---

" The 'std lib is where libraries go to die' was invented by Python. The libs are shallow, don't break backwards compat and provide a substandard experience. Things that continue to improve provide an out of tree alternative package name. "

---

---

One of the things I love above all about Python and Ruby are the kitchen-sink standard libraries. The node ecosystem is deeply frustrating in this respect.

reply

krylon 2 hours ago [-]

It has been a while since I did anything with Python, but I did like its standard library. It was reasonably comprehensive without feeling bloated, and the documentation was pretty good (mostly).

Having a good standard library also makes deployment easier. (In Go, OTOH, I tend to care less, even though its standard library is quite good, because thanks to static linking, deployment is always easy, no matter how many third-party libraries I use.)

reply

---

steveklabnik 11 hours ago [-]

Rust will already allow you to have multiple versions of transitive dependencies.

reply

bassislife 2 hours ago [-]

Which I guess is normal since it does not create any dependency cycle. A new version might as well be thought as a completely different package (of perhaps similar functionality).

reply

sitkack 5 hours ago [-]

This needs to be screamed from the hill tops!

reply

---

https://luarocks.org/stats/this-week

---

kibwen 12 hours ago [-]

  > Empirically, languages that have large standard 
  > libraries (e.g. Java, Python, Go) seem to do better than 
  > their competitors.

You seem to be overlooking the ultimate counterexample: C. :P

reply

brandonbloom 11 hours ago [-]

You were being down voted, maybe for perceived snark, but I think you raise an interesting point.

To me, C did have a standard library: Unix. It's a runtime system too! Due to the nature of the original C bootstrapping process it just happens to be possible to remove this standard library, and Windows was evidence of this.

There is another interesting potential counter example: Lua. It's minimalistic standard library is part of what makes it so attractive for embedding, eg. in game engines. However, Lua's embedding API is so good, you could almost say that it comes with a large standard library too: Your existing C code!

...

 pjmlp 6 hours ago [-]

> To me, C did have a standard library: Unix. It's a runtime system too!

Fully agree. We just ended up with ANSI C + POSIX, because the standard bodies refused to put everything into the same bag.

In the early days, most C compilers were anyway shipping partial UNIX APIs on top of their K&R and ANSI implementations.

reply

 ---

ufo 10 hours ago [-]

Lua's lack of a stdlib is also a curse. I can't imagine how many incompatible versions of string.trim and OOP libraries are out there in the wild right now...

Things have been getting better lately because of Luarocks but its still an uphill battle.

reply

String trim is just:

  foo:gsub("%s*$", "")

or

  foo:gsub("^%s*", "")

The standard idiom for OOP in Lua is a one-liner:

  return setmetatable(self, mt)

where mt.__index has all the methods. How you assign to mt.__index can vary across modules according to style, but that's a _purely_ asethetic issue. The mechanics are identical. Using a module to accomplish it creates a useless dependency.

There are many criticisms one could make of Lua, but I don't think those two particular criticisms are legit. They're classic bikeshedding.

reply

JoshTriplett? 6 hours ago [-]

If you're reading a pile of string processing code, seeing

    s.rstrip()

helps make code self-documenting, compared to

    s:gsub("%s*$", "")

I don't want to argue for a massive standard library (for instance, I don't think Python should have shipped modules for dbm, bdb, sqlite, or XML-RPC), but simple string processing seems like a good thing to standardize.

reply

---

comex 6 hours ago [-]

> You seem to be overlooking the ultimate counterexample: C. :P

I think one reason (of many) that C++ has replaced C almost completely for new development is the STL. Of course, the STL fundamentally depends on the language feature of templates, which you can only approximate in C, but considering that Java and Objective-C, among other languages, lasted pretty long with no generics and only non-type-safe containers, I think C could have benefitted greatly from basic things like resizable arrays, hash tables, trees, better strings, etc. in the standard library. Now it is probably too late for it to matter (which most people consider a good thing).

reply

krylon 2 hours ago [-]

FWIW, the last time I cooked up something in C, I liked Judy very much: http://judy.sourceforge.net/

It has slightly awkward but very simple API, and it's very fast.

reply

---

bitmadness 9 hours ago [-]

This is exactly the main problem with Haskell. A stunning language with a lousy standard library. In my opinion, Haskell should offer arrays and maps as built-ins (like Go) and ship with crypto, networking, and serialization in the standard library (I know serialization is already there, but everyone seems to prefer Cereal, so...)

reply

---

kibwen 12 hours ago [-]

Go isn't immune to the problem either. See the `flag` package, which is something that new users are encouraged to avoid in favor of e.g. https://github.com/jessevdk/go-flags .

reply

...

the_mitsuhiko 6 hours ago [-]

Everybody I have ever seen do CLI applications in Go will recommend heavily against it ((the flag package))

reply

---

..."Python simplejson vs json. Most working Python developers I know try simplejson first...because simplejson got much faster as it evolved outside of the standard library[0]"

---

steveklabnik 13 hours ago [-]

In the Ruby world, very few people use the standard library because it's got so many flaws, and they can't be fixed. So you end up with Nokogiri rather than REXML, all the various HTTP libs rather than net/*, etc. So it just ends up being bytes sent over the wire, wasting disk and bandwidth...

reply

anamoulous 9 hours ago [-]

I wouldn't say very few people use the standard library?

CSV, logger, json, fileutils, tempfile, pp, and on and on are used all the time...

reply

steveklabnik 9 hours ago [-]

There are some parts that are good, and some parts that are bad, for sure.

reply

---

shadowmint 9 hours ago [-]

Sql?

It does all the wrong things; singletons, no testability, cgo for implementations, side effects and you have to use every database differently based on their individual semantics.

Virtually everyone I've ever spoken to either uses a high level wrapper around the sql library or a no-sql solution.

That's the definition of 'stdlib is where packages go to die'.

It's not that the API is unusable, it's just basically not used by the community because there are other better things out there...but you're stuck with it forever, because it's there and some people do use it, and changing or removing it would be a breaking change.

Anyhow, we're just speculating. Does anyone actually collect metrics about the usage of different parts of the stdlib for any language?

Without hard data to back it up, you couldn't really make a strong argument either way.

reply

saysjonathan 1 hour ago [-]

I didn't see anyone actually mention sql so I'll just assume your first line is to be interpreted as "sql is the counterexample of why Go's standard library is not as great as it may seem."

>Virtually everyone I've ever spoken to either uses a high level wrapper around the sql library or a no-sql solution.

How does that reflect the quality of the std lib implementation? All the high-level wrappers I've seen still utilize database/sql, they just provide convenience methods on top of the existing functionality. Are people using NoSQL? databases because database/sql is so bad or merely because that technology fits their project's requirements?

>That's the definition of 'stdlib is where packages go to die'.

steveklabnik's example of Ruby XML parsing libraries is a better example of this, if only because the std lib implementations are almost completely ignored by all other gems. Go's database/sql is actively used outside of the std lib to great affect, whether in wrappers and ORMs or in implementing other SQL databases (like Postgres).

reply

---

rcfox 9 hours ago [-]

Out of curiousity: what do the 5 string types do differently?

reply

geekingfrog 9 hours ago [-]

String: Linked list of Char. Nice for teaching, horrible in every other aspects. Text and lazy Text: modern strings, with unicode handling and so on. ByteString? and lazy ByteString?: these are actually arrays of bytes. Used to represent binary data. Because haskell is lazy by default, and sometimes you want strictness (mostly for performances), there are two variants of Text and ByteString?, and going from one flavor to the other requires manual conversion.

reply

tibbe 8 hours ago [-]

Risking to go off-topic a bit, I think the lazy versions of Text and ByteString? wouldn't have been needed if we had nice abstractions for streams (lists are not, they cause allocation we cannot get rid of) so that you don't need to implement a concrete stream type (e.g. lazy Text and lazy ByteString?) for every data type.

Rust does this well with iterators, for example.

reply

wyager 8 hours ago [-]

The problem is that streams actually have very complicated semantics when they interact with the real world. What does it mean to traverse an effectful stream multiple times? Can you even do that?

Data.Vector provides a very efficient stream implementation for vector operation fusion, but it's unsuitable for iterators/streams that interact with the real world. Pipes, on the other hand, combined with FreeT?