Revision 5 not available (showing current revision instead)

proj-oot-ootLibrariesNotes9

" 6/30/10 stupid idea 247: worlds smallest vector maths library Here is a tiny 3d vector maths library:

typedef float vec3[3]; vec3 *vec(vec3 *v, float x) {v[0]=v[1]=v[2]=x;return v;} float *mav(vec3 *v1,vec3 *v2,vec3 *v3,vec3 *v4,vec3 *v5){ for (int i=0; i!=3; i++) v3[i]=v1[i]*v4[i]+v2[i]*v5[i]; return v3[0]+v3[1]+v3[2]; } vec3 VZERO={0,0,0}; vec3 VONE={1,1,1}; vec3 VMONE={-1,-1,-1};

Thats it. It can do zero, set, add, sub, length2, dotprod, negate, lerp, centroid. e.g add is : mav(&v1, &v2, &v3, &VONE, &VONE ); v3=v2+v1

Lerp means that centroid (the centre vertex of a quad) can be found without a division by using two lerps. For a moment it seems cool until you realise the calls to it dont compress all that well. The challenges remaining are xprod, normalise and max/min. then I have everything for handling my 3d modeller qoob.If I can make minor adjustments to get those, this might be worth persuing. "

-- [1]

the author is trying to make a small (or compressible) CGI file format and renderer for demo intros. But for us the most notable thing here is the list of vector math library functions that e considers important:

---

https://github.com/japaric/steed

rec. by [2]

"

Current functionality

Check the API docs, but to summarize the functionality that interfaces with the Linux kernel:

    Standard I/O (stdin, stdout, stderr)
    File I/O
    Filesystem operations (std::fs)
    std::net: TCP, UDP. lookup_host is missing.
    Dynamic memory allocation (thanks to ralloc!)
    std::time
    Minimal thread support.

Yup, that's all! I did say it was very early days, didn't I? "

https://japaric.github.io/steed/steed/index.html

---

" In its minimum configuration, SQLite requires only the following routines from the standard C library:

    memcmp()
    memcpy()
    memmove()
    memset() 
    strcmp()
    strlen()
    strncmp() 

In a more complete build, SQLite also uses library routines like malloc() and free() and operating system interfaces for opening, reading, writing, and closing files. "

---

rec. by [3]:

https://www.codeproject.com/Articles/15156/Tiny-C-Runtime-Library

"Ever designed a simple utility program, such as a hex-dump program, only to find your simple program is a full 64K, optimized for size, when all it does is read a file and print to stdout? Ever wonder what happened to those good ol' DOS days where programs had to be small? Where a COM file was limited to 64K? Or when you can write a bare-bones DOS-style protected mode operating system kernel in about 64K?

...

Of course, by replacing the CRT, programs that rely on specifics of the Microsoft CRT will fail. For instance, if you go digging into the FILE structure, or expect a certain header on your memory allocations, or rely on the buffering features of stdio, or use locales, runtime checks, or C++ exception handling, you can't use this library. This library is aimed for use by small, simple programs, such as a hex-dump command line program or the many UNIX-style tools like cat or grep. "

---

knightOS, an OS for calculators. Seems like mostly a small and cool stdlib:

http://www.knightos.org/documentation/reference/

---

https://github.com/Quuxplusone/from-scratch

---

some immutable data structure libraries:

a Rust one: https://docs.rs/im/10.0.0/im/

a popular C++ one: https://github.com/arximboldi/immer

a misc/random one: http://smallcultfollowing.com/babysteps/blog/2018/02/01/in-rust-ordinary-vectors-are-values/

---

" A good list of system calls can be found here, but a common list of important ones is:

    open()
    close()
    read()
    write()
    lseek()
    stat()
    mmap()
    munmap()
    madvise()
    mprotect()
    fcntl()
    ioctl()
    connect()
    accept()
    bind()
    access()
    select()
    getpid()
    getuid()
    gettimeofday()
    clock_gettime/()
    time()
    mkdir()

" [4]

---

[5]

" Results

After recompiling the program with libctiny and the method above, the EXE jumped from a giant 64K to a much more reasonable 4K! (4096 bytes to be exact). For comparison, the entire code section of the linker map file is reproduced below: Hide Copy Code

0001:00000000 ?DumpFile?@@YAXPAD@Z 00401000 f hd.obj 0001:0000013d _main 0040113d f hd.obj 0001:0000021a _fopen 0040121a f libct:file.obj 0001:000002a7 _fread 004012a7 f libct:file.obj 0001:000003c2 _fwrite 004013c2 f libct:file.obj 0001:0000048b _fgetc 0040148b f libct:file.obj 0001:000004b6 _printf 004014b6 f libct:printf.obj 0001:000004ef _memset 004014ef f libct:memory.obj 0001:0000050e __doexit 0040150e f libct:initterm.obj 0001:0000053a _mainCRTStartup 0040153a f libct:crt0tcon.obj 0001:000005f5 _malloc 004015f5 f libct:alloc.obj 0001:00000607 __init_args 00401607 f libct:argcargv.obj 0001:00000705 __ismbcspace 00401705 f libct:isctype.obj

"

---

small GUI library:

https://github.com/ocornut/imgui

---

https://github.com/fantasyland/fantasy-land https://github.com/rpominov/static-land https://github.com/sanctuary-js/sanctuary-type-classes https://github.com/sanctuary-js https://github.com/sanctuary-js/sanctuary https://github.com/fluture-js/Fluture

https://monet.github.io/monet.js/

---

How does Fluture differ from other solutions?

The description of Fluture states that it's an alternative to Promises, so it's only natural that people want to compare it. In my article comparing Futures to Promises I write:

    On the surface Futures are just like Promises, but with the different behaviors of the .then method extracted into three distinct functions, each with a single responsibility.

The then method of a Promise is massively overloaded: You can give it zero to two arguments, both are mixed types (Nil and Function). The return values of the functions are also overloaded: You can return any value, but returning something with a then-method has a particular meaning. Throwing an error also has special meaning.

Extracting all of these behaviors to separate functions makes it easier to abstract over, and clarifies developer intent, making it simpler to detect mistakes.

I've also written about the differences between Fluture and other libraries which have a similar structure, in my article comparing them. I'll go into this when answering why I developed Fluture.

---

" The colors: the color format has been chosen so that more sensible and neutral colors are more likely than "coder colors". YUV has been chosen over HSV because there is relatively universal hardware support for YUV buffers (and I also think it is easier to get richer gradients with YUV than with HSV).

Trigonometric functions: I pondered for a long while whether to include SIN and ATAN2 and I finally decided to do so. A lot of demoscene tricks depend, including all kinds of rotating and bouncing things as well as more advanced stuff such as raycasting, depends on the availability of trigonometry. Both of these operations can be found in the FPU instruction set of the x86 and are relatively fundamental mathematical stuff, so we're not going into library bloat here. " [6]

---

" As you can see GLSL has more surprises. The GPU has hardware accelerated angle, trigonometric and exponential functions. Some of those functions are: sin(), cos(), tan(), asin(), acos(), atan(), pow(), exp(), log(), sqrt(), abs(), sign(), floor(), ceil(), fract(), mod(), min(), max() and clamp(). "

fract:

https://thebookofshaders.com/glossary/?search=fract

" Compute the fractional part of the argument

float fract(float x)

fract() returns the fractional part of x. This is calculated as x - floor(x). "

clamp:

https://thebookofshaders.com/glossary/?search=clamp

" Constrain a value to lie between two further values

float clamp(float x, float minVal, float maxVal)

clamp() returns the value of x constrained to the range minVal to maxVal. The returned value is computed as min(max(x, minVal), maxVal)."

---

" coding a GUI... With kivy you can target desktop and mobile with one code base, and some skins are pretty (https://imgur.com/a/IjM0XGW). With QT you can do powerful things on the desktop. With wx you can do simple GUI simply. "

---

Cannolib

https://github.com/joncatanio/cannolib Cannolib provides library support for Cannoli, including a number of types and modules that offload work that would have otherwise been done by the Cannoli compiler. Cannolib provides the implementation of the overall type system as well as built-in functions similar to those defined in the Python library. https://docs.python.org/3/library/functions.html#built-in-functions

" Some implementations of various standard library components provide a proof of concept for others and consequently were omitted. For instance, we support Python lists but omit sets. Although Cannolib does have some type restrictions, the compiler provides a fair amount of support. While not exhaustive, types include: numbers, strings, booleans, lists, tuples, functions, and classes.

...

Cannoli was designed around a few benchmarks that include features analyzed in this thesis. In order to run these benchmarks, Cannolib includes math and sys modules. These modules do not com- pletely mirror those in the Python standard library but do provide support for the aforementioned benchmarks. Along with the built-in modules are a subset of built-in functions like print, len, open, and enumerate.

" -- [7]

---

time advice from [8]:

" why this is such a dope format:

    Easy sorting: It arranges all components from large to small, so pretty much any novice in a programming language could easily sort a list of timestamps from latest to oldest.
    Timezone information: At the far end it includes the offset: -08:00, for example (well, not in this above example specifically, since in JavaScript you’d also need to parse this out manually with getTimezoneOffset()). This isn’t as high-fidelity as when we were talking earlier about storing the qualified string version of a timezone, but it does give us information we wouldn’t have otherwise. You don’t get this if you pass around UNIX epoch time, for example.
    No locale problems: You don’t have problems with Month/Day/Year formats getting confused with Day/Month/Year formatted dates. More on this in a few.

So yeah, use ISO 8601 over the wire. It’s the most popular way of tackling a shared standard for time, so don’t be the jerk who does it in some fancypants custom manner. "

recommended time libs:

" One of the ways you can do this on the web is using the Intl API. Intl supports a number of different hooks and abilities to handle locale- and language-based changes for numbers, plurals, and dates and times.

...

Some solid libraries to take a gander at:

    moment.js The classic time library in JavaScript. Date manipulation, formatting, pretty much everything you’d need.
    date-fns More modern approach to a moment.js-like experience for handling dates on the web.
    github/time-elements Web component extension to the <time> element. Also includes auto-updating timestamps, as well as some locale help.

"

---

another hash table implementation

https://probablydance.com/2018/05/28/a-new-fast-hash-table-in-response-to-googles-new-fast-hash-table/ https://news.ycombinator.com/item?id=17176713

---

db concurrency

some embedded db libs:

https://github.com/spacejam/sled

https://github.com/mozilla/mentat

---

the CIL standard, https://www.ecma-international.org/publications/files/ECMA-ST/ECMA-335.pdf , Part IV has various 'profiles' which are subsets of the defined std libs. The smallest profile is 'kernel', which is required to be supported for all CIL implementations.

Here is basic functionality (in addition to library domains) which is NOT included in std (page 438, section IV.4.1, Features excluded from the Kernel Profile):

which libraries ARE in the kernel profile:

---

on a secondary hash function to assign to buckets:

https://news.ycombinator.com/item?id=17328756 https://probablydance.com/2018/06/16/fibonacci-hashing-the-optimization-that-the-world-forgot-or-a-better-alternative-to-integer-modulo/

---

https://hackernoon.com/timsort-the-fastest-sorting-algorithm-youve-never-heard-of-36b28417f399 https://news.ycombinator.com/item?id=17436591 ---

Pipelines – a guided tour of the new IO API in .NET

https://blog.marcgravell.com/2018/07/pipe-dreams-part-1.html

---

dnautics 4 days ago [-]

Having programmed in the BeOS? API long ago, and now a BEAM language developer, I noticed that basically they ((he is replying to a comment about Haiku OS)) were implementing a crude and unsafe actor model. There must be modern libraries that provide that at a lower level than say the erlang VM, but with better safety guarantees than C++.

reply

tormeh 4 days ago [-]

Actix[0] is probably what you're looking for. I don't think the actors in Actix are network transparent, though, which is a bummer for distributed applications, but wouldn't matter for an OS.

0: https://actix.rs/

reply

losvedir 4 days ago [-]

I think a lower level, type safe, actor based framework is the idea behind the Pony language, which I've been meaning to check out.

reply

---

https://sorting.cr.yp.to/

---

https://github.com/kitao/pyxel

---

NortySpock? 13 hours ago [-]

Building on your reply, I have found that Processing (Java) and P5 (JavaScript?) have a similar easy-to-use design and the documentation is decent. (It's not opinionated on the color palette though.)

I've been using P5 to write retro video games to learn JavaScript?, and it's been a lot of fun using that to learn JS. (Vanilla JS sucks, but the learning process is at least fun).

https://processing.org

https://p5js.org

reply

---

https://rurban.github.io/safeclib/doc/safec-3.3/index.html http://www.drdobbs.com/cpp/the-safe-c-library/214502214

---

"

We’ve done a thorough review of all of Julia’s APIs to improve consistency and usability. Many obscure legacy names and inefficient programming patterns have been renamed or refactored to more elegantly match Julia’s capabilities. This has prompted changes to make working with collections more consistent and coherent, to ensure that argument ordering follows a consistent standard throughout the language, and to incorporate (the now faster) keyword arguments into APIs where appropriate. "

[9]

---

ppl's opinions on Win32 vs DotNet? vs UWP:

https://news.ycombinator.com/item?id=14307331

---

discussion of html-ish UI libraries:

https://news.ycombinator.com/item?id=17733515

---

libcwidget library

https://packages.debian.org/jessie/libcwidget-dev

---

commandline argparse

http://click.pocoo.org/5/

---

" 23.2 Basic System Services The following table summarizes the more common system services. Call Code (rax) System Service Description

0 SYS_read Read characters rdi = file descriptor (of where to read from) rsi = address of where to store characters rdx = count of characters to read If unsuccessful, returns negative value. If successful, returns count of characters actually read.

1 SYS_write Write characters rdi = file descriptor (of where to write to)

Call Code (rax) System Service Description rsi = address of characters to write rdx = count of characters to write If unsuccessful, returns negative value. If successful, returns count of characters actually written.

2 SYS_open Open a file rdi = address of NULL terminated file name rsi = file status flags (typically O_RDONLY) If unsuccessful, returns negative value. If successful, returns file descriptor.

3 SYS_close Close an open file rdi = file descriptor of open file to close If unsuccessful, returns negative value.

8 SYS_lseek Reposition the file read/write file offset. rdi = file descriptor (of where to write to) rsi = offset rdx = origin If unsuccessful, returns negative value.

57 SYS_fork Fork current process.

59 SYS_execve Execute a program rdi = Address of NULL terminated string for name of program to execute.

60 SYS_exit Terminate executing process. rdi = exit status (typically 0)

Call Code (rax) System Service Description

85 SYS_creat Open/Create a file. rdi = address of NULL terminated file name rsi = file mode flags If unsuccessful, returns negative value. If successful, returns file descriptor.

96 SYS_gettimeofday Get date and time of day rdi = address of time value structure rsi = address of time zone structure If unsuccessful, returns negative value. If successful, returns information in the passed structures. " -- [10] appendix C

Error Codes " Error Code Symbolic Name Description -1 EPERM Operation not permitted. -2 ENOENT No such file or directory. -3 ESRCH No such process. -4 EINTR Interrupted system call. -5 EIO I/O Error. -6 ENXIO No such device or address. -7 E2BIG Argument list too long. -8 ENOEXEC Exec format error. -9 EBADF Bad file number. -10 ECHILD No child process. -11 EAGAIN Try again. -12 ENOMEM Out of memory. -13 EACCES Permission denied. -14 EFAULT Bad address. -15 ENOTBLK Block device required. -16 EBUSY Device or resource busy. -17 EEXIST File exists. -18 EXDEV Cross-device link. -19 ENODEV No such device. -20 ENOTDIR Not a directory. -21 EISDIR Is a directory. -22 EINVAL Invalid argument. -23 ENFILE File table overflow. -24 EMFILE Too many open files. -25 ENOTTY Not a typewriter. -26 ETXTBSY Text file busy. -27 EFBIG File too large. -28 ENOSPC No space left on device. -29 ESPIPE Illegal seek. -30 EROFS Read-only file system. -31 EMLINK Too many links. -32 EPIPE Broken pipe. -33 EDOM Math argument out of domain of function. -34 ERANGE Math result not representable.

Only the most common error codes are shown. A complete list can be found via the Internet or by looking on the current system includes files. For Ubuntu, this is typically located in /usr/include/asm-generic/errno-base.h . " [11] appendix C

---

https://github.com/cfenollosa/os-tutorial/tree/master/23-fixes/libc

---

https://build-system.fman.io/pyqt5-tutorial

---

" C and C++ runtime libraries

The following ARM runtime libraries are provided to support compiled C and C++:

C standardlib

    This is a C library consisting of:
        All functions defined by the ISO C99 library standard.
        Target-dependent functions used to implement the C library functions in the semihosted execution environment. You can redefine these functions in your own application.
        Functions called implicitly by the compiler.
        ARM extensions that are not defined by the ISO C library standard, but are included in the library.

C microlib

    This is a C library that can be used as an alternative to C standardlib. It is a micro-library that is ideally suited for deeply embedded applications that have to fit within small-sized memory. The C micro-library, microlib, consists of:
        Functions that are highly optimized to achieve the minimum code size.
        Functions that are not compliant with the ISO C library standard.
        Functions that are not compliant with the 1985 IEEE 754 standard for binary floating-point arithmetic.

C++

    This is a C++ library that can be used with C standardlib. It consists of:
        functions defined by the ISO C++ library standard
        the Rogue Wave Standard C++ library
        additional C++ functions not supported by the Rogue Wave library
        functions called implicitly by the compiler.
    The C++ libraries depend on the C library for target-specific support. There are no target dependencies in the C++ libraries.
    "

[12]

---

https://github.com/dhall-lang/Prelude

---

Google core libraries for Java

https://github.com/google/guava

---

" what happens when you’re accessing a hash table from multiple threads? Do you have to do some synchronization somehow, or what? In CCL, hash tables happen to be thread-safe and they use this fancy-pants lock-free algorithm that does all this complicated stuff and tends to work pretty well for read-mostly hash tables. There’s keyword arguments to make different kinds of hash tables that might work better in certain circumstances. " [13]

---

not sure if we want to do this:

" I wondered why there’s a reduce in clojure.core but no fold? (It’s just a different arity of reduce that’s missing the initial state.) " [14]

---

" I wondered what the equivalent of F#’s Seq.zip was in Clojure but a quick search turned up nothing. I didn’t realize you could simply pass more than one collection to map, and the step function would then take an argument per collection. This seemed really like a really elegant use of variable-arity functions and there are several examples of this technique in clojure.core. " [15]

---

[16] says "I do miss nullable comparison operators", linking to [17]. I think the idea is that the left-nullable variant of '>', '?>', returns 'false' when the left side is null. There is alos >? and ?>?.

The use is that you can replace a filter like row.TestData?1.HasValue? && row.TestData?1.Value > 2 with row.TestData?1 ?> 2

---

https://blogs.windows.com/buildingapps/2018/12/04/announcing-open-source-of-wpf-windows-forms-and-winui-at-microsoft-connect-2018/

---

ipython starts really slow on WSL. I'm guessing because a zillion files are scanned/open. Design library packaging so that this doesn't happen, at least not upon startup.

---

" As of a few days ago, Wasmjit is able to run Nginx 1.15.3 in user space ... All the complex bits of the POSIX API required for Nginx have been implemented, including signal handling and forking. ... Kernel space support still requires more work. Emscripten delegates certain large APIs like getaddrinfo() and strftime() to the host implementation, and these would require re-implementation in the kernel. In addition, implementing kernel space versions of fork(), execve(), and signal handling still needs to be done. We will publish performance benchmarks when this work is completed.

"

-- [18]

---

EA STL:

http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2271.html

---

https://github.com/idea4good/GuiLite

---

https://github.com/pybee/ouroboros

---

" The first tool for producing WebAssembly? was Emscripten. It emulates a particular OS system interface, POSIX, on the web. This means that the programmer can use functions from the C standard library (libc).

To do this, Emscripten created its own implementation of libc. This implementation was split in two — part was compiled into the WebAssembly? module, and the other part was implemented in JS glue code. This JS glue would then call into the browser, which would then talk to the OS.

...

What will be in wasi-core?

wasi-core will contain the basics that all programs need. It will cover much of the same ground as POSIX, including things such as files, network connections, clocks, and random numbers.

And it will take a very similar approach to POSIX for many of these things. For example, it will use POSIX’s file-oriented approach, where you have system calls such as open, close, read, and write and everything else basically provides augmentations on top.

But wasi-core won’t cover everything that POSIX does. For example, the process concept does not map clearly onto WebAssembly?. And beyond that, it doesn’t make sense to say that every WebAssembly? engine needs to support process operations like fork. But we also want to make it possible to standardize fork.

This is where the modular approach comes in. This way, we can get good standardization coverage while still allowing niche platforms to use only the parts of WASI that make sense for them.

...

Languages like Rust will use wasi-core directly in their standard libraries. For example, Rust’s open is implemented by calling __wasi_path_open when it’s compiled to WebAssembly?.

For C and C++, we’ve created a wasi-sysroot that implements libc in terms of wasi-core functions.

...

WASI gives us a way to extend this security even further. It brings in more concepts from capability-based security.

Traditionally, if code needs to open a file, it calls open with a string, which is the path name. Then the OS does a check to see if the code has permission (based on the user who started the program).

With WASI, if you’re calling a function that needs to access a file, you have to pass in a file descriptor, which has permissions attached to it. This could be for the file itself, or for a directory that contains the file.

This way, you can’t have code that randomly asks to open /etc/passwd. Instead, the code can only operate on the directories that are passed in to it.

...

This makes it possible to safely give sandboxed code more access to different system calls — because the capabilities of these system calls can be limited.

And this happens on a module-by-module basis. By default, a module doesn’t have any access to file descriptors. But if code in one module has a file descriptor, it can choose to pass that file descriptor to functions it calls in other modules. Or it can create more limited versions of the file descriptor to pass to the other functions.

...

These concepts come from capability-oriented systems, like CloudABI? and Capsicum. One problem with capability-oriented systems is that it is often hard to port code to them. But we think this problem can be solved.

If code already uses openat with relative file paths, compiling the code will just work.

If code uses open and migrating to the openat style is too much up-front investment, WASI can provide an incremental solution. With libpreopen, you can create a list of file paths that the application legitimately needs access to. Then you can use open, but only with those paths.

We think wasi-core is a good start. It preserves WebAssembly’s? portability and security, providing a solid foundation for an ecosystem.

But there are still questions we’ll need to address after wasi-core is fully standardized. Those questions include:

    asynchronous I/O
    file watching
    file locking

"

---

https://github.com/WebAssembly/WASI/blob/master/design/WASI-core.md

(dup?:) https://github.com/CraneStation/wasmtime/blob/master/docs/WASI-api.md

---

https://security.googleblog.com/2019/03/open-sourcing-sandboxed-api.html?m=1

---

In K, "There are only six predefined adverbs and (unlike J) there is no provision for user-defined ones." [19]. K adverbs are HOFs, i think. Should probably include their 6.

---

In the examples in [20], we see that the following HOFs (adverbs) seem to be useful:

map reduce scan (like reduce, but each intermediate reduction is output) map until fixpoint (that is, apply function f to argument x, then apply function f to that result, etc, over and over until f returns as output the thing that it was input) fmap (functor map/catamorphism) (by which i mean, apply some function mapped over a list, but if the list has sublists, then recurse into these and only apply the function to 'atoms' -- we can generalize this by (a) using Oot's concept of 'boundaries' to dynamically define 'atoms' (so some sublists could be counted as 'atoms' temporarily), and (b) letting the function test some predicate on its input to determine whether to recurse further (so, a different way of dynamically defining 'atom'))

it also has 'join', although i guess that is a 'verb' not an 'adverb'

---

[21] gives some examples of fmap (functor map/catamorphism) being useful, although here the flatmap-ness is built-in to other operators (like negation and addition of a constant):

" For example, given a: (1; (2; 3; (4 5; 6); 7); 8; 9), then -a is (-1; (-2; -3; (-4 -5; -6); -7); -8; -9), 100+a is (101; (102; 103; (104 105; 106); 107); 108; 109), and 10 20 30 40 + a is (11; (22; 23; (24 25; 26); 27); 38; 49). "

---

? would be useful as the trinary conditional operator, though (like in C). i guess that doesn't need to be punctuation tho? could use a single capital for that?

---

https://asylo.dev/docs/reference/runtime.html

---

https://haskell-distributed.github.io/

"Cloud Haskell: Erlang-style concurrent and distributed programming in Haskell. The Cloud Haskell Platform consists of a generic network transport API, libraries for sending static closures to remote nodes, a rich API for distributed programming and a set of platform libraries modelled after Erlang’s Open Telecom Platform."

---

" Other standard library modules are simply inferior to alternatives on PyPI?. The http.client documentation advises readers to use Requests, and the datetime module is confusing compared to its competitors such as arrow, dateutil, and moment. "

---

detaro 5 hours ago [-]

afaik it's not in the UI, but the dataset is published and and access provided by other sites: https://pypistats.org/

reply

---

yingw787 11 hours ago [-]

I agree with Amber’s point that more stuff should be moved from the standard library to PyPI?. I made my first pull request to CPython during the development sprints this year, and it’s honestly not the best experience. Everything is built from scratch in CI after every commit, even a documentation change. There’s nowhere near enough CI builds and pipelines for everything Python supports. Pull requests are outstanding for several months, and there’s at least a thousand PRs open when I checked this morning.

I’m not sure if Python’s ideal solution is to reduce stdlib and have endorsed packages in PyPI?, but it would be an improvement over the current process.

reply

---

killjoywashere 55 minutes ago [-]

> Standard Library Modules Crowd Out Innovation

This heading is the essential problem in innovation writ large: some giant can ignore you and squash you without any effort at all, without even considering your existence.

reply

---

bscphil 3 hours ago [-]

>Obviously it doesn't apply to everyone, and it certainly doesn't apply to most startups or open source developers, but I spent most of the last 20 years working in environments where you have to get permission for every third party library you bring on to the network.

Situations like this will really make you appreciate "batteries included". I think this particular issue is fairly revealing of the attitudes common among programmers of different languages. I think it's a good thing to be skeptical of a program pulling in a bunch of standard libraries over the Internet. It worries me when I find something on Github I want to try and I can't download and compile it without it pulling in 30 or 100 other libraries that I haven't looked at or decided to trust come along for the ride. I don't like that way of doing software, and unfortunately it's the norm in node and starting to become a norm in Rust as well. Real security fails have been caused this way in node's case at least.

Even in cases when Python programs depend on external libraries, I usually don't need to use pip for anything because Python programs will pull in dependencies provided by your distribution just fine. (My distribution doesn't even have any Rust libraries, so even if dynamic linking is possible in Rust not many people are shipping software that way.)

"Download by default" is a worse way of doing things, and it makes me sad to see newer languages like Go and Rust embracing it.

reply

---

" tanin 3 hours ago [-] ... people have to resort to ugly solutions for a simple problem (here's an example: https://stackoverflow.com/questions/363944/python-idiom-to-r...) https://stackoverflow.com/questions/363944/python-idiom-to-return-first-item-or-none

reply "

---

aetherspawn 3 hours ago [-]

To be fair, I don’t think any Haskeller would actually write this project without attoparsec and maybe lens by choice. They’re basically base libraries for this sort of thing.

---

" Newtonsoft.JSON is _the_ most downloaded third-party library for .NET: https://www.nuget.org/stats/packages I like it, very flexible, in a good way. "

---

https://docs.microsoft.com/en-us/dotnet/standard/design-guidelines/index

---

http://www.maizure.org/projects/decoded-gnu-coreutils/index.html

---

https://ocaml.janestreet.com/ocaml-core/latest/doc/base/Base/index.html https://github.com/janestreet/base

https://github.com/janestreet/core

http://batteries.forge.ocamlcore.org/

---

There was a recent effort to write a nice intro for a haskell package since haskell pkgs tend to have poor docs: https://www.reddit.com/r/haskell/comments/9t7jmp/haskell_worse_than_go_ocaml_yes_this_is_a/e8vc60b/

Here is an example of a good intro to a Haskell package:

https://haskell-containers.readthedocs.io/en/latest/intro.html

Here is a template for haskell package docs that that good intro used:

https://github.com/m-renaud/haskell-rtd-template

---

https://pl-rants.net/posts/libraries-vs/

---

http://libcxx.llvm.org/

---

https://github.com/denoland/deno_std

---

https://preshing.com/20130605/the-worlds-simplest-lock-free-hash-table/

"Junction is a library of concurrent data structures in C++. It contains several hash map implementations:" https://github.com/preshing/junction

---

https://github.com/preshing/turf

" Turf is a configurable C++ platform adapter. It defines a common API for:

    Thread creation
    Thread affinities
    Thread IDs
    Atomic operations
    Mutexes
    Condition variables
    Read-write locks
    Semaphores
    Events
    Timers
    Virtual memory
    Heap allocators
    Asserts

It then implements those things using POSIX, Win32, Mach, Linux, Boost, C++11 and possibly other platform APIs. You configure Turf to use the API you want. "

---

https://doc.rust-lang.org/nightly/std/prelude/index.html

---

something like Python join but with an option to add the delimited before the first item, and/or after the last item, only if there are any items

---

https://gitlab.com/z0mbie42/rust_gui_ecosystem_overview

https://news.ycombinator.com/item?id=20775773

---

https://github.com/git/git/blob/master/banned.h

/* * This header lists functions that have been banned from our code base, * because they're too easy to misuse (and even if used correctly, * complicate audits). Including this header turns them into compile-time * errors. */

  1. define BANNED(func) sorry_##func##_is_a_banned_function
  2. undef strcpy
  3. define strcpy(x,y) BANNED(strcpy)
  4. undef strcat
  5. define strcat(x,y) BANNED(strcat)
  6. undef strncpy
  7. define strncpy(x,y,n) BANNED(strncpy)
  8. undef strncat
  9. define strncat(x,y,n) BANNED(strncat)
  10. undef sprintf
  11. undef vsprintf
  12. ifdef HAVE_VARIADIC_MACROS
  13. define sprintf(...) BANNED(sprintf)
  14. define vsprintf(...) BANNED(vsprintf)
  15. else
  16. define sprintf(buf,fmt,arg) BANNED(sprintf)
  17. define vsprintf(buf,fmt,arg) BANNED(sprintf)
  18. endif

dblock 5 hours ago [-]

Windows programmers are familiar with StrSafe?.h, https://en.wikipedia.org/wiki/Strsafe.h and https://github.com/dotnet/coreclr/blob/master/src/pal/inc/st... which go a lot further.

Also found https://github.com/mubix/netview/blob/master/banned.h on Github with a better list.

reply

---

mculib

newlib

---

freertos, zephyr, rt-thread, uC/OS-II, MyNewt?, SylixOS?, LIteOS?, AliOS? Things

---

could call the core/base/default lib 'substd' to keep it straight -- then the one preceding that can be called subsubstd.

---

https://pymotw.com

---