![[Home]](http://www.bayleshanks.com/cartoonbayle.png)
---
" Math functions
For many years, users literally begged SQLite devs to add basic functions like sqrt(), log() and pow(). The answer was always about the same:
SQLite is called ‘lite’ for a reason. If you need functions, add them yourself.
An understandable position indeed. But refusing to add the square root? At the same time implementing window functions, recursive queries and other advanced SQL magic? Seriously?
Maybe SQLite developers prefer to focus on features that large customers are willing to pay for. Anyway, after 20 years we now have mathematical functions!
Here is the full list:
acos(X) acosh(X) asin(X) asinh(X) atan(X) atan2(X,Y) ceil(X) ceiling(X) cos(X) cosh(X) degrees(X) exp(X) floor(X) ln(X) log(B,X) log(X) log10(X) log2(X) mod(X,Y) pi() pow(X,Y) power(X,Y) radians(X) sin(X) sinh(X) sqrt(X) tan(X) tanh(X) trunc(X)
"
---
"
samatman 8 hours ago [–]
`RETURNING` will substantially clean up my code, and I already have one migration which could have just been a `DROP COLUMN`, so this is great news.
On the subject of "it's called 'lite' for a reason", my wishlist does include library functions for working with RFC 3339 timestamps. SQLite already ships with a fairly large suite of JSON tools, which are optional to compile into the library, so there's precedent.
Datetimes are of those things which is incredibly annoying to get right, and really belongs inside the database. RFC 3339 timestamps are already well designed, since if you stick to UTC (and if you don't store timezone data separately you deserve those problems), lexical order is temporal order, but queries which would be rendered in English as "return all accounts where last payment is ninety days prior to `now`" isn't really possible with string comparisons.
Also, with the JSON library, you can use a check constraint to fail if a string isn't valid JSON, another affordance I would love to have for datetimes.
Grateful for what we just got, though! Just daydreaming...
reply
nalgeon 8 hours ago [–]
SQLite has ISO-8601 compatible date functions, isn't that enough?
sqlite> select datetime('now', '-90 days');
2020-12-12 21:44:22https://sqlite.org/lang_datefunc.html
reply
samatman 4 hours ago [–]
Beats a swift kick in the pants!
You're right, that was a bad example. Maybe it's just me, but I've never figured out how to get SQLite to do a query like "select from orders where the order was on a Tuesday in Pacific time". I don't think you can; that requires predicates, and all I see are strftime and some useful pre-cooked variations on it.
reply "
---
ritchie46 7 hours ago [–]
Hi Author here,
Polars is not an alternative to PyArrow?. Polars merely uses arrow as its in-memory representation of data. Similar to how pandas uses numpy.
Arrow provides the efficient data structures and some compute kernels, like a SUM, a FILTER, a MAX etc. Arrow is not a query engine. Polars is a DataFrame? library on top of arrow that has implemented efficient algorithms for JOINS, GROUPBY, PIVOTs, MELTs, QUERY OPTIMIZATION, etc. (the things you expect from a DF lib).
Polars could be best described as an in-memory DataFrame? library with a query optimizer.
Because it uses Rust Arrow, it can easily swap pointers around to pyarrow and get zero-copy data interop.
DataFusion? is another query engine on top of arrow. They both use arrow as lower level memory layout, but both have a different implementation of their query engine and their API. I would say that DataFusion? is more focused on a Query Engine and Polars is more focused an a DataFrame? lib, but this is subjective.
Maybe its like comparing Rust Tokio vs Rust async-std. Just different implementations striving the same goal. (Only Polars and DataFusion? can easily be mixed as they use the same memory structures).
reply
---
on Debian, installing npm installs:
gyp libc-ares2 libjs-inherits libjs-is-typedarray libjs-psl libjs-typedarray-to-buffer libnode-dev libnode72 libssl-dev libssl1.1 libssl1.1:i386 libuv1-dev node-abbrev node-ajv node-ansi node-ansi-align node-ansi-regex node-ansi-styles node-ansistyles node-aproba node-archy node-are-we-there-yet node-asap node-asn1 node-assert-plus node-asynckit node-aws-sign2 node-aws4 node-balanced-match node-bcrypt-pbkdf node-bl node-bluebird node-boxen node-brace-expansion node-builtin-modules node-builtins node-cacache node-call-limit node-camelcase node-caseless node-chalk node-chownr node-ci-info node-cli-boxes node-cliui node-clone node-co node-color-convert node-color-name node-colors node-columnify node-combined-stream node-concat-map node-concat-stream node-config-chain node-configstore node-console-control-strings node-copy-concurrently node-core-util-is node-cross-spawn node-crypto-random-string node-cyclist node-dashdash node-debbundle-es-to-primitive node-debug node-decamelize node-decompress-response node-deep-extend node-defaults node-define-properties node-delayed-stream node-delegates node-detect-indent node-detect-newline node-dot-prop node-duplexer3 node-duplexify node-ecc-jsbn node-editor node-encoding node-end-of-stream node-err-code node-errno node-es6-promise node-escape-string-regexp node-execa node-extend node-extsprintf node-fast-deep-equal node-find-up node-flush-write-stream node-forever-agent node-form-data node-from2 node-fs-vacuum node-fs-write-stream-atomic node-fs.realpath node-function-bind node-gauge node-genfun node-get-caller-file node-get-stream node-getpass node-glob node-got node-graceful-fs node-gyp node-har-schema node-har-validator node-has-flag node-has-symbol-support-x node-has-to-string-tag-x node-has-unicode node-hosted-git-info node-http-signature node-iconv-lite node-iferr node-import-lazy node-imurmurhash node-inflight node-inherits node-ini node-invert-kv node-ip node-ip-regex node-is-npm node-is-obj node-is-object node-is-path-inside node-is-plain-obj node-is-retry-allowed node-is-stream node-is-typedarray node-isarray node-isexe node-isstream node-isurl node-jsbn node-json-parse-better-errors node-json-schema node-json-schema-traverse node-json-stable-stringify node-json-stringify-safe node-jsonify node-jsonparse node-jsonstream node-jsprim node-latest-version node-lazy-property node-lcid node-libnpx node-locate-path node-lockfile node-lodash node-lodash-packages node-lowercase-keys node-lru-cache node-make-dir node-mem node-mime node-mime-types node-mimic-fn node-mimic-response node-minimatch node-minimist node-mississippi node-mkdirp node-move-concurrently node-ms node-mute-stream node-nopt node-normalize-package-data node-npm-bundled node-npm-package-arg node-npm-run-path node-npmlog node-number-is-nan node-oauth-sign node-object-assign node-once node-opener node-os-locale node-os-tmpdir node-osenv node-p-cancelable node-p-finally node-p-is-promise node-p-limit node-p-locate node-p-timeout node-package-json node-parallel-transform node-path-exists node-path-is-absolute node-path-is-inside node-performance-now node-pify node-prepend-http node-process-nextick-args node-promise-inflight node-promise-retry node-promzard node-proto-list node-prr node-pseudomap node-psl node-pump node-pumpify node-punycode node-qs node-qw node-rc node-read node-read-package-json node-readable-stream node-registry-auth-token node-registry-url node-request node-require-directory node-require-main-filename node-resolve node-resolve-from node-retry node-rimraf node-run-queue node-safe-buffer node-semver node-semver-diff node-set-blocking node-sha node-shebang-command node-shebang-regex node-signal-exit node-slash node-slide node-sorted-object node-spdx-correct node-spdx-exceptions node-spdx-expression-parse node-spdx-license-ids node-sshpk node-ssri node-stream-each node-stream-iterate node-stream-shift node-strict-uri-encode node-string-decoder node-string-width node-strip-ansi node-strip-eof node-strip-json-comments node-supports-color node-tar node-term-size node-text-table node-through node-through2 node-timed-out node-tough-cookie node-tunnel-agent node-tweetnacl node-typedarray node-typedarray-to-buffer node-uid-number node-unique-filename node-unique-string node-unpipe node-uri-js node-url-parse-lax node-url-to-options node-util-deprecate node-uuid node-validate-npm-package-license node-validate-npm-package-name node-verror node-wcwidth.js node-which node-which-module node-wide-align node-widest-line node-wrap-ansi node-wrappy node-write-file-atomic node-xdg-basedir node-xtend node-y18n node-yallist node-yargs node-yargs-parser nodejs nodejs-doc
---
https://github.com/gruns/icecream
kissgyorgy 18 hours ago [–]
You can do the same thing with Python 3.8+ by using f-strings and just appending "=" to the variable name:
>>> print(f"{d['key'][1]=}")
d['key'][1]='one'reply
grun 8 hours ago [–]
Hey! I'm Ansgar. I wrote Icecream.
f-strings's `=` is awesome. I'm overjoyed it was added to Python. I use it all the time.
That said, IceCream? does bring more to the table, like:
etc
I hope that helps!
reply
est 16 hours ago [–]
You can also use the breakpoint() introduced in py3.7 via PEP533
https://www.python.org/dev/peps/pep-0553/
reply
---
https://github.com/nothings/stb
---
http://smtlib.cs.uiowa.edu/index.shtml
---
a paper about the cons of fork:
A fork()in the road https://www.microsoft.com/en-us/research/uploads/prod/2019/04/fork-hotos19.pdf https://dl.acm.org/doi/abs/10.1145/3317550.3321435
what they suggest instead:
" High-level: Spawn.In our opinion, most uses of fork andexec would be best served by a spawn API...Theposix_spawn()API can ease such refactoring. Ratherthan requiring that all parameters affecting a new pro-cess be provided at a single call-site (as is the case forCreateProcess()), spawn attributes are set by extensibly-defined helper functions. A post-forkclose(), for example,can be replaced by a pre-spawn call that records a “closeaction” to occur in the child. Unfortunately, this means thatthe API is specified as if it were implemented by fork andexec, although it is not actually required [32].The main drawback ofposix_spawn()is that it is not acomplete replacement for fork and exec. Some less-commonoperations, such as setting terminal attributes or switchingto an isolated namespace, are not yet supported. It also lacksan effective error-reporting mechanism: failures occurring inthe context of the child before it begins execution (such as in-valid file descriptor parameters) are reported asynchronouslyand are indistinguishable from the child’s termination. Theseshortcomings can and should be corrected. ... Low-level: Cross-process operations....an alternative model where system calls thatmodify per-process state are not constrained to merely thecurrent process, but rather can manipulate any process towhich the caller has access. This yields the flexibility andorthogonality of the fork/exec model, without most of itsdrawbacks: a new process starts as an empty address space,and an advanced user may manipulate it in a piecemeal fash-ion, populating its address-space and kernel context prior toexecution, without needing to clone the parent nor run codein the context of the child. ExOS? [43] implemented fork inuser-mode atop such a primitive. Retrofitting cross-processAPIs into Unix seems at first glance challenging, but mayalso be productive for future research. ... Copy-on-write memory...POSIX would benefit from an API for using copy-on-writememory independently of forking a new process. Bittau[16]proposedcheckpoint()andresume()calls to take copy-on-write snapshots of an address space, thus reducing the over-head of security isolation. More recently, Xu et al.[82]ob-served that fork time dominates the performance of fuzzingtools, and proposed a similarsnapshot()API. These designsare not yet general enough to cover all the use-cases outlinedabove, but perhaps can serve as a starting point "
related discussion: https://gist.github.com/nicowilliams/a8a07b0fc75df05f684c23c18d7db234 https://news.ycombinator.com/item?id=30502392 https://news.ycombinator.com/item?id=19621799
the author of the gist comments in https://news.ycombinator.com/item?id=30502392:
" I vehemently disagree with those who say that vfork() is much more difficult to use correctly than fork(). Neither is particularly easy to use though. Both have issues to do with, e.g., signals. posix_spawn() is not exactly trivial to use, but it is easier to use it correctly than fork() or vfork(). And posix_spawn() is extensible -- it is not a dead end.
My main points are that vfork() has been unjustly vilified, fork() is really not good, vfork() is better than fork(), and we can do better than vfork(). That said, posix_spawn() is the better answer whenever it's applicable. "
so.. sounds like posix_spawn is what we should focus on?
---
more opinions on the 'fork' model:
https://news.ycombinator.com/item?id=26984986
---
" Rust is the only programming language I've used that attempts to expose operating system primitives like environment variables, command arguments, and filesystem paths and doesn't completely mess it up. Truth be told, this is kind of a niche topic. But as I help maintain a version control tool which needs to care about preserving content identically across systems, this topic is near and dear to my heart.
In POSIX land, primitives like environment variables, command arguments, and filesystem paths are char*, or a bag of null-terminated bytes.
On Windows, these primitives are wchar_t*, or wide bytes.
On both POSIX and Windows, the encoding of the raw bytes can be... complicated.
Nearly every programming language / standard library in existence attempts to normalize these values to its native string type, which is typically Unicode or UTF-8. That's doable and correct a lot of the time. Until it isn't.
Rust, by contrast, has standard library APIs like std::env::vars() that will coerce operating system values to Rust's UTF-8 backed String type. But Rust also exposes the OsString? type, which represents operating system native strings. And there are function variants like std::env::vars_os() to access the raw values instead of the UTF-8 normalized ones.
Rust paths internally are stored as OsString?, as that as the value passed to the C API to perform filesystem I/O. However, you can coerce paths to String easily enough or define paths in terms of String without jumping through hoops. " [1]
---
" It is easy to forget that JavaScript? comes with a lot of batteries included (despite the claim that it doesn’t have a standard library). For example: You can handle arrays, objects, iterate over keys and values, split strings, filter, map, have prototypical inheritance and so on and so forth. All that is built into the JavaScript? engine. WebAssembly? comes with nothing, except arithmetic. "
---
https://observablehq.com/@observablehq/plot
---
mitchs 6 minutes ago [–]
Or just cast the pointer to uint##_t and use be##toh and htobe## from <endian.h>? I think this is making a mountain out of a mole hill. I've spent tons of time doing wire (de)serialization in C for network protocols and endian swaps are far from the most pressing issue I see. The big problem imo is the unsafe practices around buffer handling allowing buffer over runs.
reply
Animats 2 hours ago [–]
Rust gets this right. These primitives are available for all the numeric types.
u32::from_le_byte(bytes) // u32 from 4 bytes, little endian
u32::from_be_byte(bytes) // u32 from 4 bytes, big endian
u32::to_le_bytes(num) // u32 to 4 bytes, little endian
u32::to_be_bytes(num) // u32 to 4 bytes, big endianThis was very useful to me recently as I had to write the marshaling and un-marshaling for a game networking format with hundreds of messages. With primitives like this, you can see what's going on.
reply
(those are comments on https://justine.lol/endian.html )
---
https://github.com/golang/go/issues/45955 proposal: slices: new package to provide generic slice functions #45955
---
https://github.com/skullchap/chadstr
---
https://github.com/FrozenVoid/C-headers
---
https://arrayfire.org/docs/index.htm
---
https://github.com/SerenityOS/serenity/tree/master/AK AK is SerenityOS?'s C stdlib analog
---
"LLVM’s built-in unordered containers have a mode where iteration order is reversed" to catch bugs where someone accidentally/unknowingly used an unordered container when they needed an ordered one
---
various flaws in Golang's libraries, having to do with filesystem file attributes, filesystem paths, clocks, cross-platform ifdefs; and suggestions about how it should be (usually based on how Rust does the same thing): https://fasterthanli.me/articles/i-want-off-mr-golangs-wild-ride
---
twic on Feb 28, 2020 [–]
A post about fixing Date in JavaScript? got me thinking about why it took so long for languages to get good date/time APIs.
I think it's because it took so long to accept that date and time really is complicated.
If you sit down and work it out carefully, you end up with Joda-Time (more or less - not in all the details, but in the set of abstractions). If you balk at that and make something simpler, you make a subtly but fundamentally broken API.
It took a long time for us to get comfortable with the level of complexity in Joda-Time, but now nobody thinks a serious date/time API can be substantially simpler.
It sounds to me like you and the author are saying that Go does this balking systematically.
alexhutcheson on Feb 28, 2020 [–]
The author of Joda-Time actually thinks that even Joda-Time didn't get it quite right, and believes the java.time libraries in Java 8 and above (aka JSR-310[1]) are better than Joda-Time: https://blog.joda.org/2009/11/why-jsr-310-isn-joda-time_4941...
It turns out that abstractions for time are really hard to get right.
[1] https://jcp.org/en/jsr/detail?id=310
dehrmann on Feb 29, 2020 [–]
Props to him for not having an ego with that. Jodatime even recommends using JSR-310 time for new development.
nogridbag on Feb 29, 2020 [–]
The author of Joda was the primary person responsible for the new Java Date and Time API (JSR-310).
---
" sql.Result (the return value of Exec) has a LastInsertId?() that's an int64, so if you're using uuids, you can't use that at all and have to call Query instead and manage generated IDs yourself. "
-- https://news.ycombinator.com/item?id=22445165
uhoh-itsmaciek on Feb 28, 2020 [–]
Not to mention that sql.Result (the return value of Exec) has a LastInsertId?() that's an int64, so if you're using uuids, you can't use that at all and have to call Query instead and manage generated IDs yourself.
HelloNurse? on Feb 28, 2020 [–]
This is a more ridiculous symptom of bad library design than the filesystem trouble mentioned by the article.
In the real world, executing most SQL statement could be made to return a semi-useful integer according to simple and consistent rules (e.g. affected row count, -1 if there's no meaningful integer).
But the official Go documentation
https://golang.org/pkg/database/sql/#Result
makes it quite clear that the Go design committee decided to imitate a remarkably limited and inelegant MySQL? function that returns the value of an auto-increment column, not even realizing that only a few statements have auto-increment columns to begin with. I'd call this a negative amount of design effort.
LastInsertId returns the integer generated by the database in response to a command. Typically this will be from an "auto increment" column when inserting a new row. Not all databases support this feature, and the syntax of such statements varies.
(Of course, MySQL?