proj-oot-ootLibrariesNotes6

AsyncAwait? 1 day ago [-]

> a lot of the modules I picked for my projects in Rust are not actively maintained

There's an effort underway to 'bless' crates for common tasks, (ie tokio, futures for async, hyper for http etc.), so this will get better soon.

reply

galangalalgol 1 day ago [-]

Do you think there will ever be a standalone package that has all the blessed crates? Getting approval to use boost at work was hard enough. Getting them to approve any random 3rd tier dependency I might unintentionally pull down from crates is not even worth asking.

reply

steveklabnik 1 day ago [-]

It's not clear. The Rust core team proposed such a thing, but it was resoundingly rejected by the community. There are some people working on projects like this, http://github.com/brson/stdx for example.

reply

---

http://stackoverflow.com/questions/28459498/why-are-java-streams-once-off

---

itaysk 12 hours ago [-]

I really wanted to love Scala, and for the first few days I did. But it was a never ending story learning this language. It is so complicated and feature rich that there's just too much to learn. I couldn't just read someone's code without stumbling on some special syntax or language feature that handles a once in a lifetime use case that I didn't know about. Eventually I gave up on Scala.

reply

acjohnson55 11 minutes ago [-]

In my opinion, the tough part is that you need to truly understand each feature. Not just one use case.

Although, once you do, you can really get to work composing the features in interesting ways to get really useful effects. Like implicits + higher-kinded types to get ad hoc polymorphism. Or subtyping + pattern matching to achieve incredibly self-descriptive business logic. I would certainly agree that Scala appeals to people who like to be able to experiment and engage with more theoretical concepts while getting things done.

But let's not pretend that most languages don't have this issue. Programming with major Ruby and Python libraries tends to eventually require you to learn what's going on under the hood in the same way. I would argue that once you start digging under the surface, Scala is much more elegantly designed than those languages.

Go is about the only language that seems to say, "forget theory, we're simply about getting things done". Not my thing, but I understand the appeal!

reply

---

sreque 5 hours ago [-]

As someone who is currently having to deal with a monolithic Java app built on spring, hibernate, aspect weaving, code drenched in null checks, and other questionable architecture choices, this article makes me want to switch to Scala, and the reasons are mainly community, not programming language.

I think you could do a lot of the above In Java although the code would be a little boiler-platey. But why do people care about boiler plate so much? Hasn't GoLang? shown us that boilerplate is not our greatest enemy?

reply

dustingetz 5 hours ago [-]

It's not the boilerplate. When you take some of the best java developers on the planet, after a couple years you end up with a AbstractSingletonProxyFactoryBean?, so what is a mere mortal supposed to end up with? Evolve any Java codebase for N years and the result is the same.

http://docs.spring.io/spring-framework/docs/2.5.x/api/org/sp...

You could also make a similar argument for Scala. Not boilerplate, but other problems. There are some epic scala flamewars out there. A mortal could be forgiven for looking at an "idiomatic" scala codebase and scratching their head, looking at all this higher order "stuff" and wondering, "but where is the actual code, that does the actual thing?" And I'm not sure there exists a human on the planet who knows what idiomatic scala is. The whole point of the language was to make some impossible compromises, and as such you can kind of feel forces inside the language in tension with itself, not in alignment. Perhaps FP and OOP are not meant to be unified.

reply

pdpi 4 hours ago [-]

> Perhaps FP and OOP are not meant to be unified.

The only thing that keeps FP and OOP from integrating properly is the "Everything is an Object" mentality. Pure-data structs aren't objects, and you start making loads of weird compromises when you force them into an object-shaped hole. Java (and, I guess, C++) kind of pushes classes as better structs, and lots of bizarre problems come from there.

I find that, when I'm programming in the small, on the lower-level implementation details, FP design principles serve me best. Inversely, when thinking about the structure of my applications, OOP principles suit the problem better.

This sort of design is what leads some people to say that Erlang is the most OOP language out there — Erlang processes map very naturally to objects in this sense, and it is, obviously, a functional language through-and-through.

reply

mafribe 4 hours ago [-]

   The only thing that keeps FP and OOP from 
   integrating properly is the "Everything 
   is an Object" mentality. 

Actually ... in Scala everything is an object.

Functions are special cases of objects. That's Scala's great insight and it reverses what people used to think: everything is a function, and OOP is a special case of FP (this is e.g. OCaml's idea). If you look at Scala's foundations (DOT -- the dependent object calculus) this becomes pretty clear.

Could you be referring to the difference between pure programming and stateful programming? Note that OO can be done in a pure way.

reply

---

cdegroot 5 hours ago [-]

I once drank the Scala cool-aid as well. Even wrote an enthusiastic article about it [1]. Since then, my enthusiasm has been replaced by disappointment: the JVM interop is a lie, as Java libraries often are a bad fit, people won't have time to write the extensive wrapper code needed to hide it, and as a result ugliness starts permeating your codebase - yes, bad, but that's how teams under time pressure work; the multi-paradigm-and-widening (back in 2009, Scala was a smaller language) is one big headache - everyone has an opinion of how you should write code in Scala, so inevitably you spend a lot of time debating features you should or shouldn't use in your teams (instead of writing code. In Elixir, we never have to debate paradigms, we write code); sbt is a sorry mess - no-one understands it and it's slow.

I still think it's a very well designed language, the choice of the JVM+Java ecosystem as the target just has bogged it down with legacy crap like everything else on that environment and the designers should start to make some choices instead of offloading that on every single development team in the world. I think that "single-paradigm" is one of the reason that simple languages that get shit done, like Golang and Elixir, are so successful these days.

Were I bound to the JVM, I'd still give it a good look (not sold on Clojure or Kotlin yet and it _is_ nicer and less frustrating than Maven+Java if you keep it simple); I'm just not married to the JVM so can freely consider other options.

[1] http://www.artima.com/weblogs/viewpost.jsp?thread=260478

reply

---

thepiwo 20 minutes ago [-]

Scala quicky got my favourite language. I really like handling data using functional and imutable concepts.

Great libraries for db access (slick) and json handling (sparay-json) were not mentioned in the article, I wonder why slick is not even mentioned in the comments, it is really nice!

Using scala with these and combined with akka and futures feels like a so natural way of programming for me.

reply

paulddraper 5 hours ago [-]

I wouldn't characterize Scala as a better Java any more than I would Python as a better C.

Kotlin is a better Java.

Groovy is a scriptable Java.

Scala is a "worse Haskell" or an "extensible Swift", with close Java interoperability (that last part leading to the warts).

reply

---

http://man.cat-v.org/plan_9/2/intro : bind open read dup fork seek stat remove fd2path getwd pipe exec chdir segattach exits wait sleep notify lock errstr

bind: mount pipe "creates a connected pair of file descriptors, useful for bidirectional local communication." segattach: attach other memory segments exits: exit/halt/terminate wait: "A parent process may call wait(2) to wait for some child to terminate." notify: "There is a notification mechanism for telling a process about events such as address faults, floating point faults, and messages from other processes. A process uses notify(2) to register the function to be called (the notification handler) when such events occur."

http://man.cat-v.org/plan_9/5/intro : version auth error flush attach walk open create read write clunk remove stat wstat

version auth error flush clunk are protocol metastuff (flush aborts a transaction, clunk deallocates a file descriptor), leaving:

attach: open a server's 'filetree'; like mount, i guess walk: like cd (but speculative/pure function) open create read write remove stat: what you think wstat: change file properties (so stat is read, wstat is write)

my summary (changing some names): mount open close read write dup fd2path seek stat wstat remove fork pipe exec halt wait walk chdir getwd segattach notify sleep lock auth? version?

---

[1] [2] :

[3] : exit read write open close chdir lseek getpid mkdir rmdir dup pipe brk ioctl

[4] "describes some common system calls" but "More complex system calls, like exit which invokes handlers have actual implementations in C or assembly code and will not be found in a templated text file like this." [5]

[6] "13 important library or system calls " : memset fopen getcwd getuid snprintf fwrite fflush fclose system sleep opendir readdir atoi

"On Unix, Unix-like and other POSIX-compliant operating systems, popular system calls are" [7] : open, read, write, close, wait, exec, fork, exit, and kill

" System calls can be roughly grouped into five major categories:

    Process Control
        load
        execute
        end, abort
        create process (for example, fork on Unix-like systems, or NtCreateProcess in the Windows NT Native API)
        terminate process
        get/set process attributes
        wait for time, wait event, signal event
        allocate, free memory
    File management
        create file, delete file
        open, close
        read, write, reposition
        get/set file attributes
    Device Management
        request device, release device
        read, write, reposition
        get/set device attributes
        logically attach or detach devices
    Information Maintenance
        get/set time or date
        get/set system data
        get/set process, file, or device attributes
    Communication
        create, delete communication connection
        send, receive messages
        transfer status information
        attach or detach remote devices

" [8]

[9] "The most common POSIX system calls" :

abort access chdir chmod chown chroot close closerange confstr confstr_names ctermid dup dup2 environ error execv execve fchdir fchmod fchown fdatasync fdopen fork forkpty fpathconf fstat fstatvfs fsync ftruncate getcwd getcwdu getegid geteuid getgid getgroups getloadavg getlogin getpgid getpgrp getpid getppid getresgid getresuid getsid getuid initgroups isatty kill killpg lchown link listdir lseek lstat major makedev minor mkdir mkfifo mknod nice open openpty pathconf pathconf_names pipe popen putenv read readlink remove rename rmdir setegid seteuid setgid setgroups setpgid setpgrp setregid setresgid setresuid setreuid setsid setuid stat stat_float_times stat_result statvfs statvfs_result strerror symlink sysconf sysconf_names system tcgetpgrp tcsetpgrp tempnam times tmpfile tmpnam ttyname umask uname unlink unsetenv urandom utime wait wait3 wait4 waitpid write

[10] : " 40 of the more important system call:

     GENERAL CLASS              SPECIFIC CLASS                 SYSTEM CALL
     ---------------------------------------------------------------------
     File Structure             Creating a Channel             creat()
     Related Calls                                             open()
                                                               close()
                                Input/Output                   read()
                                                               write()
                                Random Access                  lseek()
                                Channel Duplication            dup()
                                Aliasing and Removing          link()
                                Files                          unlink()
                                File Status                    stat()
                                                               fstat()
                                Access Control                 access()
                                                               chmod()
                                                               chown()
                                                               umask()
                                Device Control                 ioctl()
     ---------------------------------------------------------------------
     Process Related            Process Creation and           exec()
     Calls                      Termination                    fork()
                                                               wait()
                                                               exit()
                                Process Owner and Group        getuid()
                                                               geteuid()
                                                               getgid()
                                                               getegid()
                                Process Identity               getpid()
                                                               getppid()
                                Process Control                signal()
                                                               kill()
                                                               alarm()
                                Change Working Directory       chdir()
     ----------------------------------------------------------------------
     Interprocess               Pipelines                      pipe()
     Communication              Messages                       msgget()
                                                               msgsnd()
                                                               msgrcv()
                                                               msgctl()
                                Semaphores                     semget()
                                                               semop()
                                Shared Memory                  shmget()
                                                               shmat()
                                                               shmdt()
     ----------------------------------------------------------------------

more comprehensive categories including some more esoteric syscalls: [11]

list of syscalls: [12] [13]

[14] timeout.stp from Section 4.3.4, “Monitoring Polling Applications” helps you identify which applications are polling by pointing out which ones used the following system calls most frequently: " : poll select epoll itimer futex nanosleep signal

"topsys.stp lists the top 20 system calls used by the system per 5-second interval. It also lists how many times each system call was used during that period.... : gettimeofday read ioctl poll close open select write writev futex recvmsg socket clock_gettime rt_sigprocmask sendto setitimer stat time sigreturn fstat

quick ref card with one-line description of ~190 syscalls: [15]

the lower-numbered ones in that list seem more fundamental. The first 54 are:

exit fork read write open close waitpid creat link unlink execve chdir time mknod chmod lchown stat lseek getpid mount umount setuid getuid stime ptrace alarm fstat pause utime access nice sync kill rename mkdir rmdir dup pipe

some notable ones after that include: sys_signal ioctl ulimit chroot symlink readlink uname poll

pre-summary (copied the lists of syscalls from things above yet within this section):

exit read write open close chdir lseek getpid mkdir rmdir dup pipe brk ioctl memset fopen getcwd getuid snprintf fwrite fflush fclose system sleep opendir readdir atoi open, read, write, close, wait, exec, fork, exit, and kill poll select epoll itimer futex nanosleep signal gettimeofday read ioctl poll close open select write writev futex recvmsg socket clock_gettime rt_sigprocmask sendto setitimer stat time sigreturn fstat exit fork read write open close waitpid create link unlink execve chdir time mknod chmod lchown stat lseek getpid mount umount setuid getuid stime ptrace alarm fstat pause utime access nice sync kill rename mkdir rmdir dup pipe sys_signal ioctl ulimit chroot symlink readlink uname poll files: creat open close read write lseek dup link unlink stat fstat access chmod chown umask ioctl processes: exec fork wait exit getuid geteuid getgid getegid getpid getppid signal kill alarm chdir IPC (pipelines, messages, semaphores, shared memory): pipe msgget msgsnd msgrcv msgctl semget semop shmget shmat shmdt Process Control (conceptual names): load execute create_process terminate_process get_proc_attrs set_proc_attrs wait_for_time wait_event signal_event memory (conceptual names): alloc dealloc File management (conceptual names): create delete open close read write reposition get_attrs set_attrs Device Management (conceptual names): request_device release_device read write reposition get_dev_attrs set_dev_attrs logically_attach logically_detach Information Maintenance (conceptual names): get_time set_time get_system_data set_system_data Communication (conceptual names): create_channel, delete_channel send_msg recv_msg transfer_status_info attach_remove_dev detach_remote_dev

my summary (changing some names): open close read write seek chdir mkdir rmdir link unlink stat rename create dup pipe delete_pipe pipe_info ioctl brk/malloc/mdealloc getpid exit wait exec fork kill get/set_proc_attrs poll signal alarm gettime mount umount chroot ulimit access chmod chown uname/cpuid/proc_ish_info sync write_stat get_time set_time

---

pre-summary of the previous two sections (plan 9 important syscall and popular linux syscalls):

open close read write seek chdir mkdir rmdir link delet stat rename create write_stat getwd dup fd2path pipe delete_pipe pipe_info ioctl brk/malloc/mdealloc segattach getpid halt wait exec fork kill get/set_proc_attrs poll signal alarm notify sleep lock gettime mount umount chroot ulimit access chmod chown uname/cpuid/proc_ish_info/version auth sync get_time set_time walk

my summary of the previous two sections: open close read write walk seek readinfo remove writeinfo remove dup mount poll sleep get_time fork exec halt wait getpid kill pipe signal cpuid/uname/myinfo/version ioctl malloc, mdealloc

(already copied the previous to todo_maybe in boot_reference)

TODO see also those other lists of popular linux syscalls that i made

---

https://docs.microsoft.com/en-us/dotnet/articles/fsharp/language-reference/symbol-and-operator-reference/

---

 ak217 1 day ago | parent | flag | favorite | on: Silent Data Corruption Is Real

crc32c is not weakish, and was chosen for a reason: crc32c has widespread hardware acceleration support that remains faster than any hash, and crc32c can be computed in parallel (unlike a hash, it has no hidden state, so you can sum independently computed block checksums to get the overall blob checksum).

[16]

---

 "
 It's not just that. I also need plugins for basic functionality that RN does not provide it out of the box, like playing audio, accessing the file system, reading sqlite databases, etc.
 "

---

https://github.com/Gabriel439/post-rfc/blob/master/sotu.md

---

"I've written smallish to largish programs in Vue, React, Ember, Polymer - and in all of those cases tried, wherever possible to do so in RXJS and Typescript"

"RxJS? is going to be huge" -- someone else

"CycleJs?. Rx like and TypeScript?"

"Cyclejs currently seems to be the only sane observable/typescript based framework to me."

---

ekimekim 3 days ago [-]

But once you have milliseconds, microseconds are nice to have...

The linux kernel (and many other applications) solve this with a tuple of 64-bit ints (seconds, nanoseconds) where 0 <= nanoseconds <= 999999999. Compare this to simply 64 bits of nanoseconds, which would run out in roughly 2554 CE.

Other systems still (perhaps most commonly) are using double floats for seconds. Under that scheme, nanoseconds were only representable until Feb 17th 1970. The last representable microsecond will be some time in 2106, and the last representable second won't be for another 150 million years or so.

Personally, I'm happy with the precision afforded by floats. Timing uncertainly (outside niche applications) is generally much larger than a single nanosecond, and even microseconds are a bit suspect.

reply

tdhoot 3 days ago [-]

Why not just use int32 for nanoseconds then?

reply

Dylan16807 2 days ago [-]

The nanoseconds field in timespec is actually a long, for some reason. I have no idea why it's not fixed-size.

Let's not even get into timeval which uses the same size field for microseconds.

reply

keithlfrost 3 days ago [-]

On the other hand, using a 64 bit integer to count seconds moves the problem 292 billion years forward, and we can probably let the Omega Point handle it from there.

reply

twic 3 days ago [-]

On the other other hand, getting greedy and using a 64 bit integer to count nanoseconds only moves the problem 448 years forwards. That's long enough to seem like it's perfectly safe, but short enough that it might just not be.

reply

jabl 3 days ago [-]

libuv, which sits underneath node.js, uses 64-bit nanosecond timestamps. And by that time, everything will be javascript.

THE END IS NIGH1111!!!!!

reply

...elsewhere...

Year 2262

The Go programming language has a UnixNano? API that counts nanoseconds since 1970 as a 64-bit signed integer.[17] This value will overflow on 2262-04-11.

---

so it seems like 64-bit ns time from Unix epoch is fairly common, but this overflows in just ~250 years.

this suggests considerating either us (microsecond) time, so we'd have 250k years, but which is annoying because present-day processors have cycle times of about half a ns, or using >64-bit time. 128-bit time gives us about 10^30, which could let us use femtoseconds (10^-15 secs; the fastest chemical reactions take ~15 femtoseconds [17]), and about 30 million years. The age of the universe is thought to be about 14 billions years. So, in 128-bit, we could also give up femtoseconds and have signed picosecond resolution to the beginning of the universe. 1.2 picoseconds is the "switching time of the world's fastest transistor (845 GHz, as of 2006) [18]".

So, we seem to have 4 choices:

i would probably prefer 64-bit us time, but 64-bit ns time seems to be more popular. As "twic" said on HN, "using a 64 bit integer to count nanoseconds only moves the problem 448 years forwards. That's long enough to seem like it's perfectly safe, but short enough that it might just not be.". Are ns really needed? There seem to be other people who think so, and mb they know more than me. To get both ns resolution and a longer addressable duration, we need more than 64 bits, and 128-bits is the next logical jump up. Seems to me that an addressable duration of the whole lifetime of the universe is nice, so 128-bit picosecond resolution seems nice to me; otoh by the time we need that we might somehow have cycle times much faster than a picosecond, in which case femtosecond time might be better.

So, i lean towards either 64-bit microsecond time, or 128-bit picosecond time.

For Boot, i'm guessing 64-bit microsecond datetime is the way to go, signed, with 0 at the Unix epoch? Since Boot is 16-bit, we could also consider 16-bit time, but that's quite limited, as with even second resolution we get less than a day.

OK, having proposed 64-bit signed microsecond datetimes, since this is different from what other people decided on, let's look at the disadvantages.

Getting time is a funny thing. I guess it is an I/O operation, but it's a weird one, because the input is coming not from the "outside", from the user, but rather is fundamentally tied to the implementation itself. This and the considerations above make me think that perhaps platform-dependency is unavoidable, and we should embrace that rather than focusing too much on a single standard time format.

So, i guess what we should actually do, rather than mandating 64-bit microseconds as the one true time format, is to provide a way to ask for a clock with certain properties, and let the implementation decide whether it will satisfy the request. Like the way that Linux gettime passes a clock_id integer with the request. So we don't need to specify that 'microseconds must be available'; perhaps ALL resolutions are 'available', but if the implementation can't actually provide it then it just adds zeros to what it can provide. And, since Boot (and Ovm) allow (but do not require) opaque words with greater-than-16-bit semantics, we can let the implementation decide what time bitwidth it wants to offer, too. We may want to specify a way for a 16-bit clock_id integer to encode all possible clock requests, however. This seems do-able. With just 4 bits, we can choose any resolution from second to yoctosecond (10^-24; three metric prefixes below femtoseconds) (and more). With just 3 more bits, we can choose any bitwidth from 16 to 256 (and more). With just 1 more bit, we can ask for monotonicity or not; with 2 more bits, whether the time is absolute, per-cpu, per-process, or undefined relative time (maybe cycles since OS/platform/Ovm-implementation start). And that's only 10 bits so far (and we might want to leave the other 6 bits or so for implementation-specific stuff).

How to query the system's clock capabilities? It's tempting to have a request for the clock with id 0 represent such a query. But that's a special case, and we want to avoid special cases (they make typechecking etc harder). So we should have some sort of meta 'get system capabilities' thingee that includes clock capabilities in its API. Perhaps clock_id 0 would correspond to asking for 16-bit second-resolution not-necessarily-monotonic relative time, and "SHOULD" be supported whenever possible.

--

http://clqr.boundp.org/

---

brudgers 1 day ago

parent flag favorite on: Lisp Quickstart

Disclaimer: I like Scheme and love Racket but have an ongoing exercise in Common Lisp. [1]

Common Lisp has the advantage of having done a pretty good job at the hard problem of naming things and so ordinary activities have names with regular semantics like

removeand remove-if-- generally some-nametests against a value and some-name-iftests against a predicate. Functions on sequences have keywords as supporting parts of speech that make it possible to state intent rather than write an implementation -- Common Lisp contains several domain specific languages that have proven to be useful.

One way to learn them is to read Steele's, Common Lisp: the language.

It's not that a programmer can't write their own version of

copy-treeand a naive implementation is fairly easy. It's not but nominally more difficult or verbose than writing it in a Scheme or Clojure. The difference is that Common Lisp's built in copy-treeis probably sophisticated for performance and robustness reasons. Schemes come out of a pedagogical context and there not having a built-in copy-treecan be a feature because of what people may learn by its absence. Racket's Student Languages push this toward the limit.

To put it another way, Assembly Code has fewer constructs than Python, but often there is an advantage that comes with Python's higher level abstractions.

[1] Why the hurry: http://norvig.com/21-days.html

-- https://news.ycombinator.com/item?id=13950138

---

asitdhal 12 hours ago [-]

For those who wants to know c++17 features

http://stackoverflow.com/questions/38060436/what-are-the-new...

std::optional and std::any are one of the good things. std::apply and std::invoke can help you write better functional c++17 code.

Anyway thanks for C++17. For sometime, I thought I will learn Rust.

reply

---

"The time package is the bane of tests and testable code....The cure is to mock out time... So everywhere that was using the time package now needs to take your special clock interface that wraps time and then for tests you pass in a fake time that you can control. "

---

" In Juju we used Gustavo Nieyemer’s gocheck to run our tests. Gocheck’s test suite style encouraged full stack testing by reducing the developer overhead for spinning up a full Juju server and mongo database before each test. Once that code was written, as huge as it was, you could just embed that “base suite” in your test suite struct, and it would automatically do all the dirty work for you. This meant that our unit tests took almost 20 minutes to run even on a high end laptop, because they were doing so much for each test. It also made them brittle (because they were running so much code) and hard to understand and debug. To understand why a test was passing or failing, you had to understand all the code that ran before the open brace of your test function, and because it was easy to embed a suite within a suite, there was often a LOT that ran before that open brace.

In the future, I would stick with the standard library for testing instead. I like the fact that test with the standard library are written just like normal go code, and I like how explicit the dependencies have to be. If you want to run code at the beginning of your test, you can just put a method there… but you have to put a method there. "

---

"Multi-OS Mistakes

In the beginning when we were ramping up Windows support, there were a few OS specific bugs (we all developed on Ubuntu, and so Windows bugs often didn’t get caught until CI ran). They basically boiled down to two common mistakes related to filesystems.

The first was assuming forward slashes for paths in tests. So, for example, if you know that a config file should be in the “juju” subfolder and called “config.yml”, then your test might check that the file’s path is folder + “/juju/config.yml” - except that on Windows it would be folder + “\juju\config.yml”.

When making a new path, even in tests, use filepath.Join, not path.Join and definitely not by concatenating strings and slashes. filepath.Join will do the right thing with slashes for the OS. For comparing paths, always use path.ToSlash? to convert a filepath to a canonical string that you can then compare to.

The other common mistake was for linux developers to assume you can delete/move a file while it’s open. This doesn’t work on Windows, because Windows locks the file when it’s open. This often came in the form of a defer file.Delete() call, which would get FIFO’d before the deferred file.Close() call, and thus would try to delete the file while it was still open. Oops. One fix is to just always call file.Close() before doing a move or delete. Note that you can call Close multiple times on a file, so this is safe to do even if you also have a defer file.Close() that’ll fire at the end of the function.

None of these were difficult bugs, and I credit the strong cross platform support of the (((Go))) stdlib for making it so easy to write cross platform code. "

---

giancarlostoro 1 day ago [-]

>> Just over 2 years ago Microsoft open-sourced the entire .NET framework,

Except they open-sourced a new .NET stack, not really the entire .NET Framework, .NET Core and .NET Framework are similar but not fully backwards compatible. I've been porting .NET Framework code to .NET Core and depending on how specialized your project is you may not always find the same libraries supporting your project. A project that perfectly compiles with .NET 2.0 will not just compile for .NET Core, at least not as easily as using a .NET Framework 2.0 project in .NET Framework 4.6. I love Microsoft and .NET but I think Core is executed poorly. You also have EntityFramework? vs. EntityFramework? Core which lacks features from EF which were deemed "unused" which bites some people in the face.

reply

manigandham 1 day ago [-]

.NET Standard/Core 2.0 will add back many APIs. It'll be the real release of the newer framework, similar to how the original .NET Framework took until v2 to get things into shape.

.NET Standard is all about compatibility - so you can use that to determine which framework you can/need to use. However .NET Core is a distinctly designed for cross-platform reach so it will be a break from .NET Framework in some ways - this is just a given, otherwise they would've just released another .NET Framework version instead of doing all this.

reply

---

https://clojure.org/reference/transducers

---

http://articulate-lisp.com/project/abcs.html http://articulate-lisp.com/examples/snippets.html

---

https://github.com/google/gemmlowp "gemmlowp: a small self-contained low-precision GEMM library" "This is not a full linear algebra library, only a GEMM library: it only does general matrix multiplication ("GEMM")."

throwaway71958 1 day ago [-]

Note however, that on Intel it's actually slower than run off the mill float32 linear algebra library like Eigen or OpenBLAS?. Its main forte seems to be ARM.

reply

---

random blog post on implementing GEMM (generalized matrix multiplication): https://petewarden.com/2015/04/20/why-gemm-is-at-the-heart-of-deep-learning/

---

example of using logic-programming to solve the n-queens problem. Embedded into type-level programming in (a GHC-flags-enabled variant of) Haskell:

https://aphyr.com/posts/342-typing-the-technical-interview

starts by defining peano arithmetic, basic boolean functions, map and filter, cons-based lists, any, listconcat, apply (partial function application), conj (conj list x = cons x list).

---

random immutable data structure library:

https://github.com/arximboldi/immer

---

" . Immutable.js is nice, but you have to drop lodash. seamless-immutable silently fails when you try to change things (which then does work in production). "

---

https://github.com/caolan/async/blob/v1.5.2/README.md#filter

---

https://github.com/strml/node-toobusy

---

efficient glob by not backtracking from a subsequent * thru a previous *:

https://research.swtch.com/glob

---

a library for basic binary functions (written in an application-specific scripting language, Papyrus, for game modding)

---

"If you’re bold you can even pick a set of eslint rules to force immutability. And then there’s Ramda. That’s the secret sauce; it makes data manipulation a joy, in some spots I even like its functions better than their Clojure counterparts."

ramda vs lodash: https://gist.github.com/vvgomes/451ea5ca2c65e87c92e4

ramda vs lodash, underscore: http://funkyjavascript.com/i-think-ramda-might-be-better-than-lodash-or-underscore/

ramda vs lodash, underscore, immutable.js: https://www.reddit.com/r/javascript/comments/3537kx/underscore_lodash_ramda/

---

" GNU libstdc++ is broken ... memory leak ... Since "malloc is slow," libstdc++ "helpfully" adds its own memory allocator layer between you and the C library. This one implements its own caching and pooling, and searching around the web yields many examples of people complaining about it.

It turns out that there is a somewhat convoluted way to disable it globally: set the environment variable "GLIBCPP_FORCE_NEW". After doing this, CPU use increased slightly but memory use stabilized. Recalling jemalloc I now once again tried sticking it under the controller in place of glibc's malloc and both CPU load and memory use dropped to substantially less than either stock configuration. More importantly everything became stable once again. "

faragon 10 hours ago [-]

The problem is forgetting that dynamic memory usage is not "free" (as in "gratis" or "cheap"). In fact, using std::string for long-lived server processes doing intensive string processing (e.g. parsing, text processing, etc.) is already known to be suicidal since forever, because of memory fragmentation.

For high load backend processing data, you need at least a soft real-time approach: avoid dynamic memory usage at runtime (use dynamic memory just at process start-up or reconfig, and rely on stack allocation for small stuff, when possible).

I wrote a C library with exactly that purpose [1], in order to work with complex data (strings -UTF8, with many string functions for string processing-, vectors, maps, sets, bit sets) on heap or stack memory, with minimum memory fragmentation and suitable for soft/hard real-time requirements.

[1] https://github.com/faragon/libsrt https://github.com/faragon/libsrt

reply

arunc 14 hours ago [-]

I encountered exactly the same issue few years ago in UIDAI in one of our large scale biometric matchers and the resolution was exactly the same. After a week of debugging I found that the libstdc++ allocator was the culprit. I found [1] and confirmed the same, which helped in fixing this issue.

The thing that was more interesting (or sad) was to know that the GCC developers didn't expect the multithreaded applications to be long running.

"Operating systems will reclaim allocated memory at program termination anyway. "

[1] https://gcc.gnu.org/onlinedocs/libstdc++/manual/mt_allocator... https://gcc.gnu.org/onlinedocs/libstdc++/manual/mt_allocator_impl.html

reply

[27]

--- " Haskell has some very nice libraries

You really have very nice libraries, written by people doing really useful things.

Conduit and Parsec are the basis of a lot of ngless code.

Here is an excellent curated list of Haskell library world (added May 4) " https://github.com/snoyberg/conduit https://hackage.haskell.org/package/parsec https://github.com/Gabriel439/post-rfc/blob/master/sotu.md

---

" How can I convert the str representation of a dict, such as the following string, into a dict?

s = "{'muffin' : 'lolz', 'foo' : 'kitty'}"

I prefer not to use eval. What else can I use? ... Starting in Python 2.6 you can use the built-in ast.literal_eval:

>>> import ast >>> ast.literal_eval("{'muffin' : 'lolz', 'foo' : 'kitty'}") {'muffin': 'lolz', 'foo': 'kitty'}

This is safer than using eval. As its own docs say:

>>> help(ast.literal_eval) Help on function literal_eval in module ast:

literal_eval(node_or_string) Safely evaluate an expression node or a string containing a Python expression. The string or node provided may only consist of the following Python literal structures: strings, numbers, tuples, lists, dicts, booleans, and None.

"

---

threeseed 4 days ago [-]

...Clojure uses SQL's Timestamp (why ?)...

reply

lvh 4 days ago [-]

...Clojure uses it because it's the only sane time instance in the stdlib. It is not an uncommon class to use for that reason; the underlying reason is that j.u.Date sucks...

reply

desdiv 4 days ago [-]

>clojure.inst requires java.sql.Timestamp

Just curious, why haven't they switched to java.time.LocalDateTime?? Java 8 has been out for 3 years now.

reply

loevborg 4 days ago [-]

Clojure has a fantastic track record of not breaking existing code. Changing clojure's `#inst` to a different class would probably entail such a backward-incompatible change....

---

" Floats implementing PartialOrd? but not Ord is maddening. (Or rather, the *_by_key functions operating on Ord rather than PartialOrd? is what’s frustrating.) Yes, this is technically correct, but it’s also hugely inconvenient to not be able to min_by_key on a collection of f32s that are definitely not NaN?. I’ve seen a handful of workarounds for this, but none of them have been my favorite. f32s are incredibly, incredibly common in games, and it’s frustrating that tasks like “find the point in set S that’s closest to point P” are not as straightforward as they could be. "

---

"Lessons Learned from Porting 300 C/C++ Projects to Buck Build

With Buckaroo we are turning the huge ecosystem of C/C++ projects into a collection of easily composable building blocks. We can’t do this alone, but to kickstart the community we took it upon ourselves to port 300 projects to the Buck build system.

The libraries we chose were based on their popularity on GitHub?, StackOverflow? and requests from our mailing list. They ranged from tiny header-only libraries to monolithic C++ projects to old, but critical, C libraries." [28] (May 16 2017). So, what were these ~300 C/C++ libraries that they deemed most important? They appear to be listed in https://github.com/LoopPerfect/buckaroo-recipes/tree/master/recipes . They are:

aantron/better-enums.json agauniyal/rang.json aigamedev/btsk.json andrewrk/libsoundio.json angelcode/angelscript.json arashpartow/wykobi.json ariya/fastlz.json aseprite/pixman.json assimp/assimp.json bartgrantham/qs-parse.json bitcoin/libbase58.json boost/accumulators.json boost/algorithm.json boost/align.json boost/any.json boost/array.json boost/asio.json boost/assert.json boost/assign.json boost/atomic.json boost/bimap.json boost/bind.json boost/chrono.json boost/circular-buffer.json boost/concept-check.json boost/config.json boost/container.json boost/context.json boost/conversion.json boost/core.json boost/coroutine2.json boost/coroutine.json boost/crc.json boost/date-time.json boost/detail.json boost/endian.json boost/exception.json boost/fiber.json boost/filesystem.json boost/foreach.json boost/format.json boost/functional.json boost/function.json boost/function-types.json boost/fusion.json boost/graph.json boost/hana.json boost/heap.json boost/integer.json boost/interprocess.json boost/intrusive.json boost/io.json boost/iostreams.json boost/iterator.json boost/lambda.json boost/lexical-cast.json boost/locale.json boost/lockfree.json boost/logic.json boost/math.json boost/move.json boost/mpl.json boost/multi-array.json boost/multi-index.json boost/multiprecision.json boost/numeric-conversion.json boost/optional.json boost/parameter.json boost/phoenix.json boost/pool.json boost/predef.json boost/preprocessor.json boost/process.json boost/program-options.json boost/property-map.json boost/property-tree.json boost/proto.json boost/ptr-container.json boost/python-headers.json boost/random.json boost/range.json boost/ratio.json boost/rational.json boost/regex.json boost/serialization.json boost/signals2.json boost/signals.json boost/smart-ptr.json boost/spirit.json boost/static-assert.json boost/system.json boost/thread.json boost/throw-exception.json boost/timer.json boost/tokenizer.json boost/tti.json boost/tuple.json boost/type-index.json boost/typeof.json boost/type-traits.json boost/ublas.json boost/unordered.json boost/utility.json boost/uuid.json boost/variant.json boost/wave.json boost/winapi.json boost/xpressive.json bulletphysics/bullet3.json cavewhere/squish.json celtoys/remotery.json cesanta/frozen.json cgal/cgal.json cgal/installation.json cgal/profiling-tools.json cgal/stl-extension.json chaiscript/chaiscript.json chriskohlhoff/asio.json chriskohlhoff/urdl.json chrismanning/jbson.json corvusoft/kashmir.json cuberite/polarssl.json cuberite/sqlitecpp.json danbloomberg/leptonica.json datasift/served.json ddinu/observable.json dlecocq/apathy.json docopt/docopt.cpp.json dropbox/json11.json eggs-cpp/variant.json eigen/eigen.json eile/tclap.json eranpeer/fakeit.json erdc-cm/szip.json ericniebler/range-v3.json facebook/fatal.json facebook/yoga.json fidoproject/fido.json fmtlib/fmt.json freetype/freetype2.json fukuchi/libqrencode.json fuzzylite/fuzzylite.json github/cmark.json glennrp/libpng.json glfw/glfw.json google/benchmark.json google/boringssl.json google/brotli.json google/cctz.json google/compact-enc-det.json google/double-conversion.json google/draco.json google/flatbuffers.json google/fplutil.json google/fruit.json google/gflags.json google/glog.json google/gmock.json google/gtest.json google/guetzli.json google/leveldb.json google/mathfu.json google/motive.json google/re2.json google/snappy.json gpakosz/whereami.json graphql/libgraphqlparser.json howardhinnant/date.json icu/icu4c.json ijg/jpeg.json ikalnytskyi/termcolor.json ipkn/crow.json jarikomppa/soloud.json jbeder/yaml-cpp.json jech/dht.json jedisct1/libsodium.json jgarzik/univalue.json jkriege2/tinytiff.json jkuhlmann/gainput.json juce/juce-audio-basics.json juce/juce-audio-devices.json juce/juce-audio-formats.json juce/juce-audio-processors.json juce/juce-audio-utils.json juce/juce-box2d.json juce/juce-core.json juce/juce-cryptography.json juce/juce-data-structures.json juce/juce-events.json juce/juce-graphics.json juce/juce-gui-basics.json juce/juce-gui-extra.json juce/juce-opengl.json juce/juce-osc.json juce/juce-tracktion-marketplace.json juce/juce-video.json kcat/openal-soft.json khovratovich/equihash.json khronosgroup/egl.json kinetiknz/cubeb.json lballabio/quantlib.json leethomason/tinyxml2.json lemenkov/libyuv.json libfann/fann.json libusb/libusb.json libuv/libuv.json littlstar/trim.json lmdb/lmdb.json loopperfect/neither.json lsalzman/enet.json luadist/libiconv.json luadist/libnoise.json luadist/libpng.json luadist/libtiff.json luadist/libzzip.json luadist/toluapp.json luadist/zlib.json lua/lua.json madler/zlib.json miloyip/rapidjson.json mobius3/tweeny.json mohaps/tinysha1.json munificent/wren.json njlr/libdb.json njlr/smallsha1.json njlr/tweetnacl.json njlr/utfcpp.json nlohmann/json.json no1msd/mstch.json nothings/stb.json nsf/termbox.json nyorain/nytl.json ocornut/imgui.json oktal/pistache.json openlink/iodbc.json open-source-parsers/jsoncpp.json openssl/openssl.json philsquared/catch.json pocoproject/cpp-parser.json pocoproject/cpp-unit.json pocoproject/crypto.json pocoproject/data.json pocoproject/foundation.json pocoproject/json.json pocoproject/mongodb.json pocoproject/net.json pocoproject/netssl-openssl.json pocoproject/seven-zip.json pocoproject/util.json pocoproject/xml.json pocoproject/zip.json preshing/junction.json preshing/turf.json python/cpython.json randombit/botan.json rbock/sqlpp11.json reactive-extensions/rxcpp.json rsms/immutable-cpp.json sandstorm/capnproto.json scoopr/vectorial.json sergiusthebest/plog.json sfml/audio.json sfml/graphics.json sfml/network.json sfml/system.json sfml/window.json sparsehash/sparsehash.json stockfish/stockfish.json superv1234/ecst.json superv1234/vrm-core.json superv1234/vrm-pp.json s-yata/marisa-trie.json synesis/b64.json taocpp/operators.json taocpp/pegtl.json taocpp/sequences.json tencent/libco.json tessil/hopscotch-map.json tiny-dnn/tiny-dnn.json torch/th.json tplgy/cppcodec.json unittest-cpp/unittest-cpp.json vinniefalco/beast.json webmproject/libwebp.json weidai11/cryptopp.json xiph/flac.json xiph/ogg.json xiph/vorbis.json zaphoyd/websocketpp.json zeromq/cppzmq.json zeromq/czmq.json zeromq/libzmq.json zeromq/zmqpp.json zeromq/zyre.json zeux/pugixml.json zintinio/happyhttp.json zserge/expr.json zserge/jsmn.json

---

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects

---

https://js.libhunt.com/project/nvd3/vs/c3

---

https://developers.google.com/speed/libraries/

---

dplyr

https://cran.r-project.org/web/packages/dplyr/vignettes/dplyr.html

---

http://www.evanmiller.org/statistical-shortcomings-in-standard-math-libraries.html lists 8 functions that are crucial for stats but not in the std C library:

" Where To Go, and How To Get There

I realize that implementing all of these specialized numerical functions sounds like a lot of work, but in fact the work is essentially finished: the free Cephes library contains a host of mathematical functions with names and argument orders nearly identical to their <math.h> counterparts, as well as all of the functions I have mentioned above. So my proposal boils down to “graduating” the following eight double-precision functions, along with their single-precision and extended precision counterparts, from Cephes into the standard C library:

/* Regularized incomplete beta function */ double incbet(double a, double b, double x);

((note: the original post at http://www.evanmiller.org/statistical-shortcomings-in-standard-math-libraries.html has hyperlinks to these functions)

/* Inverse of incomplete beta integral */ double incbi(double a, double b, double y);

/* Regularized incomplete gamma integral */ double igam(double a, double x);

/* Complemented incomplete gamma integral */ double igamc(double a, double x);

/* Inverse of complemented incomplete gamma integral double igami(double a, double p);

/* Normal distribution function */ double ndtr(double x);

/* Inverse of Normal distribution function */ double ndtri(double y);

/* Bessel function of non-integer order */ double jv(double v, double x);

The Cephes folks seem to be stingy when it comes to doling out letters in function names, so the C committee may want to add a few characters to the above for clarity. But other than that minor taxonomic concern, these functions are essentially ready to go.

So if you know someone who is involved in maintaining a standard math library of any kind, I encourage you to send them the “Elusive Eight” functions above and ask whether they’re doing anything to implement them. Without the functions, regular programmers can’t begin to use statistics; and without statistics, they can’t distinguish chance from patterns, either for themselves on behalf of their users.

" [29]

---

" Go has also eliminated the ternary operator, and, for reasons that appear to be political, does not have integer Min and integer Max functions. From what I can gather on the mailing list threads, the language designers are against polymorphism, as well as adding letters to function names, so unlike the standard C library which operates on float, double, and long double, as well as int and long where appropriate (e.g. absolute value), the Go standard math library operates only on float64. Since there’s no implicit casting to floats, this is rather annoying if you’re using integers, such as when you are counting things. It also makes Go somewhat less useful for heavy number-crunching where you might want single- or extended-precision versions of floating-point functions. "

---

" (Incidentally, the only language I know that gets polymorphism right for dealing with multiple kinds of mathematical objects is Julia — though last time I checked it was still lacking long double / float80 support.) "

---

http://pbpython.com/effective-matplotlib.html

https://speakerdeck.com/jakevdp/pythons-visualization-landscape-pycon-2017

https://startupsventurecapital.com/essential-cheat-sheets-for-machine-learning-and-deep-learning-researchers-efb6a8ebd2e5

---