proj-oot-ootNotes26

http://editorconfig.org/

---

" There's a huge productivity boost in just the fact that your development process is as follows: boot REPL, run your program and then modify your program (while running). If anything is to break, or hasn't been implemented, the debugger just stops your program, let's you inspect your program state freely and you can update the code that failed and have another go.

I'm normally very much to the strong, static typing side of languages. But to me, the Common Lisp experience is dynamic done right, it gives all the tools to handle the problems that arise from a dynamically typed language (even though Common Lisp is actually partially strongly typed) with is a great boon to productivity.

reply

TeMPOraL? 15 hours ago [-]

You're definitely not the only one. I'm in the camp too. The language isn't perfect, but it's so much better than the popular dynamic ones that it's a shame it's still relatively obscure. And the development model of Common Lisp is what I strongly miss everywhere else.

reply

graphene 11 hours ago [-]

Honest question, how is this different from what you can do in e.g. python? The python interpreter supports reloading of modules and evaluation of expressions, is there functionality that CL has above and beyond this that makes it more powerful?

reply

TeMPOraL? 10 hours ago [-]

Can you recompile a function on a running program and have it pick up the new definition? Can you change a class at runtime, and then adjust already existing objects to match the new definition in a controlled way, without data loss? Can you forget to define a function, or initialize a variable, only to be allowed by the debugger to provide a new definition / value on the fly and continue execution?

Those are some examples off the top of my head, but the general point is that unlike mainstream languages, Lisp languages evolved into environment for interactive construction and modification on a continuously running program, which is a bit different way of working than the regular write-compile-run cycle.

reply

aerique 4 hours ago [-]

Some, most?, of the things you mention are available in other languages but they generally do not come together so well and they feel clunky.

I think it's not so much whether the features exist in other languages since we all know they can be added but the whole being more than the sum of its parts.

reply

Arelius 4 hours ago [-]

With Javascript as an example, it can indeed be used in this fashion, as I expect also Python. But the key difference is the community of CL is this is a standard way of working, and the tools are built around this.

"

---

a friend of mine stated that two great things about R (compared to eg Python) are:

---

" swirl and the little data scientist's predicament Jeff Leek 2014/08/13

Editor's note: This is a repost of "R and the little data scientist's predicament". A brief idea for an update is presented at the end in italics.

I just read this fascinating post on _why, apparently a bit of a cult hero among enthusiasts of the Ruby programming language. One of the most interesting bits was The Little Coder’s Predicament, which boiled down essentially says that computer programming languages have grown too complex - so children/newbies can’t get the instant gratification when they start programming. He suggested a simplified “gateway language” that would get kids fired up about programming, because with a simple line of code or two they could make the computer do things like play some music or make a video.

I feel like there is a similar ramp up with data scientists. To be able to do anything cool/inspiring with data you need to know (a) a little statistics, (b) a little bit about a programming language, and (c) quite a bit about syntax.

Wouldn’t it be cool if there was an R package that solved the little data scientist’s predicament? The package would have to have at least some of these properties:

    It would have to be easy to load data sets, one line of not complicated code. You could write an interface for RCurl/read.table/download.file for a defined set of APIs/data sets so the command would be something like: load(“education-data”) and it would load a bunch of data on education. It would handle all the messiness of scraping the web, formatting data, etc. in the background.
    It would have to have a lot of really easy visualization functions. Right now, if you want to make pretty plots with ggplot(), plot(), etc. in R, you need to know all the syntax for pch, cex, col, etc. The plotting function should handle all this behind the scenes and make super pretty pictures.
    It would be awesome if the functions would include some sort of dynamic graphics (withsvgAnnotation or a wrapper for D3.js). Again, the syntax would have to be really accessible/not too much to learn.

That alone would be a huge start. In just 2 lines kids could load and visualize cool data in a pretty way they could show their parents/friends.

Update: Now that Nick and co. have created swirl the technology is absolutely in place to have people do something awesome quickly. You could imagine taking the airplane data and immediately having them make a plot of all the flights using ggplot. Or any number of awesome government data sets and going straight to ggvis. Solving this problem is now no longer technically a challenge, it is just a matter of someone coming up with an amazing swirl module that immediately sucks students in. This would be a really awesome project for a grad student or even an undergrad with an interest in teaching. If you do do it, you should absolutely send it our way and we'll advertise the heck out of it! "

---

yen223 2 days ago [-]

I'm working with a large, 5 year old Django codebase. My humble opinion: Django itself is actively hindering our ability to deliver new features.

Some pain points:

There are more, but one thing's for sure, I probably won't be using Django for my next project.

reply

ibejoeb 2 days ago [-]

I'm a heavy Django user, but I work on a lot of data-intensive projects. I do my data modeling in the database and expose it to Django, and it feels pretty clean to me. I use the ORM for what I would consider its true purpose: converting relations and attributes to objects and properties. I use it very lightly, otherwise, like for filtering and sorting.

If you're using a good database, you accommodate writes through the views as well. I rarely find a good reason to do this, but when it comes up, it works great. It is especially useful when dealing with legacy schemas.

reply

wolf550e 2 days ago [-]

It won't help you with handling code size so big you really want types, but one advice I would give is ditch class based views. I find that function views much easier to reason about, even if it leads to code duplication. The ability to read the code linearly without jumping through inheritance hierarchies makes everything better.

reply

brianwawok 2 days ago [-]

Me too. I never got CBV. I usually use more than one model in a view.

reply

---

mixmastamyk 2 days ago [-]

Used both and off the bat found Flask breezy and Django bureaucratic. But then the next day with Flask you look up how to do something elementary like login (for fsck sakes) and find there's alternatives of various qualities and scopes, and the flask-security guy has disappeared. Found myself spending a full week up front wrangling tokens and other crap instead of the project at hand, sigh. Django starts to shine at that point.

Lately I've been yearning for a "Flango" distribution with flask and the top twenty extensions supported by a single dev team.

reply

nZac 2 days ago [-]

Checkout Keg (https://github.com/level12/keg)

It could use some documentation and some community TLC, but Level 12 is a solid Python shop with a pragmatic approach to app development. There are libraries for auth, login, webgrids, SQLAlchemy, forms, etc etc.

Disclaimer: I used to work for Level 12 and wrote a large production app atop keg.

reply

bradmwalker 1 day ago [-]

https://websauna.org ?

reply

---

jedberg 2 days ago [-]

I feel like Django has too much magic. I agree that it is easier to get quick wins with Django, and if you're building a small hobby site or something, then it's great.

But if you're building something that you hope you'll have to rapidly scale, Django is going to hurt you. It's way harder to scale due to both its heavy reliance on hidden magic its tight integration with its data store.

When you want to rapidly scale, the easiest way to do that is if your data store and application aren't so tightly intertwined that you have to scale both to solve a bottleneck in either.

reply

coldtea 2 days ago [-]

>But if you're building something that you hope you'll have to rapidly scale, Django is going to hurt you. It's way harder to scale due to both its heavy reliance on hidden magic its tight integration with its data store.

Considering that Django has 1/5th as much "magic" as Rails, and that Rails powers some of the biggest websites on the world, I'd say "citation needed".

reply

jrs95 2 days ago [-]

And Django powers some huge sites as well. IMO the magic doesn't really hurt you there as long as your still working with a relatively monolithic piece of software.

reply

a3n 2 days ago [-]

What is "magic,' and how do you measure it so that you can say things like "1/5?"

reply

coldtea 2 days ago [-]

Ever seen Penn and Teller?

Joking aside, generally we call "magic" uses of reflection, monkey-patching, auto-magically configured parts, parts of a framework that do way more than what we explicitly tell them, etc.

"1/5 less magic" is what we call in casual conversation "making a point" and not an actual measurement. It is however more than 85% factually correct.

The Django project leaders have stated that one of their goals was to not have "magic" in the framework, and in fact, one of the early (~ Django 1.2 or so) refactoring efforts was about removing "magic" too-clever parts for more explicitness.

reply

---

znpy 21 hours ago [-]

"Tools, not policies".

I must say, this page reminds me of how lightyears ahead Perl was/is in terms of tooling.

For example: are you coding a package? Before even starting brainstorming, just boostrap your package with ExtUtils::MakeMaker?. Long story short, that package will boostrap a mostly blank (but customizable via various command-line switches) Perl package with most things you will need for development and distribution: a directory structure, a Makefile (for building and installing), some pre-set comments, mostly blank test files etc etc.

This is something that, for example, the Go programming language team has rightfully picked up in terms of directory structure conventions and tooling (gofmt).

reply

grenoire 17 hours ago [-]

JS ecosystem tries to do that by having bootstrapping repositories.

Although I will have to say that it is nowhere near as high-quality and convenient as Go has it. Standardising best practices most often seem too restrictive to programmers, but Go has done an excellent job at drawing that line and staying behind it.

reply

---

ganonm 21 hours ago [-]

This is misleading advice. Most linters can be configured to fail upon detecting a 'TODO'.

I make liberal use of TODOs when developing, typically for reminding myself to e.g. localize a string I've hardcoded during prototyping or to refactor kludgy code.

A pre commit git hook gets triggered to run the linter when I attempt to commit which helps avoid the possibility of adding code containing 'TODOs'. Obviously I can just add the '-n' flag to not trigger hooks, but this requires intention.

In short, using TODOs appropriately is a massive boost to productivity and the alternative is either to just do those random tasks that pop into your head during the course of writing code or to just hope that you remember to do it later.

reply

---

" SQL injection is almost never fixed by using an escaping function, as by far the most common problem with SQL injection is letting the client specify field and column names, or things that aren't quoted (e.g. numerical id's) and so don't benefit from escaping. " [1]

--

rurban 8 days ago [-]

Well, amongst Lispers we do argument a lot about the missing or overarchitectured bits. The ffi, threads, the conditions system, the MOP, ... But only the various FFI's pose a minor problem. It's complaining on a very high level.

reply

daly 2 days ago [-]

Probably the most difficult area in the CL standard is handling pathnames. There are so many possible pitfalls. FFI is an area that, despite best efforts, is likely not a good candidate for inclusion in a standard. It would be better to have all of the implementations converge on a convention since the underlying foreign systems will change over time. Not everything that "just works" has to be in a standard.

---

> Why not Python as one target host language? Lists and hashmaps (dictionaries) are pretty prevalent there and I believe continuously developed Lisp dialect is still missing.

I've spent time on this! It's a difficult problem. In Lua and JS, you can attach keys and values to arrays. In other words, an array can also be a hashmap. Not so in Python -- or rather, not with the standard Python dict() type. And then you run into problems of interoperability and performance: if it's a non-standard type, other libraries run into problems consuming it. This is less true of Python than other languages, though, so I've been trying to find a happy medium.

There are ways of using solely arrays, but it involves having some sort of Keyword type. A normal lisp has those, and typically they use '(a b :x 42) to create an "array" where the first two elements are "a" and "b" and the key "x" has value 42. You can do something similar for Python, and it's an active area of research for me.

jarpineh 10 days ago [-]

>> What about host language access? I'd rather not give up using existing JS and Lua libraries.

>Easy:

  $ npm i leftpad
  $ LUMEN_HOST=node lumen
  > (require 'leftpad)
  function
  > ((require 'leftpad) "foo" 10)
  "0000000foo"

Huh. It just worked. With rlwrap this feels (at least at the very first try) regular REPL!

---

" Example: Alias declarations

My second example is support for alias declarations in Go.

Over the past few years, Google has established a team focused on large-scale code changes, meaning API migration and bug fixes applied across our codebase of millions of source files and billions of lines of code written in C++, Go, Java, Python, and other languages. One thing I've learned from that team's work is the importance, when changing an API from using one name to another, of being able to update client code in multiple steps, not all at once. To do this, it must be possible to write a declaration forwarding uses of the old name to the new name. C++ has #define, typedef, and using declarations to enable this forwarding, but Go has nothing.

"

---

thomasfoster96 3 days ago [-]

Proposals [0] that made it into ES8 (“what’s new”):

The first five have been available via Babel and/or polyfills for ~18 months or so, so they’ve been used for a while now.

[0] https://github.com/tc39/proposals/blob/master/finished-propo...

reply

---

HugoDaniel? 3 days ago [-]

I would really love to see an object map function. I know it is easy to implement, but since they seem to be gaining ranks through syntax sugar, why not just have a obj.map( (prop, value) => ... ) ? :)

reply

thomasfoster96 3 days ago [-]

Is Object.entries(obj).map((prop, value) => ...) close enough? Object.entries is newly standard in ES8.

reply

HugoDaniel? 3 days ago [-]

Not really object map. Does not return a new object ;)

reply

thomasfoster96 3 days ago [-]

Ah, well this would return a new object:

    Object.entries(obj).map((prop, value) => ...).reduce((newOdj, [prop, value]) => newObj[prop] = value);

I see your point about verbosity :)

reply

---

gpu.js Compiles a subset of JavaScript? to GPU code.

" gpu.js relies on the assumption that the kernel function is using only a subset of legal JavaScript? syntax:

    1D, 2D, 3D array of numbers or just numbers as kernel input
    1D, 2D, 3D array of numbers as kernel output
    Number variables
    Arithematic operations (+, -, *, /, %)
    Javascript Math operations (Math.floor() and etc.)
    for loops (of fixed sizes only!)
    if and else statements
    No variables captured by a closure" -- http://gpu.rocks/

---

https://www.kingoftheether.com/contract-safety-checklist.html

---

" Viper (maintained by Vitalik Buterin) are a promising step in this direction. Viper includes by default basic security mechanisms, such as bounded looping constructs, no integer overflows, and prevents other basic bugs that developers shouldn’t have to reason about."

---

mbrock 17 hours ago [-]

Yoichi Hirai and others are working on completing proof assistant support for EVM, meaning you will be able to prove program properties using logical pre/post-conditions with machine checked proofs.

There are already useful tools for symbolic execution, and I have a hunch that this kind of verification will be important along with model checking.

reply

HaseebQ? 18 hours ago [-]

See: https://www.youtube.com/watch?v=voyespPGQZI (Directions in Smart Contract Research A Selection - Philip Daian)

reply

---

" Default-private is a safe, sane default.

And there is also a positive lesson for language design. These kinds of bugs would be greatly reduced if the default policy in Solidity were "default-private." Almost every other language uses default-private, forcing a smart contract programmer to take extra steps to expose a procedure to outsiders. The tradeoff in Solidity is, frankly, wrong. Default-public is too error prone. ...

There are other features that one could add to Solidity.

Chief among the new features to be added to Solidity, besides default-private, are new types for fixed-point arithmetic, a common requirement for financial software. As we saw with Bancor, smart contracts rolling their own arithmetic is not a good proposition. Second on the list would be overflow-protected arithmetic types. That would address the features that everyone currently implements with SafeMath?. Finally, I'd suggest better type inference. This post provides a good list of gripes to start out with. All of these new features and types would have little to no impact on existing contracts.

...

has anticipated and built in provisions, such as pre- and post-conditions, that are useful for maintaining invariants, a critical feature for smart contracts. " [2]

here's the SafeMath? stuff:

https://github.com/nemequ/portable-snippets/blob/master/safe-math/README.md

---

 ARaybold • 14 hours ago

With regard to your observation "as the database community painstakingly discovered, most programmers cannot handle power": one aspect of power that may be harmful in this sort of application is polymorphism. I have spent more time than I would have liked trying to figure out what poorly designed and undocumented code is doing, and it is made much harder when you have to figure out which of potentially many candidates is going to respond to a method call, especially if that decision is made at runtime (function pointers and other uses of functions as first-class entities can create the same problem.) One aspect of this is that it allows one piece of code to handle many cases, but now those cases are implicit, rather than explicitly enumerated in the code. Maybe Pascal or even Fortran would be a better model for a smart contract language than Javascript. (IIRC, didn't Vitalik start with a Pascal-like language?) Or Dafny...

• Reply • Share ›

---

"

peoplewindow 19 days ago

parent favorite on: Underhanded Solidity Coding Contest

Solidity has far worse problems than not being an advanced research language. Just being a sanely designed normal language would be a big step up. Solidity is so riddled with bizarre design errors it makes PHP 4 look like a work of genius.

A small sampling of the issues:

Everything is 256 bits wide, including the "byte" type. This means that whilst byte[] is valid syntax, it will take up 32x more space than you expect. Storage space is extremely limited in Solidity programs. You should use "bytes" instead which is an actual byte array. The native 256-bit wide primitive type is called "bytes32" but the actual 8-bit wide byte type is called "int8".

Strings. What can we say about this. There is a string type. It is useless. There is no support for string manipulation at all. String concatenation must be done by hand after casting to a byte array. Basics like indexOf() must also be written by hand or implementations copied into your program. To even learn the length of a string you must cast it to a byte array, but see above. In some versions of the Solidity compiler passing an empty string to a function would cause all arguments after that string to be silently corrupted.

There is no garbage collector. Dead allocations are never reclaimed, despite the scarcity of available memory space. There is also no manual memory management.

Solidity looks superficially like an object oriented language. There is a "this" keyword. However there are actually security-critical differences between "this.setX()" and "setX()" that can cause wrong results: https://github.com/ethereum/solidity/issues/583

Numbers. Despite being intended for financial applications like insurance, floating point is not supported. Integer operations can overflow, despite the underlying operation being interpreted and not implemented in hardware. There is no way to do overflow-checked operations: you need constructs like "require((balanceOf[_to] + _value) >= balanceOf[_to]);"

You can return statically sized arrays from functions, but not variably sized arrays.

For loops are completely broken. Solidity is meant to look like JavaScript? but the literal 0 type-infers to byte, not int. Therefore "for (var i = 0; i < a.length; i ++) { a[i] = i; }" will enter an infinite loop if a[] is longer than 255 elements, because it will wrap around back to zero. This is despite the underlying VM using 256 bits to store this byte. You are just supposed to know this and write "uint" instead of "var".

Arrays. Array access syntax looks like C or Java, but array declaration syntax is written backwards: int8[][5] creates 5 dynamic arrays of bytes. Dynamically sized arrays work, in theory, but you cannot create multi-dimensional dynamic arrays. Because "string" is a byte array, that means "string[]" does not work.

The compiler is riddled with mis-compilation bugs, many of them security critical. The documentation helpfully includes a list of these bugs .... in JSON. The actual contents of the JSON is of course just strings meant to be read by humans. Here are some summaries of miscompile bugs:

In some situations, the optimizer replaces certain numbers in the code with routines that compute different numbers

Types shorter than 32 bytes are packed together into the same 32 byte storage slot, but storage writes always write 32 bytes. For some types, the higher order bytes were not cleaned properly, which made it sometimes possible to overwrite a variable in storage when writing to another one.

Dynamic allocation of an empty memory array caused an infinite loop and thus an exception

Access to array elements for arrays of types with less than 32 bytes did not correctly clean the higher order bits, causing corruption in other array elements.

As you can see the decision to build a virtual machine with that is natively 256-bit wide led to a huge number of bugs whereby reads or writes randomly corrupt memory.

Solidity/EVM is by far the worst programming environment I have ever encountered. It would be impossible to write even toy programs correctly in this language, yet it is literally called "Solidity" and used to program a financial system that manages hundreds of millions of dollars. " [3]

"

int_19h 3 hours ago

parent flag favorite on: 153k Ether Stolen in Parity Multi-Sig Attack

Just skimming through the Solidity docs, I see a lot of unwise decisions there aside from the weird visibility defaults.

All state is mutable by default (this includes struct fields, array elements, and locals). Functions can mutate state by default. Both are overridable by explicit specifiers, much like C++ "const", but you have to remember to do so. Even then, the current implementation doesn't enforce this for functions.

Integers are fixed-size and wrap around, so it's possible to have overflow and underflow bugs. Granted, with 256 bits of precision by default that's harder to do than usual... but still pretty easy if you e.g. do arithmetic on two inputs.

Operators have different semantics depending on whether the operands are literals or not. For example, 1/2 is 0.5, but x/y for x==1 and y==2 is 0. Precision of the operation is also determined in this manner - literals are arbitrary-precision, other values are constrained by their types.

Copy is by reference or by value depending on where the operands are stored. This is implicit - the operation looks exactly the same in code, so unless you look at declarations, you don't know what it actually does. Because mutability is pervasive, this can can have far-reaching effects.

Map data type doesn't throw on non-existing keys, it just returns the default value.

The language has suffixes for literals to denote various units (e.g. "10 seconds" or "1000 ether"). This is purely syntactic sugar, however, and is not reflected in the type system in any way, so "10 second + 1000 ether" is valid code.

Statements allow, but do not require, braces around bodies. This means that dangling "else" is potentially an issue, as is anything else from the same class of bugs (such as the infamous Apple "goto fail" bug).

Functions can be called recursively with no special effort, but the stack size is rather limited, and it looks like there are no tail calls. So there's the whole class of bugs where recursion depth is defined by contract inputs.

Order of evaluation is not defined for expressions. This in a language that has value-returning mutating operators like ++!

Scoping rules are inherited from JS, meaning that you can declare variables inside blocks, but their scope is always the enclosing function. This is more of an annoyance than a real problem, because they don't have closures, which is where JS makes it very easy to shoot yourself in the foot with this approach to scoping. "

[4]

---

Animats 19 days ago [-]

See Wikipedia.[5] Decision tables allow most of the things you really need in a contract. Termination is guaranteed, because there are no loops. Processing is simple; the evaluator goes down the table rows until one evaluates to true. There's a simple tabular way to look at a decision table, so ordinary humans can read them.

Actions should have database-like transactional properties - either everything in the action happens, or none of it does. If you do a send and an update in an action, both must happen, or neither does.

The big question is what primitives you're allowed to call from the table. They'll look a lot like the ones for Solidity, but need to support atomic transactions.

---

pmahoney 1 hour ago [-]

There are two distinct features: immutability and single-assignment. Erlang is famous for single-assignment, and also happens to have largely immutable values, but they are not the same thing.

Immutability prevents things like in-place appending to an array, or in-place modification of a string.

Single-assigment means that the value bound to "someVariable" cannot be changed. E.g. `someVariable = new String("hello"); someVariable = new String("goodbye");` is illegal. But it still may be possible to mutate the value `someVariable.substitute("hello", "goodbye")` if the language allows mutation.

reply

---

" I'm pretty sure that MOST research on program verification has been done in the context of Turing complete systems, from the work of Floyd and Hoare, to the symbolic execution of Deutsch and King, to temporal logic, TLA+, etc. "

---

" use v6.d.PREVIEW;

    my $channel = Channel.new;
    my @ten_tasks = (^10).map: {
        start {
            my $thread = $*THREAD.id;
            await $channel;
            say "HOLY MOLEY SOMEONE STOLE MY THREAD" if $thread != $*THREAD.id;
        }
    }
    $channel.send("Ring ring") for ^10;
    $channel.close;
    await @ten_tasks;

Guess what it outputs?

Nothing! Just kidding, you’ll see a lot of distressed messages from tasks that passed out wait for the phone to ring, and woke up wearing someone else’s OS thread.1

"

---

"Coroutines and fibers are not m:n threads. There are important implementation and usability distinctions."

---

Every major language, or at least the ones that matter for backend development, has support for thread multiplexing / coroutines / fibers, whatever. Perhaps not in the core language syntax or standard library. But it's easy to implement with native code, and so SOMEONE has implemented it in a library if the language has an FFI.

Java, and all of the other JVM-based languages in turn, have Quasar (http://docs.paralleluniverse.co/quasar/).

Ruby has support for primitive fibers baked into the standard language (https://ruby-doc.org/core-2.1.1/Fiber.html), and likewise community gems with more robust functionality like Quasar.

Python 3 likewise has this out of the box (https://www.python.org/dev/peps/pep-0492/).

The list goes on and on: https://en.wikipedia.org/wiki/Coroutine#Programming_language....

RX14 5 hours ago [-]

Unfortunately, unless the IO in the stdlib uses or is modified to schedule these fibers around evented IO, they're essentially useless as soon as you use an external library. Go and Erlang do this, Crystal does this but with only N:1 multiplexing (but that will change before 1.0), but I don't know about other languages.

reply

anonacct37 35 minutes ago [-]

Very true. Node does this too, in the sense that all node libraries are async/callback aware or have async/callback versions of those.

I much prefer the go/erlang version of this. Playing the "what color is your function" game or "will it block" is no fun.

reply

---

for private methods/fields/encapsulation, yet still allowing debugging, could have an 'access privilages' system similar or identical to the 'spheres' of impurity/effects; eg 'debug' would be like 'superuser', having the ability to snoop on/alter everyone's 'private' methods and fields, the ability to do logging and caching and everything else from within impure functions, etc

---

aylmao 1 hour ago [-]

I learned to program in ActionScript? 2 on Macromedia Flash MX back in high-school. In spite of all the (deserved) hate Flash gets, we got to give it credit too.

It's thus no wonder it caught on like wildfire and there was so much content for it. It was a good option for technical projects and creative ones, beginners and experts. I definitely don't want to see Flash making a comeback on web, but I wouldn't mind seeing it in standalone applications (assuming security doesn't become an issue), and I could see its value on education, granted, with the right editors and tools.

reply

---

" Go code will look at least a little familiar to C programmers. It carries over C’s primitive types, as well as its semantics with regard to values and pointers. From the start I had a decent mental model of how things are passed to functions in Go, and under what circumstances the caller can expect data to change. Like C, Go shuns classes in favor of plain structs, but then lets you make code more object-oriented with interfaces, which I’ll discuss in a minute.

"

---

" Another feature where a pragma seems out of place is inlining, although in this case I can’t really blame Rust for it. You can tell Rust you think a function should be inlined with a #inline pragma above the function (similar to C’s inline keyword). I am not an expert in compilation and linking, so maybe this is a dumb idea, but I’ve always thought it should be up to the caller to say which functions they’d like inlined, and the function should just mind its own business. (Did someone in the crowd just say “Amen”?) "

(i like this)

---

" Bug: Renaming a function and forgetting to rename all of the overrides. Your Automobile class has a brake() function, and you decide to rename it to applyBrakePedal(). You get some compiler errors and fix up all of the callers, but you forget that the Batmobile subclass overrides brake() with logic to deploy the drag parachute, and now when Batman slams on the brakes, the parachute fails to deploy and he smashes into Macy's in spectacular fashion.

The Swift Fix: Override functions must use the override keyword. Now when you rename the superclass's function, all the subclass functions fail to compile because they're not overriding anything. Why did no one think of this before? I know you Eclipse users out there just click Refactor and go back to sleep, but for folks who use more primitive IDEs either by necessity or by choice, this is a great language feature. "

---

" it’s hard to say that, for instance , Clojure’s list comprehension is nicer than Elixir’s one.

Clojure:

(defn pythagorean-triples [n] (for [x (range 1 (inc n)) y (range x (inc n)) z (range y (inc n)) :when (= (+ (* x x) (* y y)) (* z z))] [x y z]))

Elixir:

pytha3 = fn(n) -> for x <- 1..n, y <- x..n, z <- y..n, x*x+y*y == z*z, do: {x,y,z} end

But, you know. Lisp has some strange kind of beauty for me. "

---

" Any decent PL course would emphasize the first-class-everything (procedures , obviously), lexical scooping for proper closures, strong but dynamic typing (no implicit coersions) etc. No matter which sect one belongs to - ML or Scheme - these are universal fundamentals, the well-researched foundation. "

---

" Patrick Collison emailed me with some information on Croma, the version of Lisp he created that won him Esat BT Young Scientist of the Year (see yesterday's post).

    Croma's partly an attempt to correct everything I see wrong with CL/Scheme (1 namespace and continuations like Scheme, but all sorts of everday useful stuff snarfed from CL like proper lambda-list keywords and so on). It also sticks in various experimental things: 1st class macros (bad idea, I've since decided). I spent a while re-thinking assignment, and Croma uses a strange new idea of “overloading” the values of data objects: you can add in an additional function value (sort of like CL) that'll be used instead of its value in the _function_ position if it exists, to make possible assignment that's generalised and extendable but also idiomatic.
    All of this is geared towards an integrated continuation-based web development system. Using a fairly elaborate standard library, you can do stuff like (link "Click here" (fn (hdrs) (str 'you 'clicked "me")) in the middle of HTML pages. And, of course, it gets rid of HTTP transactional/statelessness cruft, as has been documented by Graham and Queinnec.
    Croma's not quite in a world-useable state, but it's getting there. I'll GPL it as soon as I think others might find it useful, anyway."

---

-- [6]

---

dunno if i wrote this down but from ideas from http://www.antipope.org/charlie/blog-static/2010/08/where-we-went-wrong.html :


" ... >> Even basic operation like removing an element from an array using an index is a pain.

> I think it's fine. If you use a named list {} you can "delete foo[bar]" ...

A mapping is not a list. It doesn't perform the same. You can't push and unshift. You can map on it without Object.stuff. Length is a costly operation. Concat without spread is a pain. And how do you get the "next 3 elements after the 2nd" ?

...

> Out of curiosity, what's the pain with removing an index using `splice(index, 1)`?

Finding it. Remembering it. Getting it right. Reading it back.

Remove the 3rd and get it in Python:

res = a.pop(4)

In JS:

var res = a[4]; a.splice(4, 1); usually a comment to remind you that you pop

"

---

RAD UI delphi lazarus

http://www.lazarus-ide.org/

https://news.ycombinator.com/item?id=14973706

---

https://codedependents.com/2014/01/27/11-best-practices-for-low-latency-systems/

" Choose the right language

Scripting languages need not apply. Though they keep getting faster and faster, when you are looking to shave those last few milliseconds off your processing time you cannot have the overhead of an interpreted language. Additionally, you will want a strong memory model to enable lock free programming so you should be looking at Java, Scala, C++11 or Go.

Keep it all in memory

I/O will kill your latency, so make sure all of your data is in memory. This generally means managing your own in-memory data structures and maintaining a persistent log, so you can rebuild the state after a machine or process restart. Some options for a persistent log include Bitcask, Krati, LevelDB? and BDB-JE. Alternatively, you might be able to get away with running a local, persisted in-memory database like redis or MongoDB? (with memory >> data). Note that you can loose some data on crash due to their background syncing to disk.

Keep data and processing colocated

Network hops are faster than disk seeks but even still they will add a lot of overhead. Ideally, your data should fit entirely in memory on one host. With AWS providing almost 1/4 TB of RAM in the cloud and physical servers offering multiple TBs this is generally possible. If you need to run on more than one host you should ensure that your data and requests are properly partitioned so that all the data necessary to service a given request is available locally.

Keep the system underutilized ... Keep context switches to a minimum ... Keep your reads sequential

All forms of storage, wither it be rotational, flash based, or memory performs significantly better when used sequentially. When issuing sequential reads to memory you trigger the use of prefetching at the RAM level as well as at the CPU cache level. If done properly, the next piece of data you need will always be in L1 cache right before you need it. The easiest way to help this process along is to make heavy use of arrays of primitive data types or structs. Following pointers, either through use of linked lists or through arrays of objects, should be avoided at all costs.

Batch your writes

This sounds counterintuitive but you can gain significant improvements in performance by batching writes. However, there is a misconception that this means the system should wait an arbitrary amount of time before doing a write. Instead, one thread should spin in a tight loop doing I/O. Each write will batch all the data that arrived since the last write was issued. This makes for a very fast and adaptive system.

Respect your cache

With all of these optimizations in place, memory access quickly becomes a bottleneck. Pinning threads to their own cores helps reduce CPU cache pollution and sequential I/O also helps preload the cache. Beyond that, you should keep memory sizes down using primitive data types so more data fits in cache. Additionally, you can look into cache-oblivious algorithms which work by recursively breaking down the data until it fits in cache and then doing any necessary processing.

Non blocking as much as possible ... Async as much as possible ... Parallelize as much as possible "

---

"

I had some extended notes here about "less-mainstream paradigms" and/or "things I wouldn't even recommend pursuing", but on reflection, I think it's kinda a bummer to draw too much attention to them. So I'll just leave it at a short list: actors, software transactional memory, lazy evaluation, backtracking, memoizing, "graphical" and/or two-dimensional languages, and user-extensible syntax. If someone's considering basing a language on those, I'd .. somewhat warn against it. Not because I didn't want them to work -- heck, I've tried to make a few work quite hard! -- but in practice, the cost:benefit ratio doesn't seem to turn out really well. Or hasn't when I've tried, or in (most) languages I've seen. But who knows? There are always exceptions, and I'm wrong far more often than I'm right, so please don't let me stop you if your heart is set on it! "

---

" Date: 2017-08-19 12:15 pm (UTC) From: [identity profile] https://openid-provider.appspot.com/naasking I think this is a good list from the programming language theory perspective, but I think there are also HCI issues that programming languages have barely touched upon.

For instance, our current languages are great at describing and composing algorithms. However, empirical studies have shown that people intuitively describe programs using reactive/temporal semantics, where "if" and "when" describe top-level reactive conditions, and only *after* those conditions are satisfied does algorithmic code run.

Instead, we currently jam reactive abstractions into our algorithmic languages, since that seems more natural from an implementation perspective, ie. describing how reactive expressions are constructed and execute. It's the opposite of what HCI tells us is most natural though.

So I'd like to see a rethinking of how languages are layered, and a temporal logic for globally reasoning about a program's reactive behaviour. This naturally dovetails with your suggestions on modular programming too, because top-level reactive expressions must compose and/or be type checked for consistency.

    Link Reply Thread Hide 1 comment 

Re: One or two more Date: 2017-08-19 05:36 pm (UTC) graydon2: (Default) From: [personal profile] graydon2 Oh definitely.

On the subject of such layering, you might find the Hume language interesting. In addition to an expression/coordination layering, it's also organized around a finer-grained layering of levels of static analysis. "

---

microcolonel 6 days ago [-]

> the Minix 3 kernel is at ~20kloc, while the Linux kernel is now at, what ~11mloc!? Not even redhat can audit that shit properly. (another reason we need a Hurd microkernel, but I digress)

Well, I don't think Hurd is going anywhere. They missed a crucial opportunity to move from Mach to L4 and they simply didn't have the manpower. What we might focus on is migrating facilities (drivers, core services like TCP, system services) from Linux, OpenBSD?, Illumos, and Minix3(especially that daemon that can restart things even when the filesystem daemon goes kaput) to a well-designed L4 like seL4. At that point, at least we have some hope of taming the beast.

---

" My recent experience with acmart, my first maintained Racket library, makes me think that writing libraries is even higher overhead in Racket than in CL, which is already mediocre. "

---

jackdaniel 6 days ago [-]

I think it is a pretty shallow view at standard. CL implementations grow and have agreed interfaces (or portability layers). A few examples:

all these are not mentioned in the standard at all, but are implemented and used in production environment.

Having base standard and extensions which may be used portably (via portability layers) is better situation than having no standard (for instance python or ruby) and a reference implementation.

reply

KirinDave? 6 days ago [-]

> Having base standard and extensions which may be used portably (via portability layers) is better situation than having no standard (for instance python or ruby) and a reference implementation.

Is it though?

I agree CL has (after slightly over 2 decades) worked out a standardization path for some core features that you simply cannot live without (e.g., usocket).

But all these variants are noticeably dated abstractions and in many cases the more modern expressions (e.g., threading vs more modern concurrency tooling) can't (or don't) use the compat libraries.

jackdaniel 5 days ago [-]

Good example of what you're talking about is lparallel library (check out lparallel.org) - it is modern concurrency library build on top bordeaux-threads. lfarm is example of another library workinng on top of usocket which does the same for distributed computing.

reply

---

"It's true, letting go of some CL features is painful and you'll never get them back. CL's condition-restarts are an especially poignant loss in this modern era of Golang's almost comically regressive error handling schemes."

---

 jeffdavis 21 hours ago [-]

I've been having a little trouble using rust for a little project: I need a tree with uplinks (meaning there are cycles). I asked on IRC a couple times and I think what I need is a weak reference inside a refcell, but it's not very easy to make it work cleanly. For one thing, it doesn't look like refcell works well with traits (the nodes in the tree are traits, not plain structs).

I'm a bit frustrated because this is so easy in C. Is there any way this can be easier?

I am imagining a special way to construct cyclical structures where everything inside would have the same lifetime and be destructed at once. We may need to hide the references from destructors though, otherwise a destructor could see a dangling reference.

reply

chancancode 20 hours ago [-]

Have you read "Learning Rust With Entirely Too Many Linked Lists"[1]? I think it will be quite helpful for these kind of situations. It walks you through all the possible tools in the language that is available to you, and at the end, if you just want to write it how you would in C, you could always do it "unsafely" with raw pointers (which is no worse than C).

---

[1] http://cglab.ca/~abeinges/blah/too-many-lists/book/

reply

reificator 14 hours ago [-]

Beat me to this by six hours. Can vouch that the above link is quite helpful when learning Rust.

reply

---

" Whole languages, such as Bash, make it a pain in the neck to take non-trivial function arguments or return non-trivial results, resulting in people's code writing things to the filesystem, hopefully "before" someone else needs to read them.

    Languages like Java encourage patterns where you instantiate a half-baked object and then set the fields later, praying that nobody accidentally tries to use it "too early" it in it's half-baked state while it's internal variables are garbage.

"

--

" Hardening the Kernel in Android Oreo 30 August 2017

...

Android Nougat worked to protect the kernel by isolating it from userspace processes with the addition of SELinux ioctl filtering and requiring seccomp-bpf support, which allows apps to filter access to available system calls when processing untrusted input.

...

Hardened usercopy

Usercopy functions are used by the kernel to transfer data from user space to kernel space memory and back again. Since 2014, missing or invalid bounds checking has caused about 45% of Android's kernel vulnerabilities. Hardened usercopy adds bounds checking to usercopy functions, which helps developers spot misuse and fix bugs in their code. ...

Privileged Access Never (PAN) emulation

...

To mitigate this, CPU vendors have introduced features such as Supervisor Mode Access Prevention (SMAP) in x86 and Privileged Access Never (PAN) in ARM v8.1. These features prevent the kernel from accessing user space directly and ensure developers go through usercopy functions. Unfortunately, these hardware features are not yet widely available in devices that most Android users have today.

Upstream Linux introduced software emulation for PAN in kernel version 4.3 for ARM and 4.10 in ARM64. ...

Together with hardened usercopy, PAN emulation has helped find and fix bugs in four kernel drivers in Pixel devices.

Kernel Address Space Layout Randomization (KASLR) Post-init read-only memory

The final hardening feature extends existing memory protections in the kernel by creating a memory region that's marked read-only after the kernel has been initialized. This makes it possible for developers to improve protection on data that needs to be writable during initialization, but shouldn't be modified after that.

"

---

https://stackoverflow.blog/2017/09/06/incredible-growth-python/?cb=1

---