proj-oot-ootDataNotes5

" Preserving order of dictionaries and kwargs

In CPython 3.6+ dicts behave like OrderedDict? by default (and this is guaranteed in Python 3.7+). This preserves order during dict comprehensions (and other operations, e.g. during json serialization/deserialization)

import json x = {str(i):i for i in range(5)} json.loads(json.dumps(x))

1. Python 2 {u'1': 1, u'0': 0, u'3': 3, u'2': 2, u'4': 4}
2. Python 3 {'0': 0, '1': 1, '2': 2, '3': 3, '4': 4}

Same applies to kwargs (in Python 3.6+), they're kept in the same order as they appear in parameters. Order is crucial when it comes to data pipelines, previously we had to write it in a cumbersome manner:

from torch import nn

1. Python 2 model = nn.Sequential(OrderedDict?([ ('conv1', nn.Conv2d(1,20,5)), ('relu1', nn.ReLU?()), ('conv2', nn.Conv2d(20,64,5)), ('relu2', nn.ReLU?()) ]))
2. Python 3.6+, how it *can* be done, not supported right now in pytorch model = nn.Sequential( conv1=nn.Conv2d(1,20,5), relu1=nn.ReLU?(), conv2=nn.Conv2d(20,64,5), relu2=nn.ReLU?()) )

Did you notice? Uniqueness of names is also checked automatically.

Iterable unpacking

1. handy when amount of additional stored info may vary between experiments, but the same code can be used in all cases model_paramteres, optimizer_parameters, *other_params = load(checkpoint_name)
2. picking two last values from a sequence

• prev, next_to_last, last = values_history

1. This also works with any iterables, so if you have a function that yields e.g. qualities,
2. below is a simple way to take only last two values from a list

• prev, next_to_last, last = iter_train(args)

...

Multiple unpacking

Here is how you merge two dicts now:

x = dict(a=1, b=2) y = dict(b=3, d=4)

1. Python 3.5+ z = {x, y}
2. z = {'a': 1, 'b': 3, 'd': 4}, note that value for `b` is taken from the latter dict.

See this thread at StackOverflow? for a comparison with Python 2.

The same approach also works for lists, tuples, and sets (a, b, c are any iterables):

[*a, *b, *c] # list, concatenating (*a, *b, *c) # tuple, concatenating {*a, *b, *c} # set, union

Functions also support this for *args and kwargs:

Python 3.5+ do_something({default_settings, custom_settings})

1. Also possible, this code also checks there is no intersection between keys of dictionaries do_something(first_args, second_args)

...

Future-proof APIs with keyword-only arguments ... In Python 3, library authors may demand explicitly named parameters by using *:

class SVC(BaseSVC?): def __init__(self, *, C=1.0, kernel='rbf', degree=3, gamma='auto', coef0=0.0, ... )

...

    Enums are theoretically useful, but
        string-typing is already widely adopted in the python data stack
        Enums don't seem to interplay with numpy and categorical from pandas
    ...
    Python 3 has stable ABI
    

" -- [1]

---

" Problems for code migration specific for data science (and how to resolve those)

    support for nested arguments was dropped

    map(lambda x, (y, z): x, z, dict.items())

    However, it is still perfectly working with different comprehensions:

    {x:z for x, (y, z) in d.items()}

    In general, comprehensions are also better 'translatable' between Python 2 and 3.

    map(), .keys(), .values(), .items(), etc. return iterators, not lists. Main problems with iterators are:
        no trivial slicing
        can't be iterated twice

    Almost all of the problems are resolved by converting result to list.

...

Course authors should spend time in the first lectures to explain what is an iterator, why it can't be sliced / concatenated / multiplied / iterated twice like a string (and how to deal with it). " -- [2]

---

" Optional Chaining #

let x = foo?.bar.baz();

this is a way of saying that when foo is defined, foo.bar.baz() will be computed; but when foo is null or undefined, stop what we’re doing and just return undefined.”

More plainly, that code snippet is the same as writing the following.

let x = (foo === null

    undefined :
    foo.bar.baz();

You might find yourself using ?. to replace a lot of code that performs repetitive nullish checks using the && operator.

Before if (foo && foo.bar && foo.bar.baz) { ... }

After-ish if (foo?.bar?.baz) { ... }

Keep in mind that ?. acts differently than those && operations since && will act specially on “falsy” values (e.g. the empty string, 0, NaN?, and, well, false), but this is an intentional feature of the construct. It doesn’t short-circuit on valid data like 0 or empty strings.

Optional chaining also includes two other operations. First there’s the optional element access which acts similarly to optional property accesses, but allows us to access non-identifier properties (e.g. arbitrary strings, numbers, and symbols):

    arr?.[0];

There’s also optional call, which allows us to conditionally call expressions if they’re not null or undefined.

    log?.(`Request started at ${new Date().toISOString()}`);

Nullish Coalescing #

You can think of this feature - the ?? operator - as a way to “fall back” to a default value when dealing with null or undefined. When we write code like

let x = foo ?? bar();

this is a new way to say that the value foo will be used when it’s “present”; but when it’s null or undefined, calculate bar() in its place.

Again, the above code is equivalent to the following.

let x = (foo !== null && foo !== undefined) ? foo : bar();

The ?? operator can replace uses of

function initializeAudio() { let volume = localStorage.volume

    // ...}

When localStorage.volume is set to 0, the page will set the volume to 0.5 which is unintended. ?? avoids some unintended behavior from 0, NaN? and "" being treated as falsy values. "

"

achou 9 hours ago [-]

I just did some refactoring on a medium size code base and here are a few things to watch out for when adopting optional chaining and the new null coalescing operator:

  foo && await foo();

is not the same as

  await foo?.();

this will work in most cases but subtly, the await wraps the undefined case into a Promise, while the original code would skip the await altogether.

String regular expression matching returns null, not undefined, so rewriting code such as:

  const match = str.match(/reg(ex)/);
  return match && match[1];

is not the same thing as:

  return match?.[1];

because the latter returns undefined, not null, in case of match failure. This can cause problems if subsequent code expects null for match failure. An equivalent rewrite would be:

  return match?.[1] ?? null;

which is longer than the original and arguably less clear.

A common idiom to catch and ignore exceptions can interact poorly with optional chaining:

  const v = await foo().catch(_ => {});
  return v?.field; // property 'field' does not exist on type 'void'

This can be easily remedied by changing the first line to:

  const v = await foo().catch(_ => undefined);

Of course, these new operators are very welcome and will greatly simplify and help increase the safety of much existing code. But as in all things syntax, being judicious about usage of these operators is important to maximize clarity.

reply

mattigames 7 hours ago [-]

You have to watch out for first and last one in JavaScript? but not on TypeScript? as it isn't possible to make that mistake because you have to type it as a promise or in the last one as void.

You can even avoid the problem in the second one by using NonNullable? TypeScript? types, but I admit that's not common so its still likely to arise.

reply

achou 7 hours ago [-]

The first example can happen in TypeScript?; foo has type

  (() => Promise<void>) | undefined

admittedly it may not be all that common to have a function-valued variable that may be undefined, but it happened in the code base I was working with.

In the last example, you're right that TypeScript? will catch this at compile time. My point was to show how this compile time error can happen from refactoring to use optional chaining, and one easy solution in this case.

reply

"

---

:

data NonEmpty? a = a :

---

want to be able to do something like this Ruby example:

  list_cart unless @cart.include? 'pineapple'

so we need:

• 'unless' control structure
• possibly, an 'include' function or operator that can go to the right of the collection and to the left of the query

---

bools in structs should have to have a default value, or they should have no default; they should not auto-default to 0 when unspecified

---

[3]