The meeting took place with the following agenda:
@mainfunctions and numeric literals
- Review indentation syntax changes
- Review implicit resolution changes
- Review autotupling, multiversal equality, name based pattern matching and finally structural types
Date and Location
The meeting took place on the 13th March 2020 throughout the day on Zoom.
The meeting wasn’t broadcast.
Minutes were taken by Jamie Thompson.
- Martin Odersky (@odersky), EPFL
- Adriaan Moors (@adriaanm), Lightbend
- Guillaume Martres (@smarter), EPFL
- Sébastien Doeraene (@sjrd), Scala Center
- Lukas Rytz (@lrytz), Lightbend
- Jamie Thompson (@bishabosha), Scala Center
- Nicolas Stucki (@nicolasstucki), EPFL
- Olivier Blanvillain (@OlivierBlanvillain)
- Darja Jovanovic (@darjutak), Process Lead
Revisiting Whitebox Inline Definitions
Martin had an idea for a potential cleanup of syntax for whitebox inline definitions:
Currently in Dotty:
inline def f() <: T = … inline given C as _ <: T = …
Downsides to this approach:
- Differences between
- Obscure syntax is bad for teaching.
- Hard to Google.
He suggested to use a modifier instead:
inline flextype def f(): T = … inline flextype given C as T = …
flextypeis more searchable.
Adrian likes the idea from
inline given to represent the return type as a bounded existential.
whitebox[T], where Martins’ response was: specialised syntax is harder to swallow, when its specific only to the whitebox and doesn’t extend to other types.
- 2: Maybe a new modifier to represent both inline and whitebox?
- Motivation is replacement for behaviour of DelayedInit:
- Initialisation code in object can cause problems for thread safety.
- Because we have top level definitions, it makes perfect sense to allow program entrypoint to be method at the top level.
- What should the program be called? We use the
@mainannotation to capture the method name and create a wrapper class of the same name in the package containing the method.
- Also provide minimal commandline argument processing to map directly to method parameters, using the
Concerns with Commandline Argument Processing
Jamie: Should it allow named arguments?
- How far should the language go to support command line parsing?
- Should the language go all the way to a bespoke solution, because the potential use cases are unknown.
- Or just stick to basic primitives with no pluggability.
- Problematic to avoid optional flags
Guillaume: All large programs need a
Addressing the above, Martin suggests that the intended use case for argument mapping is as follows:
- Ideal for test programs, simple scripts
- e.g. data science
- Any program needing more sophisticated processing should probably use
- avoids mutable array
- processing can then be delegated to a library like Scopt
Concern with FromString Type Class
- FromString is very general, but the usecase presented is just for CLI, falls apart too quickly due to custom formatting and expressing multi-argument structures such as ranges etc.
- He would stop at
FromStringtoo general to use for CLI processing, it could make evolution harder if people use it for other cases.
- Lots of options for the problem of structuring unstructured data, such as
:separated lists for class paths. A general type class is too basic for this situation.
- However, it is easier to make a local specific
FromStringjust for CLI than in haskell where you must consider global importance.
- Agrees that whatever argument processing should be able to map directly to method arguments, but something more specialised than
FromStringshould be used.
- Alternatively the current proposal could go ahead, but removing hooks for extension in the StdLib.
Martin, addressing these concerns:
- Haskell uses
Readeverywhere, if we have
toString, we should have a general
@mainmethods should have a single argument with type class to parse all command line arguments into it, which is one step removal from single argument of
- We must have support for the simple case,
@mainis about boilerplate free work, if you have a need for complex args, the complexity of the program has grown too much for this concern.
Adriaan suggests that maybe someone can try a new proposal for argument parsing.
Concern for Use Case
Sébastien: How many programs are so trivial that they dont need argument processing?
- 99% in his estimate.
- For most people, e.g. simple scripting, these just need either none or few primitive args.
Guillaume: What’s the story for refactoring from the simple case to the more complex cases?
- We should look into this
- But transitioning to full cli parser is huge step, it will be complex to join the dots.
Existing Solutions Suggested in the meeting
Guillaume: Simple, but it allows validation
@main from ammonite is more complicated (than the SIP proposal) but scales.
Sébastien: Overloaded mains look more complicated
- It generates help text if args are not correct
- We should be able to document args to main, and support should be in StdLib
The main concerns with the proposal are the interactions between the intended use case and the pressure it imposes on the Standard Library. If it is only intended for very simple ordered arguments then perhaps a pluggable
FromString type class is too simple to include in the standard library. If the language should support more complicated processing, then
FromString is too basic to be pluggable.
In a straw poll the present members of the committee were in favour of having the
@main annotation for program entry points, but perhaps someone could come up with a proposal for a more sophisticated processing solution.
Sébastien presented the proposal:
- Uses a type class which is very similar to
FromString, with several sub-types for different kinds of literals.
- takes string representation of literal, and converts to an expected type.
- the expected type also specialises which type class instance to search for.
Concerns with Requiring an Expected Type
- Only works well for named constants, or passing to unambiguous method arguments.
- In Scala we often lack a concrete expected type, e.g. the left hand side of an operation.
- How well does this scale
- Concerns for loss of precision, e.g. different behaviour of adding different numeric types etc.
Lukas in chat, to demonstrate lack of inference on LHS of an expression:
scala> val x: Long = 10_000_000_000 val x: Long = 10000000000 scala> val x: Long = 10_000_000_000 + 1 1 |val x: Long = 10_000_000_000 + 1 | ^^^^^^^^^^^^^^ | number too large
Martin in response to concerns:
- By default, each sub expression of an expression of an expected type will have the same behaviour as without generic literals,
- More specific behaviour is only activated with an expected type, making this feature perfect for named constants, (which should have an expected type if public).
Concerns for Aesthetics
- Using a string literal constant as a number would look embarassing to introduce to a Python programmer.
- Especially the
BigIntconstructor, its really ugly.
Guillaume: In Python there is no syntactic overhead for BigInt, Scala would add noise with the expected type.
- Its already embarassing enough to explain the different numeric types.
- The situation of overflow in smaller types will not be solved by generic literals, therefore you must explain the existance of
BigInt, so may as well explain its
Concerns for Use Case
Adriaan: Will this work well in specific use cases, such as
- Data Science
Concerns with Behaviour
Jamie: There is a problem with failure of conversion with
fromDigits, if literal format doesnt match defined type class, then there is silent failure
Nicolas: What can we do with final vals and constant types.
Martin, in response:
- We can take another look at the specific implementation for activating conversions, and even the type classes in the library.
- Open question: should we have a user pluggable notion of constant type for generic literals.
1. Numeric Suffixes
- Suffixes may be more suited to Scala as an expected type only works well when the whole expression is a simple literal.
- Precedent with prefixes for string interpolator, numeric suffixes could have similar pluggable implementation.
- Suffix is ad-hoc and visually ugly
- Not so bad if its a general pluggable solution, but we can have leaner syntax rules by using types.
- In general, non trivial literals shouldn’t be used inline, we should write a named constant.
- Guy Steele suggested it as a more beautiful solution.
2. Special case Factory Methods
- The less inference on things, the better
- Python behaviour of conversions is confusing to data scientists, so literals are not always clear to understand
- Assume magical way to pass any literal to an apply method on a companion object of a numeric type. Works for custom types
Scheduling Of Release Discussion
Martin suggested this was not urgent to ship in 3.0, and asks if it could be put under a flag to ensure it gets tested.
- Its tricky because it adds a lot of noise to the std lib and we must be careful what is included there.
- Could we ship it in separate library?
- The more we can delay from all proposals, the better.
There has been little testing in applications of this feature, not enough to make a judgement on its utility. With more consultation from potential users, the SIP committee can gather conclusive evidence.
There are several concerns, relating to the scope of where this feature makes sense, due to limited type inference, and the verbosity required in inline expressions. It is suggested that this is a non-issue because by default,
Long literals can’t be inferred on the LHS of an addition expression without an expected type, and non-trivial literals should be lifted to named constants.
Alternatives suggested have been user-defined numeric suffixes, and specialised factory methods for user defined types.
Martin presents the proposal as consisting of several parts:
- New control syntax (
for do, drop
- Optional braces for control expressions if you indent on new line after specific token.
- Optional braces are extended to named definitions:
- Experience showed that without a marker, indentation of 2 spaces was no longer enough to clearly distinguish member definitions.
withworked as a marker for a while but had some disambiguation issues.
- colon marker is settled as the proposal
- Totally sold as a productivity boost
- Used widely in compiler codebase for 6 months
- Does find is necessary to have vertical bars in editor for indentation
- Examples at:
thenin if expression was harder to get used to
thenfeels like noise but then with multiline conditions is harder to read without it, but was fine with the original parens on condition.
endmarker, by being optional, leads to too much bikeshedding
- we can rely on style guides to enforce consistency in a codebase.
- In general, a 10 line block with any empty lines should have an end marker.
recommends this style for class with companion: ```scala class Foo:
def foo = ???
def bar = ???
#### On Composability of Features Sébastien asks: - If the implementation of the separate features are orthogonal - As a consequence, who then has experience using the old control syntax with optional braces, as originally it was thought that indentation was dependent on the new control syntax. Martin: - The features are independent. - However it seems natural to transition to use them together in 1 step. Sebastien had a concern about the performance for parsing, compared to old style code, eg: ```scala if (x || y) && (a || b) then foo()
Martin: Lookahead in the parser is necessary, but this is possible due to operators being accepted on new lines
On Enforcing One Style
- Main worry with the whole proposal is too much choice, which could have negative feedback.
- Dotty should enforce one syntactic style, with leniency under the Scala2 language mode
Guillaume, in disagreement:
- Today its easy to port to Dotty by copy pasting code, or just changing a version number.
- e.g. StackOverflow answers wont work
- We need to decide on one choice
- Too many ways to do
- Dotty supposed to be about simplification, but this makes one of the most basic things complex
- perhaps the compiler should flag if styles are mixed
No one present at the meeting showed outright objection to the specific new syntax changes. Comments given suggest that for users in the meeting, overall there is a boost to productivity.
The main objection to the proposal is the amount of choice the user has in the syntax they may choose to use. And there is an open question of what should be restricted in the launch of Scala 3.
Implicit Resolution changes
Martin outines the rules in the documentation:
1) Types must be explicit for implicit vals and results of defs
- Exception for local block
2) Nesting of implicits is significant
- Implicits look from inside out
- Now resolution of a symbol, not a name
- No more shadowing, but nesting is important
Concerns with Nesting rule
- We should be careful with what code from Scala 2 will change under the new rules.
- Concretely in the community build, compilation of Scalacheck inferred a different
Stringthen the one under Scala 2.
- There can be new ambiguities when mixing lexical scope with inheritance scope rules
- People probably do not check that the semantics have changed if you were to change the Scala version and it compiles.
- But perhaps for this meeting we should not focus on migration, just the objective quality of the changes.
Sébastien, in response:
- if this is dangerous to change, we could delay the changes beyond 3.0
- Migration is important
Adriaan, in response:
- I don’t think we should be concerned about laziness to read new rules
- When looking for an implicit conversion to add a method, the innermost match way not type check, should we look in the outer scope for an objectively better definition?
Martin, in response:
- This could be problematic because of performance cost of backtracking. We have caches of innermost to outermost.
- Because of cost of new types such as union and intersection, we need savings elsewhere
- In terms of standardisation, is this acceptable as is to go in?
3) package prefix no longer contributes to implicit scope
Martin: Motivation is that it is too easy to pollute everywhere with implicit definitions.
Guillaume: Doesn’t appear to break much code so it seems ok so far.
4-5) clean up surprising behaviour of divergence or ambiguity
- in Scala 2 ambiguity wasnt fatal in the presence of an alternative, now it is fails immediately.
- This breaks
Not[T]encodings, which is now special cased in compiler.
- Why are the new rules better?
- My experience in Scala 2 is that diverging implicits are not very easy to understand
Guillaume: Is the specification of Scala 2 divergence in user documentation?
Martin: problem with fatal divergence now is that it seems underspecified, e.g. visitation order of search has an effect.
6) Implicit Conversions with By-Name Parameters Now Have the Same Priority
Martin: In Scala 2, by-name conversions were added in a “bolt on” fashion, making spec strange for no principled reason.
7) Context Parameters Lower Specificity of Implicits
- Worst feature change as it is seems the most arbitrary
- It may be better for reducing ambiguity, but is more complex.
Martin believes the inheritance model is very fragile, but this version is more modular.
- Believes the older version leads to easier evolution of a library in a binary compatible fashion.
- Demonstrates Scala.js Union Types as a way to evolve priorities with a new base type.
- Asks if the new rules allow to add an implicit definition of a priority in-between two other definitions while preserving binary compatability?
Martin, in response:
- Shows a demo of how to insert a new implicit in between existing definitions.
- The new scheme of priorities is more simple to grow than before because inheritance is fragile
Sébastien is satisfied with the demo, is fine either way to add to 3.0 or later. Someone should champion it.
- Will support if library authors are behind it like Cats.
- It’s hard to teach and hard to understand.
- Is it really better than status quo? Yes the old system is clunky but is well understood.
- So is it worth it?
Martin, in response:
- Yes, it’s better for the small number of library authors who really need it.
- The old way with traits is a hack, this has principles, why should we deprive ourselves of the correct solution?
- It would be a pity to remove, it took a lot of thought, and is an objectively better design.
- We should have a tutorial documentation for the new rules
There is fair skepticism for introducing the changes immediately, as the migration path from Scala 2 can be complicated if runtime behaviour changes undetected without a warning or error at compile time. Additionally there are concerns about the effort required to re-educate users with the new rules.
Sébastien: The main challenge is migration, what is the ideal scenario?
Martin: In the future, there is no autotupling, and then any parens following an infix operator is the start of a tuple and not an argument list.
- It looks like most of the use cases for requiring the extra parens have been dropped in 2.13
- What is the state of deprecation in 2.13
- Symbolic operators with multiple arguments
- We could delay beyond 3.0, and migration concerns will be less.
- It sounds reasonable to delay, and definitely warn/error at definition infix ops with multiple arguments.
- Coupled with ensuring the definitions of infix calls have the infix annotation.
There is general favorability of the proposal from present members, the main concerns are whether or not to delay the feature to aid migration.
Sébastien: has this changed since last discussion?
- not changed in very long time
strictEqualityprobably not tested in the wild much.
Martin: There has been concern about why we have two type parameters, which was resolved with the people concerned.
- Highlights an issue with
strictEqualityand not compiling the standard library:
$ dotr -language:strictEquality Starting dotty REPL... scala> Some(1) == Some(2) 1 |Some(1) == Some(2) |^^^^^^^^^^^^^^^^^^ |Values of types Some[Int] and Some[Int] cannot be compared with == or !=
- Asks if is this crucial for 3.0
Martin: Scala 2 has some checks which give a warning on comparing nonsensical types, which do not exist in dotty. So he would be uncomfortable to ship without a similar feature.
Guillaume: one solution to the warning could be disjoint checking, like implemented for match types.
Martin: highlights that the match type implementation doesnt work for traits.
Eqldoesn’t help against comparing two values of statically disjoint subtypes of an ADT, if they have the same type parameter.
- Could we combine warning on disjoint with
- Or possibly a
Disjoint[A,B]type class which could feed into warnings?
Martin: There are far more cases where things are disjoint than not so there would be much more noise.
There is mixed favorability for the proposal as is, with concern for precision of errors in the presence of known static divergence and the timing of release.
Name based pattern matching
Name based pattern matching is presented as a formalisation of the rules included in Scala 2, with a few new ones. It is used as the implementation of pattern matching for case classes in Dotty.
Sébastien: should we use
Product as a marker trait for this?
Olivier: We use it to reduce the noise in the LUB of case classes.
Sébastien demonstrates how scala-unboxed-option uses name based patten matching on a value class to achieve matching with no allocations.
Nicolas: Can we use extension methods to do name based pattern match? (Rather than extend
Guillaume: No because this is run at a later phase that does not resolve dotty extension methods.
On the concern of case class unapply
(This concern is separate to the SIP proposal)
- Some people rely on
unapplyof case class companion to extract an
Optionof some tuple, in Dotty, the
unapplymethod is the identity.
- One solution is to special-case case classes in patterns, like Scala 2.
Martin in response:
- Is it worth the effort to make the compiler more complex for the sake of binary compatability?
The main concern is the use of
Product as a marker trait, and the current inability of Dotty extension methods to be used for name-based extractors. But overall everyone present is in favour.
Selectable become a marker trait rather than require specific methods?
- Not seen any real-world usage, can some existing “Record” Style macros benefit from this.
- Seems ugly and we need to prove it can be used equivalently to
HMapin all use cases.
Martin: Performance concerns with using
Guillaume: Whatever we propose, it needs to be convenient for calling.
Martin: We need something to support Chisel in Scala 3.
Guillaume, in response:
- This is more complicated as Dotty breaks their use case.
- We really need to see it tried in a real application for its intended use case.
- To get people to really try it out, it seems you need a full tutorial. And current docs seems lacking.