Migrating a Project to Scala 2.13's Collections

Language

This document describes the main changes for collection users that migrate to Scala 2.13 and shows how to cross-build projects with Scala 2.11 / 2.12 and 2.13.

For an in-depth overview of the Scala 2.13 collections library, see the collections guide. The implementation details of the 2.13 collections are explained in the document the architecture of Scala collections.

The most important changes in the Scala 2.13 collections library are:

  • scala.Seq[+A] is now an alias for scala.collection.immutable.Seq[A] (instead of scala.collection.Seq[A]). Note that this also changes the type of Scala varargs methods.
  • scala.IndexedSeq[+A] is now an alias for scala.collection.immutable.IndexedSeq[A] (instead of scala.collection.IndexedSeq[A]).
  • Transformation methods no longer have an implicit CanBuildFrom parameter. This makes the library easier to understand (in source code, Scaladoc, and IDE code completion). It also makes compiling user code more efficient.
  • The type hierarchy is simplified. Traversable no longer exists, only Iterable.
  • The to[Collection] method was replaced by the to(Collection) method.
  • Views have been vastly simplified and work reliably now. They no longer extend their corresponding collection type, for example, an IndexedSeqView no longer extends IndexedSeq.
  • collection.breakOut no longer exists, use .view and .to(Collection) instead.
  • Immutable hash sets and hash maps have a new implementation (ChampHashSet and ChampHashMap, based on the “CHAMP” encoding).
  • New collection types:
    • immutable.ArraySeq is an effectively immutable sequence that wraps an array
    • immutable.LazyList is a linked list that is lazy in its state, i.e., whether it’s empty or non-empty. This allows creating a LazyList without evaluating the head element. immutable.Stream, which has a strict head and a lazy tail, is deprecated.
  • Deprecated collections were removed (MutableList, immutable.Stack, others)
  • Parallel collections are now in a separate hierarchy in a separate module.
  • The scala.jdk.StreamConverters object provides extension methods to create (sequential or parallel) Java 8 streams for Scala collections.

Tools for migrating and cross-building

The scala-collection-compat is a library released for 2.11, 2.12 and 2.13 that provides some of the new APIs from Scala 2.13 for the older versions. This simplifies cross-building projects.

The module also provides migratrion rules for scalafix that can update a project’s source code to work with the 2.13 collections library.

scala.Seq and scala.IndexedSeq migration

In Scala 2.13 scala.Seq[+A] is an alias for scala.collection.immutable.Seq[A] (“ISeq”), instead of scala.collection.Seq[A] (“CSeq”). Similarly, scala.IndexedSeq[+A] is an alias for scala.collection.immutable.IndexedSeq[A]. These changes require some planning depending on how your code is going to be used.

If you’re making a library intended to be used by other programmers, then using scala.Seq, scala.IndexedSeq, or vararg is going to be a breaking change in the API semantics. For example, if there was a function def orderFood(order: Seq[Order]): Seq[Food], previously the library user would have been able to pass in an array of Order, but it won’t work for 2.13.

  • if you cross build with Scala 2.12 and want to maintain the API semantics for 2.13 version of your library, or
  • if your library users frequently uses mutable collections such as Array

you can import collection Seq explicitly in your code.

import scala.collection.Seq

object FoodToGo {
  def orderFood(order: Seq[Order]): Seq[Food]
}

Note that this might still break the source compatibility if scala.Seq (or just Seq) appears in the source code.

val food: Seq[Food] = FoodToGo.orderFood(order) // won't compile

Since Seq, an alias for ISeq in 2.13, is narrower than CSeq, the above code will no longer compile. One workaround would be to ask your users to add toSeq, which returns ISeq.

val food: Seq[Food] = FoodToGo.orderFood(order).toSeq // add .toSeq

Another workaround might be to accept CSeq, but return ISeq.

import scala.collection.{ Seq => CSeq }
import scala.collection.immutable.{ Seq => ISeq }

object FoodToGo {
  def orderFood(order: CSeq[Order]): ISeq[Food]
}

In the future when your API is able to break the source compatibility, it might also make sense to migrate towards ISeq for both Scala 2.12 and Scala 2.13.

import scala.collection.immutable.{ Seq => ISeq }

object FoodToGo {
  def orderFood(order: ISeq[Order]): ISeq[Food]
}

Similarly, if you’re making an end-user application, unifying to CSeq might be the easier and safer initial path especially for a larger and complex code base. Switching to ISeq will be a more advanced refactoring.

Note that in Scala 2.13 the sequence passed into as a varargs as orderFood(xs: _*) must also be immutable. This is because the sequence passed in as a varargs must conform to scala.Seq according to SLS 6.6. Thus, if your API exposes varargs it will be an unavoidable breaking change. This might affect Java interoperability.

Masking scala.Seq

To use the compiler to bad the use of plain Seq, you can declare your own Seq to mask scala.Seq.

package example

import scala.annotation.compileTimeOnly

/**
  * In Scala 2.13, scala.Seq moved from scala.collection.Seq to scala.collection.immutable.Seq.
  * In this code base, we'll require you to name ISeq or CSeq.
  *
  * import scala.collection.{ Seq => CSeq }
  * import scala.collection.immutable.{ Seq => ISeq }
  *
  * This Seq trait is a dummy type to prevent the use of `Seq`.
  */
@compileTimeOnly("Use ISeq or CSeq") private[example] trait Seq[A1, F1[A2], A3]

/**
  * In Scala 2.13, scala.IndexedSeq moved from scala.collection.IndexedSeq to scala.collection.immutable.IndexedSeq.
  * In this code base, we'll require you to name ISeq or CSeq.
  *
  * import scala.collection.{ IndexedSeq => CIndexedSeq }
  * import scala.collection.immutable.{ IndexedSeq => IIndexedSeq }
  *
  * This IndexedSeq trait is a dummy type to prevent the use of `IndexedSeq`.
  */
@compileTimeOnly("Use IIndexedSeq or CIndexedSeq") private[example] trait IndexedSeq[A1, F1[A2], A3]

This might be useful during the transition period where you have to remember to import CSeq.

What are the breaking changes?

The following table summarizes the breaking changes. The “Automatic Migration Rule” column gives the name of the migration rule that can be used to automatically update old code to the new expected form.

Description Old Code New Code Automatic Migration Rule
Method to[C[_]] has been removed (it might be reintroduced but deprecated, though) xs.to[List] xs.to(List) Collection213Upgrade, Collections213CrossCompat
mapValues and filterKeys now return a MapView instead of a Map kvs.mapValues(f) kvs.mapValues(f).toMap RoughlyMapValues
Iterable no longer has a sameElements operation xs1.sameElements(xs2) xs1.iterator.sameElements(xs2) Collection213Upgrade, Collections213CrossCompat
collection.breakOut no longer exists val xs: List[Int] = ys.map(f)(collection.breakOut) val xs = ys.iterator.map(f).to(List) Collection213Upgrade
zip on Map[K, V] now returns an Iterable map.zip(iterable) map.zip(iterable).toMap Collection213Experimental
ArrayBuilder.make does not accept parens anymore ArrayBuilder.make[Int]() ArrayBuilder.make[Int] Collection213Upgrade, Collections213CrossCompat

Some classes have been removed, made private or have no equivalent in the new design:

  • ArrayStack
  • mutable.FlatHashTable
  • mutable.HashTable
  • History
  • Immutable
  • IndexedSeqOptimized
  • LazyBuilder
  • mutable.LinearSeq
  • LinkedEntry
  • MapBuilder
  • Mutable
  • MutableList
  • Publisher
  • ResizableArray
  • RevertibleHistory
  • SeqForwarder
  • SetBuilder
  • Sizing
  • SliceInterval
  • StackBuilder
  • StreamView
  • Subscriber
  • Undoable
  • WrappedArrayBuilder

Other notable changes are:

  • Iterable.partition invokes iterator twice on non-strict collections and assumes it gets two iterators over the same elements. Strict subclasses override partition do perform only a single traversal
  • Equality between collections is not anymore defined at the level of Iterable. It is defined separately in the Set, Seq and Map branches. Another consequence is that Iterable does not anymore have a canEqual method.
  • The new collections makes more use of overloading. You can find more information about the motivation behind this choice here. For instance, Map.map is overloaded:

    scala> Map(1 -> "a").map
      def map[B](f: ((Int, String)) => B): scala.collection.immutable.Iterable[B]
      def map[K2, V2](f: ((Int, String)) => (K2, V2)): scala.collection.immutable.Map[K2,V2]
    

    Type inference has been improved so that Map(1 -> "a").map(x => (x._1 + 1, x._2)) works, the compiler can infer the parameter type for the function literal. However, using a method reference in 2.13.0-M4 (improvement are on the way for 2.13.0) does not work, and an explicit eta-expansion is necessary:

    scala> def f(t: (Int, String)) = (t._1 + 1, t._2)
    scala> Map(1 -> "a").map(f)
                            ^
          error: missing argument list for method f
          Unapplied methods are only converted to functions when a function type is expected.
          You can make this conversion explicit by writing `f _` or `f(_)` instead of `f`.
    scala> Map(1 -> "a").map(f _)
    res10: scala.collection.immutable.Map[Int,String] = ChampHashMap(2 -> a)
    
  • Views have been completely redesigned and we expect their usage to have a more predictable evaluation model. You can read more about the new design here.
  • mutable.ArraySeq (which wraps an Array[AnyRef] in 2.12, meaning that primitives were boxed in the array) can now wrap boxed and unboxed arrays. mutable.ArraySeq in 2.13 is in fact equivalent to WrappedArray in 2.12, there are specialized subclasses for primitive arrays. Note that a mutable.ArraySeq can be used either way for primitive arrays (TODO: document how). WrappedArray is deprecated.
  • There is no “default” Factory (previously known as [A, C] => CanBuildFrom[Nothing, A, C]): use Factory[A, Vector[A]] explicitly instead.

Breaking changes with old syntax still supported

The following table lists the changes that continue to work with a deprecation warning.

Description Old Code New Code Automatic Migration Rule
collection.Set/Map no longer have + and - operations xs + 1 - 2 xs ++ Set(1) -- Set(2) Collection213Experimental
collection.Map no longer have -- operation map -- keys map.to(immutable.Map) -- keys  
immutable.Set/Map: the + operation no longer has an overload accepting multiple values Set(1) + (2, 3) Set(1) + 2 + 3 Collection213Upgrade, Collections213CrossCompat
mutable.Map no longer have an updated method mutable.Map(1 -> 2).updated(1, 3) mutable.Map(1 -> 2).clone() += 1 -> 3 Collection213Upgrade, Collections213CrossCompat
mutable.Set/Map no longer have a + operation mutable.Set(1) + 2 mutable.Set(1).clone() += 2 Collection213Upgrade, Collections213CrossCompat
SortedSet: the to, until and from methods are now called rangeTo, rangeUntil and rangeFrom, respectively xs.until(42) xs.rangeUntil(42)  
Traversable and TraversableOnce are replaced with Iterable and IterableOnce, respectively def f(xs: Traversable[Int]): Unit def f(xs: Iterable[Int]): Unit Collection213Upgrade, Collections213CrossCompat
Stream is replaced with LazyList Stream.from(1) LazyList.from(1) Collection213Roughly
Seq#union is replaced with concat xs.union(ys) xs.concat(ys)  
Stream#append is replaced with lazyAppendAll xs.append(ys) xs.lazyAppendedAll(ys) Collection213Upgrade, Collections213CrossCompat
IterableOnce#toIterator is replaced with IterableOnce#iterator xs.toIterator xs.iterator Collection213Upgrade, Collections213CrossCompat
copyToBuffer has been deprecated xs.copyToBuffer(buffer) buffer ++= xs Collection213Upgrade, Collections213CrossCompat
TupleNZipped has been replaced with LazyZipN (xs, ys).zipped xs.lazyZip(ys) Collection213Upgrade
retain has been renamed to filterInPlace xs.retain(f) xs.filterInPlace(f.tupled) Collection213Upgrade
:/ and /: operators have been deprecated (xs :\ y)(f) xs.foldRight(y)(f) Collection213Upgrade, Collections213CrossCompat
companion operation has been renamed to iterableFactory xs.companion xs.iterableFactory  

Deprecated things in 2.12 that have been removed in 2.13

  • collection.convert.JavaConversions. Use collection.convert.JavaConverters instead ;
  • collection.mutable.MutableList (was not deprecated in 2.12 but was considered to be an implementation detail for implementing other collections). Use an ArrayDeque instead, or a List and a var ;
  • collection.immutable.Stack. Use a List instead ;
  • StackProxy, MapProxy, SetProxy, SeqProxy, etc. No replacement ;
  • SynchronizedMap, SynchronizedBuffer, etc. Use java.util.concurrent instead ;

Are there new collection types?

scala.collection.immutable.ArraySeq is an immutable sequence backed by an array. It is used to pass varargs parameters.

The scala-collection-contrib module provides decorators enriching the collections with new operations. You can think of this artifact as an incubator: if we get evidence that these operations should be part of the core, we might eventually move them.

The following collections are provided:

  • MultiSet (both mutable and immutable)
  • SortedMultiSet (both mutable and immutable)
  • MultiDict (both mutable and immutable)
  • SortedMultiDict (both mutable and immutable)

Are there new operations on collections?

The following new partitioning operations are available:

def groupMap[K, B](key: A => K)(f: A => B): Map[K, CC[B]] // (Where `CC` can be `List`, for instance)
def groupMapReduce[K, B](key: A => K)(f: A => B)(g: (B, B) => B): Map[K, B]

groupMap is equivalent to groupBy(key).mapValues(_.map(f)).

groupMapReduce is equivalent to groupBy(key).mapValues(_.map(f).reduce(g)).

Mutable collections now have transformation operations that modify the collection in place:

def mapInPlace(f: A => A): this.type
def flatMapInPlace(f: A => IterableOnce[A]): this.type
def filterInPlace(p: A => Boolean): this.type
def patchInPlace(from: Int, patch: scala.collection.Seq[A], replaced: Int): this.type

Other new operations are distinctBy and partitionMap

def distinctBy[B](f: A => B): C // `C` can be `List[Int]`, for instance
def partitionMap[A1, A2](f: A => Either[A1, A2]): (CC[A1], CC[A2]) // `CC` can be `List`, for instance

Last, additional operations are provided by the scala-collection-contrib module. You can think of this artifact as an incubator: if we get evidence that these operations should be part of the core, we might eventually move them.

The new operations are provided via an implicit enrichment. You need to add the following import to make them available:

import strawman.collection.decorators._

The following operations are provided:

  • Seq
    • intersperse
  • Map
    • zipByKey / join / zipByKeyWith
    • mergeByKey / fullOuterJoin / mergeByKeyWith / leftOuterJoin / rightOuterJoin

Are there new implementations of existing collection types (changes in performance characteristics)?

The default Set and Map are backed by a ChampHashSet and a ChampHashMap, respectively. The performance characteristics are the same but the operation implementations are faster. These data structures also have a lower memory footprint.

mutable.Queue and mutable.Stack now use mutable.ArrayDeque. This data structure supports constant time index access, and amortized constant time insert and remove operations.

How do I cross-build my project against Scala 2.12 and Scala 2.13?

Most usages of collections are compatible and can cross-compile 2.12 and 2.13 (at the cost of some warnings, sometimes).

If you cannot get your code to cross-compile, there are various solutions:

  • You can use the scala-collection-compat library, which makes some of 2.13’s APIs available to 2.11 and 2.12. This solution does not always work, for example if your library implements custom collection types.
  • You can maintain a separate branch with the changes for 2.13 and publish releases for 2.13 from this branch.
  • You can put source files that don’t cross-compile in separate directories and configure sbt to assemble the sources according to the Scala version (see also the examples below):

    // Adds a `src/main/scala-2.13+` source directory for Scala 2.13 and newer
    // and a `src/main/scala-2.13-` source directory for Scala version older than 2.13
    unmanagedSourceDirectories in Compile += {
      val sourceDir = (sourceDirectory in Compile).value
      CrossVersion.partialVersion(scalaVersion.value) match {
        case Some((2, n)) if n >= 13 => sourceDir / "scala-2.13+"
        case _                       => sourceDir / "scala-2.13-"
      }
    }
    

Examples of libraries that cross-compile with separate source directories:

  • https://github.com/scala/scala-parser-combinators/pull/152
  • https://github.com/scala/scala-xml/pull/222
  • Some other examples are listed here: https://github.com/scala/community-builds/issues/710

Collection Implementers

To learn about differences when implementing custom collection types or operations, see the following documents:

Contributors to this page: