Scala lifted and direct embedding and large column count

In the latest version of Slick (2.0), there are two separate APIs named lifted embedding and direct embedding. Lifted embedding is the most stable and advised to use in production but we will look both of them here. Lifted embedding is explained in the Slick manual as:

The name Lifted Embedding refers to the fact that you are not working with standard Scala types but with types that are lifted into the scala.slick.lifted.Rep type constructor.

Direct embedding is something that exist as as an alternative to lifted but its clear that this is an experiment in Slick and will be further developed in coming versions.A database table is declared like this, the lifted syntax first and then the direct embedding syntax

Both are self explanatory, a table with the columns NAME and PRICE is declared. In the case of lifted embedding, all queries are lifted by an implicit conversion to a Rep type, in this case Rep[String] and Rep[Double]. For example a query like this

val q = coffees.filter(_.price > 8.0).map(_.name)

does lift the price, 8.0 and name to Rep[..]. This is not of the highest importance and is often transparent when working with Slick. It can however give some confusing compiler errors if you are not aware of this. Queries in direct mode are written a bit differently so lets take a look at this. There are two factory objects to execute queries against, Queryable and ImplicitQueryable. They both support a few familiar collection methods but many are missing. The ones supported are drop, filter, flatmap, length, map and take. To use either one a SlickBackEnd must be created, but with the ImplicitQueryable the backend and session objects only need to be assigned once to a query.

  db withDynSession {

    import scala.slick.direct.{SlickBackend, AnnotationMapper}

    val backend = new SlickBackend(scala.slick.driver.H2Driver, AnnotationMapper)

   

    val q1 = Queryable[Coffee]

    val q2 = q1.filter(_.price > 3.0).map(_.name)

    println(backend.result(q2.length, session) + ": " + backend.result(q2, session))

    val iq1 = ImplicitQueryable(Queryable[Coffee], backend, session)

    val iq2 = iq1.filter(_.price > 3.0)

    println(iq2.length + ": " + iq2.map(_.name).toSeq)

  }

If you plan to use the direct embedding access to the Scala compiler is required at run-time, so another dependency must be declared, for example like this in sbt:

libraryDependencies <+= (scalaVersion)("org.scala-lang" % "scala-compiler" % _)

With direct embedding there are some limitations in the 2.0 release. For example, typesafe database Inserts are not supported, something that is easily done with lifted embedding. Furthermore direct embedding only support the primitives String, Int, and Double in its column mapping.

I've used lifted embedding so far and don't see a reason yet for using direct embedding. There are some obvious limitations to its usage but it will be interesting to follow how this alternative API develop in future versions of Slick.

Another well known limitation of Slick is that when using the code-generator in a database-first scenario a different implementation is used for larger tables with more than 22 columns. I first considered to put this in a separate post but since I'm on the topic of Slick I might continue. The reason for not allowing tables with more than 22 columns are related to Scala only supporting tuples with 22 elements and less. Lets start with by using the code-generator on a table with 23 columns and see what happens. Still using a H2 database and have prepared a table with the 23 columns of type Int.

  val slickDriver = "scala.slick.driver.H2Driver"

  val jdbcDriver = "org.h2.Driver"

  val url = "jdbc:h2:test"

  val outputFolder = "."

  val pkg = ""

  scala.slick.model.codegen.SourceCodeGenerator.main(

    Array(slickDriver, jdbcDriver, url, outputFolder, pkg))

This will create a file Tables.scala. The interesting rows are these

  implicit def GetResultTable23Row(implicit e0: GR[Int]): GR[Table23Row] = GR{

    prs => import prs._

    Table23Row.tupled((<<[Int], <<?[Int], <<?[Int], <<?[Int], <<?[Int], <<?[Int], <<?[Int], <<?[Int], <<?[Int], <<?[Int], <<?[Int], <<?[Int], <<?[Int], <<?[Int], <<?[Int], <<?[Int], <<?[Int], <<?[Int], <<?[Int], <<?[Int], <<?[Int], <<?[Int]))

  }

    def * = (c1, c2, c3, c4, c5, c6, c7, c8, c9, c10, c11, c12, c13, c14, c15, c16, c17, c18, c19, c20, c22, c23) <> (Table23Row.tupled, Table23Row.unapply)

The documentation also have an explicit warning about long compilation times for tables with more than 25 columns so lets do a quick experiment with on table with 2, 23, and 200 columns and measure the time. I'll start with code-generation to see if there are performance effects. This code snippet is used to time the code generations.

The average times on my test computer are (in seconds) 0.0871, 0.0623 and 0.1222. The larger table runs longer by roughly a factor of 2 but this is expected since the generated file is larger. The respective file sizes are 2, 5 and 27 KB. Considering that the file size which grows by a factor larger than 10 (2 to 27KB) only doubles the execution time, the conclusion is that large tables does not severely impact the code-generation.
As a next step lets look at the compilation time. I create three new projects, called project2, project23 and project200 and add the generated code files for respective table size.

The project with a code file containing a table with 2 columns compiled in 2 seconds, the project with a single file with table of 23 columns compiled in 18 seconds. The project with the large table with 200 columns did not compile at all and I have yet to find a workaround for this.

Summary

The new Slick framework from Typesafe have two separate APIs for database queries, lifted embedding and direct embedding. Since the features of direct embedding are still limited I don't see any reasons yet to pick direct embedding over lifted embedding. The authors of Slick warned about slow compile time for larger table dimensions. In my test a single project containing a small table declaration was considerable faster to compile than a project containing a medium size table declaration. My test using a large table (200 columns) caused the compiler to fail, and the project didn't even load in an IDE like Eclipse. Slick is an interesting tool in the ORM/FRM (Functional Relation Mapper) landscape but it has limitations that are important to be aware of.