By   July 4, 2015

Serializing Custom Types for MongoDB using Salat

MongoDB stores documents in a Json like format, called BSON. Before inserting a document to MongoDB, it has to be converted into Bson. In this post, we are going to discuss how we can use Salat to convert a document into MongoDB’s compatible Bson. We will also see how we can deserialize a MongoDB document to your custom type. Here we will be focusing on Salat for such conversion.

Let’s first create a simple Scala SBT project. Here we are adding dependency for Casbah to interact with MongoDB. In order to serialize / de-serialize objects before sending them over to MongoDB. We need to add the SonaType resolver for Salat. Since we will be using DateTime, I have also included dependencies for Joda’s library.

Let’s create a type Student. It has only three fields: id (Int), name (String), admissionDate ( Joda’s DateTime).

MongoDB Installation

We first need to install MongoDB. Here we are downloading it from the official Mongo’s download page. It’s version 3.0.4. This is a packaged installer which installs it in program file directory.

Download MongoDB

Download MongoDB

After installation is completed, just go to the installation directory and run mongod.exe. This runs the database system on port 27017 by default.

You would also need a tool to view data stored in Mongo database. Here we are using MongoVue. It’s an amazing tool. There are also other useful tools for the same use case, RoboMongo is another very useful tool.

Mongo Vue

Mongo Vue

Using Casbah with Salat

Casbah is an official driver from MongoDB to interact from Scala. Here we are using Casbahs’s API to connect to mongo DB. We can specify the database and collection of interest. If either database or collection doesn’t exist, it would be created for us from MongoDB.

Casbah requires MongoDB document to insert into its collections. So the objects of our custom types must be converted into this before being inserted into mongoDB’s collections. Salat provides grater[T] for such conversion. As you can see, this is a generic type. Here T is a custom type being converted here.

As we run the code, we get the following exception. Since Joda’s DateTime is also a custom type, there is no apparent default support for this.

Joda's conversion failure

Joda’s conversion failure

Adding Support for Joda DateTime Serialization

In order to add support for Joda’s DateTime, there is already some serializer available from Casbah. It just provides some additional serializer for this purpose. Just see the updated import statements and method calls to register the serializers.

Now let’s run the code, we can see that the code successfully runs. It converts Student object to corresponding bson document, which is then inserted to Mongo DB. Just look at how successfully it has kept all the data types including Mongo’s type to hold data for dates.

Inserted Document

Inserted document

Deserializing Queried data from MongoDB

We can also read this data from the database and deserialize it back using the grater’s asObject method.

Just debug the code, here $lte from casbah is added. Since we are running the code at a later time with DateTime.now(), this should include data previously inserted. Just to make sure that we have the correct data, we are viewing in Evaluate Expression window.

Deserializing queried data

Deserializing queried data

Limitations about Embedded Case Classes

Let’s get a little more excited and start keeping data about student’s addresses. We might need to send them notifications about their dues from time to time. Here we have added StudentAddress type, plus we have added a field of the same type to Student.

Again, grater[T] is used for serialization. Here we have to use asObject method.

As we open the result in Evaluate Expression window after deserialization, Salat seems to be successfully able to deserialize it back to the same object. It has converted back even the StudentAddress based field, which seems pretty impressive.

nested case class serialization

nested case class serialization

Now let us update our code, to insert this to MongoDB. Here we are inserting the same object to mongo database. We are retrieving the result from mango using the same query code. Later we are trying to deserialize the result back to Student type using grater.

Just set a break point before after inserting the document to database. The weird thing is that, instead of sub-document, it has inserted StudentAddress based field as an array. It also has lost names of keys. This is bad. How the hell, Salat would be converting it back to the same Student object.

Embedded case class insert

Embedded case class insert

And it doesn’t. As we try to convert it back, it throws a big exception on our face.

exception while deserialization

exception while deserialization

This is painful as Salat’s documentation clearly states that embedded case classes are supported.

embedded case class support

embedded case class support

But we were able to convert it back to Student before. Apparently, Salat just didn’t understand address as an embedded document. It just keeps the info this info. While converting back, it just passes back from the same data. When it inserts data to mongo db, it just passes this confusion to casbah and let it handle the way it wants. When casbah reads it back, it just passes the way it save back to Salat. Now there is no way Salat has info about converting this array to StudentAddress, hence the exception. This theory is based on the following from Salat’s documentation on github.

embedded_class_to_casbah

Fixing Embedded Case Class Issue

Actually the issue is not the end of the world. Actually Salat does support this. It is easier to fix this. We just need to specify the package for case classes.

Now it correctly inserts the embedded case class correctly. It inserts it as embedded document. This is picked up by casbah, when queried. Salat now correctly deserializes the object to Student type.

Salat Embedded document

Salat Embedded document

I actually found the solution here:
https://groups.google.com/forum/#!topic/scala-salat/bNJeN0vA6Pc

Using Salat in Play Framework
In play framework a little bit of setup required in the code:

Just add the following code in the context:


RegisterConversionHelpers()

implicit val ctx = new Context {
    val name = "Custom_Classloader"
}

I have these import statements in my code:


import com.mongodb.casbah.Imports._
import com.mongodb.casbah.commons.conversions.scala.RegisterConversionHelpers
import org.joda.time.DateTime
import com.novus.salat._
import com.novus.salat.global._

In the absence of the above setup, you would get this error:


[GraterGlitch:

GRATER GLITCH – unable to find or instantiate a grater using supplied path name

REASON: Very strange! Path='Student' from pickled ScalaSig causes ClassNotFoundException

Context: 'global'
Path from pickled Scala sig: 'Student'


]