Circe has been the go-to parsing Json Library in Scala. The power of Circe lies in the fact that it can polymorphically derive Json String to an ADT. However, I experienced frustration the first time using Circe - partly because I was new Scala as a Programming Language and touched into the world of Functional Programming. Sometimes, the error messaged is not transparent, or there is some specific configuration that needs to go through the source code to achieve particular goals.

As I developed more and more applications with Scala, and gain more understanding in Functional Programming Paradigm, I want to share all the gotchas that I encountered while parsing Json with Circe. These are the use cases that I met in my workplace and how I solved them.

Encoding/Decoding Coproduct (Sum) Type in an ADT

Sometimes we want to decode a coproduct in ADT with a different string representation than the coproduct type.

For example, we have 4 Houses in the Hogwarts houses in Harry Potter.

We want to model the four houses as a coproduct type as an ADT, and be able to encode/decode the respective case object polymorphically:

case class House(type : HouseType)

sealed trait HouseType
case object GodricGryffindor extends Houses
case object SalazarSlyntherin extends Houses
case object RowenaRavenclaw extends Houses
case object HelgaHufflepuff extends Houses


However, after talking with other teams about the contract, they decide to send the Houses types as a Snake_Case.

"Godric_Gryffindor" => GodricGryffindor


If we use circe.generic.semiauto.{deriveEncoder,deriveDecoder}, the result of the JSON type will be GodricGryffindor.

{
"type" : "GodricGryffindor"
}


First, define the encoder and decoder instances for House.

import io.circe.generic.semiauto.{deriveEncoder, deriveDecoder}

implicit val houseEncoder:encoder[Houses] = deriveEncoder
implicit val houseDecoder:encoder[Houses] = deriveDecoder


Doing a quick transformation on the incoming Json and convert them to one of the case object.

 implicit val housesEncoder: Encoder[HouseType] = (obj: HouseType) => obj match {
case HelgaHufflepuff => Json.fromString("Helga_Hufflepuff")
case RowenaRavenclaw => Json.fromString("Rowena_Ravenclaw")
case GodricGryffindor => Json.fromString("Godric_Gryffindor")
case SalazarSlyntherin => Json.fromString("Salazar_Slyntherin")
}

implicit val housesDecoder: Decoder[HouseType] = (hcursor:HCursor) => for {
value <- hcursor.as[String]
result <- value match {
case "Helga_Hufflepuff" => HelgaHufflepuff.asRight
case "Rowena_Ravenclaw" => RowenaRavenclaw.asRight
case "Godric_Gryffindor" => GodricGryffindor.asRight
case "Salazar_Slyntherin" => SalazarSlyntherin.asRight
case s => DecodingFailure(s"Invalid house type ${s}", hcursor.history).asLeft } } yield result  We can achieve a difference in the’ type’ field member with a little work on defining the encoder and decoder the Coproduct. val gryffindor = (Houses(type = GodricGryffindor)).asJson println(gryffindor.spaces2) // { // "type" : "Godric_Gryffindor" // }  Transforming EpochMillis to Instant in a Product Type ADT You want to create a Currency class that has a field of createdDate. Sample Currency class: case class Currency(id: Int, name:String, description:String, isoCodeAlphabetic:String, createdDate:Instant)  The JSON String pass createdDate as an EpochMillis. However, you want to convert it to an Instant to make it easier to do any operation on the createdDate. Sample Currency JSON String: { "id" : 1, "name" : "US Dollars", "description" : "United States Dollar", "isoCodeAlphabetic" : "USD", "createdDate" : 1595270691417 }  Create another encode/decode instance if you want to transform specific members of the case class in Circe. You just need to create another instance in the implicit scope for converting from Long, EpochMillis to Instant.  implicit val encoder:Encoder[Instant] = Encoder.instance(time => Json.fromLong(time.toEpochMilli)) implicit val decoder:Decoder[Instant] = Decoder.decodeLong.emap(l => Either.catchNonFatal(Instant.ofEpochMilli(l)).leftMap(t => "Instant"))  Then, create an encoder/decoder for Currency: implicit val encoderCurrency: Encoder[Currency] = deriveEncoder implicit val deoderCurrency: Decoder[Currency] = deriveDecoder  Circe will look at the implicit scope to check if there is an encoder/decoder instance from one value to another. With implicit resolution, Circe can derive from one type to another if you provide the encoder/decoder instance of that type. Encode/Decode Polymorphic ADT Let’s define the JSON String that you want to derive in this use case: { "houseType" : { "type" : "Rowena_Ravenclaw", "characteristics" : [ "Loyal" ], "animalRepresentation" : "eagle" }, "number" : 12 }  And we want to convert it to: House(RowenaRavenclaw(List(Loyal),eagle),12)  Noticed that the type indicating what constructor names you want the JSON string to transformed it to (in this case, it is RowenaRavenclaw). Decoding with a regular CirceDecoder will return the case class below. House(houseType(type: "Rowena_Ravenclaw", List(Loyal),eagle),12)  How would you polymorphically decode a JSON String by matching the member of its field to the constructor name? There are two ways. The first will be regular encode and decode, and the second will use Circe.extras. Regular Encoding and Decoding The way that you structure the ADT makes a huge difference. To explain the workaround of the use case above, I will be using @JsonCodec to derive encoder and decoder for a regular case class automatically. Defining the model type for House and HouseType: @JsonCodec case class House(houseType: HousesTypes, number:Int) trait House object House { @JsonCodec case class GodricGryffindor(characteristics:List[String]) extends HousesTypes object GodricGryffindor{ val typeId: String = "Godric_Gryffindor" } case object SalazarSlyntherin extends HousesTypes { val typeId: String = "Salazar_Slyntherin" } @JsonCodec case class RowenaRavenclaw(characteristics:List[String], animalRepresentation:String) extends HousesTypes object RowenaRavenclaw{ val typeId: String = "Rowena_Ravenclaw" } @JsonCodec case class HelgaHufflepuff(animalRepresentation:String, colours:String) extends HousesTypes object HelgaHufflepuff{ val typeId: String = "Helga_Hufflepuff" } }  In the above ADT definition, we want to have an implicit encoder and decoder for the HouseType. During encoding specific HouseType, we want to append a type field to the JSON String. { "houseType" : { "type" : "Rowena_Ravenclaw", << - We want to append this based on the specific HouseType "characteristics" : [ "Loyal" ], "animalRepresentation" : "eagle" }, "number" : 12 }  Circe encoding instance:  implicit val encoder:Encoder[HousesTypes] = { // deepMerge - insert the encoded Json with another field type // Basically overriding the current encoder with the type case obj: GodricGryffindor => obj.asJson deepMerge(Json.obj("type" -> Json.fromString(GodricGryffindor.typeId))) case obj: RowenaRavenclaw => obj.asJson deepMerge(Json.obj("type" -> Json.fromString(RowenaRavenclaw.typeId))) case obj: HelgaHufflepuff => obj.asJson deepMerge(Json.obj("type" -> Json.fromString(HelgaHufflepuff.typeId))) case obj: HousesTypes => Json.obj("type" -> Json.fromString(SalazarSlyntherin.typeId)) }  We want to retrieve the type field in the houseType JSON string and decode the entire Json String based on that type during decoding. For instance, Rowena_Ravenclaw will point to RowenaRavenclaw case class.  implicit val decoder:Decoder[HousesTypes] = (cursor:HCursor) => for { tpe <- cursor.get[String]("type") result <- tpe match { case GodricGryffindor.typeId => cursor.as[GodricGryffindor] case RowenaRavenclaw.typeId => cursor.as[RowenaRavenclaw] case HelgaHufflepuff.typeId => cursor.as[HelgaHufflepuff] case SalazarSlyntherin.typeId => SalazarSlyntherin.asRight case s => DecodingFailure(s"Invalid house type${s}", cursor.history).asLeft
}
} yield result


Using Circe.extras

Circe has a specific library, circe.extras, that solved encoding/decoding polymorphic ADT.

First, let’s re-write our model by changing the HouseType to a sealed trait:

sealed trait HouseType {
def type: String
}

object HouseType {
case class GodricGryffindor(characteristics:List[String]) extends HouseType {
override def type: String = "Godric_Gryffindor"
}
case object SalazarSlyntherin extends HouseType {
override def type: String = "Salazar_Slyntherin"
}
case class RowenaRavenclaw(characteristics:List[String], animalRepresentation:String) extends HouseType {
override def type: String = "Rowena_Ravenclaw"
}
case class HelgaHufflepuff(animalRepresentation:String, colours:String) extends HouseType {
override def type: String = "Helga_Hufflepuff"
}
}


You can set the type in the JSON String as a discriminator, indicates the constructor, in a configuration, and declare the configuration implicitly.

implicit val houseTypeConfig = Configuration.default.withDiscriminator("type").copy(
transformConstructorNames = {
case "GodricGryffindor" => "Godric_Gryffindor" // from type on the right transform the case changes to the left
case "SalazarSlyntherin" => "Salazar_Slyntherin"
case "RowenaRavenclaw" => "Rowena_Ravenclaw"
case "HelgaHufflepuff" => "Helga_Hufflepuff"
}
)


I am setting the configuration to transform the constructor names. I want to convert the entire JsonString based on one of the field type. The JSON field type contains the value on the right side of the case statement (“Godric_Gryffindor”, “Salazar_Slyntherin”, …). I want to transform the JSON string to the case class on the left side of the case statement (“GodricGryffindor”, “SalazarSlyntherin”). The left side of the case statement will match the model that we defined above.

Then, during the encoding/decoding process, use deriveConfiguredEncoder and deriveConfiguredDecoder in circe.generic.extras to polymorphically enode/decode the Json string:

 implicit val house2Encoder = {
implicit val config = houseTypeConfig
deriveConfiguredEncoder[HouseType]
}

implicit val house2Decoder = {
implicit val config = houseTypeConfig
deriveConfiguredDecoder[HouseType]
}


Testing and running the above command:

val ravenClaw: HouseType = RowenaRavenclaw(characteristics = List("Loyal"), animalRepresentation = "eagle")
val ravenClawJson = ravenClaw.asJson
val ravenClawStr = ravenClawJson.noSpaces
println(ravenClaw.asJson.spaces2)
println(decode[HouseType](ravenClawStr).right.get)


Takeaway

• You can encode and decode specific Json field by providing an encoder/decoder instance of those types. Circe leverage the compiler implicit resolution to transform the JsonString to the desired ADT.
• Use CirceExtra to encode/decode Coproduct type ADT by using circe.generic.extras.Configuration.
• Use deepMerge to merge one JSON object to another and inject any specific fields that you desired.

That’s it for encoding and decoding an ADT in Circe!

All the source codes are here.

Subscribe

* indicates required

Related Posts

Create a Throttling System for Any Application with no more than 100 lines of Code

With the help of functional programming and FS2

How to Write Data to a File with FS2

A More Performant Function