If you are working with DynamoDB with Scala, chances are you will spend at least one-day implementing transformation on DynamoDB attribute value to a Scala model. Creating a function that transforms Java HashMap to Scala is not the problem - the problem is transforming the domain model to attribute value.
Imagine if you have a case class like this:
You want to transform
Foo into an attribute value map type.
Now you see that this structure is a little bit hairy. This is a lucky day where you only have 3 attributes in your algebra. How about if you have 12 attributes?
Doesn’t look at this make you frustrated? I know I did.
I know what you think, “why is simple DynamoDB model transformation in Scala so hard to achieve?”
There are a wide variety of libraries out there that helps transform DynamoDB attribute value into Scala model. Scanamo is one popular one, which helps you do an automated derivation transformation from attribute value to your domain model. However, great power also comes with great responsibility.
Why not use Scanamo?
Automatic derivation requires
Encoder/Decoder type classes. This approach can cause inflexibility since your database model schema may often change over time. It can easily break the format of the derivations. Like Circe, you will need to duplicate the same line of code of encoder and decoder, so using automatic derivation in your encoder/decoder logic increases redundancy in your codebase.
“As a result, people abuse automatic derivation, and end up with either ugly serialized data and nice code model, or nice serialized data and an ugly code model.” - Dynosaur documentation.
With Dynosaur, you have the flexibility to define your data based on the schema in a declarative way (you’ll see what I mean by that) rather than the traversal of a low-level representation.
We will use Dynosaur use a schema definition approach to transform the Scala domain model into DynamoDB AttributeValue.
I haven’t found an article which can read through (for example, while I’m commuting), especially those that can give me a good flow in learning Dynosaur step by step in various scenarios. That’s why I’m writing this blog.
I hope that this article can ease your scala application’s integration with DynamoDB.
Basics of start writing your Scala Domain model with Dynosaur schema
Dynosaur centered around schema. Once we create our schema, we can expect to transform that schema into a
DynamoValue is just a thin wrapper for DynamoDB Attribute. The primary goal of this is to create easy interoperability with the AWS SDK.
Let’s say we want to define a domain model like the below:
First, create your schema:
Then, once you create your schema you can try to transform this schema to AttributeValue:
Lastly, reading the
AttributeValue into the
Invoice schema by using
Sounds simple, eh? Let’s check out some gotchas and tips for working with this library.
Tips on Working with Dynosaur
Most of the tips on working with Dynosaur revolve around schema creation. Before we start, let’s create a domain model. Imagine that we want to encode and decode invoices in our system.
Let’s say we have an
orderCode, and a
In the metadata field, there is
region , and
Now, let’s make it more complicated where there is the status of the invoice.
Then, let’s assume that you can update the value of the database if the invoice has been received by the other services. We will make it an ADT of
This is what the model overall looks like:
Creating Schema with Regular Case Class
Creating a simple case class schema will use
Realize there is an error that Dynosaur cannot identify the schema of
It means Dynosaur didn’t provide that schema definition out of the box.
Since we use the AWS region type, we need to create the
Region schema, and we want the
Region value in our scala case class to transform into a
String value in DynamoDB.
imap represents the definition of bi-directional transformation between
We want to keep a specialized representation of
Region in our code and represent
region as a
String in DynamoDB.
Now, we can provide the
regionSchema in our
Now, let’s define the
Received ADT schema.
Define Enum Fields
Let’s define the
Received Coproduct. In this case, we will use
imapErr. The difference between
imap is that
imapErr provides a case where encoding cannot fail, but decoding can.
For example, transforming between
Received ADT to string will not fail, but decoding from a string to
Received ADT may cause a parsing error.
First, we need to create a transformation for
imapErr takes in two-parameter, one for decoding and the other for encoding.
String schema to decode the
String type to the
Received coproduct may cause errors.
toRight will transform the
Option type into an
The second argument is to encode
Received coproduct into a
Define Coproduct Schema
This is where things get very interesting and sometimes tricky. The documentation mentioned using
oneOf to define the coproduct schema. We use the
Status model from the example above as a use case.
Prism internally to choose which model for encoding or decoding.
For encoding, it pattern match to transform the model into the DynamoValue. Decoding checks whether the coproduct actually matches a given case, and if there is one successful one, it will choose that one to decode. If none of them matches, it will simply throw an error.
Let’s look at how we can do define
It looks like it is okay, but there is a possibility of ambiguity. As the document mentioned, use discriminator keys and fields for solving ambiguity.
However, one of the gotchas is to add a safeguard by adding a discriminator key and field on the coproduct to differentiate between two case classes.
Since the schema definition uses
|+|, which serves as a combination for monoid but an
orElse. It will decode whatever matches that schema definition. In the case of
Status coproduct, if we want all changes to be written to DynamoDB and have an attribute that determines which status the current value, having another constant field, such as
type that updates the current status. We need to create a constant in our schema to differentiate the model.
Another scenario is defining all optional fields in a product and wanting to have that value as a coproduct.
Decoding all optional fields in the coproduct will require discriminator keys to solve the ambiguity. Either by using a
tag or a
Let’s finish up with this model creation. if you want to create a schema for
Invoice, you can just create a
record of schema:
That’s the end of this blog post. If there’s a TLDR from this, they are
- Define discriminator keys to solve ambiguity as a safeguard in coproduct Imap is a transformation that won’t cause any error, and ImapErr is a transformation that there is a possible error in the decoding portion.
- Remember to set the most specific value first on the
|+|on coproduct because it serves like an
orElseor pattern matching. If you catch all the schema at the beginning of
|+|, you won’t get the desired schema.
I hope that you enjoy this post, and any feedback is welcome. How do you usually decode the domain model to DynamoDB Attribute? Comment down below!