(De)serialization in SQLAlchemy

am 23. 08. 2020 von Martin Ennemoser
Flask
Python
Serialization
SqlAlchemy

Motivation

Data that is sent or received by web applications is usually transferred via the HTTP protocol. The data is sent as byte array and therefore needs to be transformed to a standard format like JSON or XML. The transformation of programming language specific objects to byte representations is called serialization. The inverse operation is called deserialization and transforms  a byte representation back into a programming language specific object.
In this article I want to show how database entities, which are represented as Python objects using SQLAlchemy, can be (de)serialized. I will show that it is not hard to implement ones own serializer classes. The procedure will be explained by means of a simple Flask application.

(De)serialization in Django

Most of the available web frameworks contain libraries that have support for (de)serialization. Django for example has serialization classes which are accessible via the Django Rest Framework. Especially helpful is the class ModelSerializer: Inheriting from this class allows the (de)serialization of model instances without having to specify all fields of the model. The serialization class automatically detects all fields in the model class and maps these fields on the serialization class.

(De)serialization in Flask

Flask supports libraries for this feature as well. However, since Flask is a micro framework with minimal functionality out of the box, the necessary extensions have to be installed subsequently, after project initialization.
In the following, I want to show how automatic (de)serialization can be achieved in Flask. I use SQLAlchemy as ORM and Marshmallow as (de)serialization engine. Since Marshmallow does not support SQLAlchemy out of the box, the installation of the extension marshmallow-sqlalchemy is necessary.

Example application

A very simple Flask application, which encompasses some REST endpoints and two model classes, is used for demonstration. The model Company represents companies, the model Product represents product articles which are mapped on the database. Products can be assigned to companies by using a foreign key relationship.

Products consist basically of a name, a description and a price. Additionally, a discount can be assigned if needed. A database model like this is typically used in webshops, of course in a much more sophisticated form.

The Product model can be modified via GET/PUT/POST requests as shown below.

The Python models are implemented like this:

class Company(db.Model, CreatedModifiedMixin):
    id = db.Column(UUIDType(binary=False), primary_key=True, default=uuid.uuid4)
    company_name = db.Column(db.String(255), nullable=False, unique=True)

    def __repr__(self):
        return "<Company {} ({})".format(self.id, self.company_name)


class Product(db.Model, CreatedModifiedMixin):
    id = db.Column(UUIDType(binary=False), primary_key=True, default=uuid.uuid4)
    name = db.Column(db.String(255), nullable=False, unique=True)
    description = db.Column(db.String(255), nullable=False, unique=True)
    price = db.Column(db.String(255), nullable=False, unique=True)
    discount = db.Column(db.String(255), nullable=False, unique=True)

    # relationships
    company_id = db.Column(
        UUIDType(binary=False), db.ForeignKey("company.id"), nullable=False
    )
    company = db.relationship("Company", backref=db.backref("products", lazy=True))

    def __repr__(self):
        return "<Product {} ({})".format(self.id, self.name)

The following REST API was implemented with Flask-RESTful.
The class ProduktCreateView allows the creation of new products:

from flask_restful import Resource

class ProductCreateView(Resource):
    def post(self):
        product_data = request.get_json(force=True)
        schema = ProductSchema()
        try:
            new_product = schema.load(product_data, session=db.session)
            db.session.add(new_product)
            db.session.commit()
        except ValidationError as err:
            return err.messages, 400

        return 200

Upon making an HTTP POST request, the product data is saved in the Python dictionary product_data. The ProductSchema instance deserializes the dictionary to a new object of the Product model. Subsequently, this object will be saved to the database using SQLAlchemy. When deserializing, errors can occur when performing validation. For example, if the product description does not contain a valid string, a ValidationError is raised. In this case, the data is not written to the database and an error description will be returned to the client.

The ProductSchema-Klasse inherits from SQLAlchemyAutoSchema. This class allows the (de)serialization of product entities. The “auto” in the name denotes the automatic mapping of database fields on the serialiation class. The Meta class must include the model which is used for mapping. ProductSchema instances  know thereby which database fields of the model class can be mapped on. load_instance must be set to True in our case so that product data can be deserialized to model instances.

If load_instance was set to False, the schmea.load() call would throw an exception. include_fk means that we want to save foreign keys as well.

We want to exclude some fields intentionally from deserialization. We can achieve this by setting the load_only parameter. The field discount for example can be deserialized but not serialized.  Such fields are called write-only. The pendant to load_only is dump_only. Fields which are declared as dump_only can be serialized but not deserialized and are therefore read-only.

class ProductSchema(SQLAlchemyAutoSchema, FilteredSchema):
    class Meta:
        model = Product
        load_instance = True
        include_fk = True

    discount = auto_field(load_only=True)

The second view class ProductEntityView allows alteration and retrieval of single products:

class ProductEntityView(Resource):
    def get(self, id):
        product = db.session.query(Product).filter_by(id=id).first()
        schema = ProductSchema(many=False)
        return schema.dump(product), 200

    def put(self, id):
        product = db.session.query(Product).filter_by(id=id).first()
        product_data = request.get_json(force=True)
        try:
            schema = ProductSchema()
            schema.load(
                product_data, instance=product, session=db.session, partial=True
            )
        except ValidationError as err:
            return err.messages, 400
        return 200

ProductEntityView employs ProductSchema in the same way as in ProductCreateView. In get(), all matching products will be retrieved from the database and serialized with schema.dump(). With the many=False parameter, the schema class knows that only one instance has to be serialized. If a database query retrieves multiple product objects which should be serialized in the following, many=True must be passed to the schema initializer.

put() is responsible for changing single products. schema.load() takes the new product data and updates the instance which was loaded from the database. partial=True means that only a part of the product entity will be updated and not the whole object. Same as with post(), a ValidationError is thrown if the data from the request is not valid.

Conculsion

(De)serialization of database objects, which are accessed by SQLAlchemy, can be performed easily by employing the class SQLAlchemyAutoSchema from the marshmallow-sqlalchemy extension. The advantage of this class is that no complicated mapper classes have to be written. The mapping happens automatically thanks to marshmallow. The only two challenges are the proper configuration of the serialization class and the passing of the correct parameters to schema.load() and schema.dump(). However, this article hopefully showed that this is also not too hard to get right.

 

The source code for this article is available here.

Subscribe
Notify of
guest
0 Comments
Inline Feedbacks
View all comments
Flask