Functional testing in an environment of Flask micro-services

You can find the source code written in this article in the flask-boilerplate that we use at Theodo.

Functionally Testing an API consists in calling its routes and checking that the response status and the content of the response are what you expect.

It is quite simple to set up functional tests for your app. Writing a bunch of HTTP requests in the language of you choice that call your app (that has previously been launched) will do the job.

However, with this approach, your app is a blackbox that can only be accessed from its doors (i.e. its URLs). Although the goal of functional tests is actually to handle the app as a blackbox, it is still convenient while testing an API to have access to its database and to be able to mock calls to other services (especially in a context of micro-services environment).

Moreover it is important that the tests remain independent from each other. In other words, if a resource is added into the database during a test, the next test should not have to deal with it. This is not easy to handle unless the whole app is relaunched before each test. Even if it is done, some tests require different fixtures. It would be tricky to handle.

With this first approach, our functional tests were getting more complex than the code they were testing. I would like to share how we improved our tests using the flask test client class.

You don’t need to know about flask/python to understand the following snippets.

The API allows to post and get users. First we can write a route to get a user given its id:

# src/route/user.py
from model import User

# When requesting the URL /user/5, the get_user_by_id will be executed with id=5
@app.route('/user/<int:id>', methods=['GET'])
def get_user_by_id(self, id):
    user = User.query.get(id)
    return user.json  # user.json is a dictionary with user data such as its email

This route can be tested with the flask test client class:

# test/route/test_user.py
import unittest
import json

from server import app
from model import db, User

class TestUser(unittest.TestCase):

    # this method is run before each test
    def setUp(self):
        self.client = app.test_client()  # we instantiate a flask test client

        db.create_all()  # create the database objects
        # add some fixtures to the database
        self.user = User(
            email='joe@theodo.fr',
            password='super-secret-password'
        )
        db.session.add(self.user)
        db.session.commit()

    # this method is run after each test
    def tearDown(self):
        db.session.remove()
        db.drop_all()

    def test_get_user(self):
        # the test client can request a route
        response = self.client.get(
            '/user/%d' % self.user.id,
        )

        self.assertEqual(response.status_code, 200)
        user = json.loads(response.data.decode('utf-8'))
        self.assertEqual(user['email'], 'joe@theodo.fr')

if __name__ == '__main__':
    unittest.main()

In these tests:

  • all tests are independent: the database objects are rebuilt and fixtures are inserted before each test.
  • we have access to the database via the db object during the tests. So if you test a ‘POST’ route, you can check that a resource has been successfuly added into the database.

Another benefit is that you can easily mock a call to another API. Let’s improve our API: the get_user_by_id function will call an external API to check if the user is a superhero:

# src/client/superhero.py
import requests

def is_superhero(email):
    """Call the superhero API to find out if this email belongs to a superhero."""
    response = requests.get('http://127.0.0.1:5001/superhero/%s' % email)
    return response.status_code == 200
from client import superhero
# ...
@app.route('/user/<int:id>', methods=['GET'])
def get_user_by_id(self, id):
    user = User.query.get(id)
    user_json = user.json
    user_json['is_superhero'] = superhero.is_superhero(user.email)
    return user_json

To prevent the tests from depending on this external API, we can mock the client in our tests:

# test/route/test_user.py
from mock import patch
#...

@patch('client.superhero.is_superhero')  # we mock the function is_superhero
def test_get_user(self, is_superhero_mock):
    # when is_superhero is called, it returns true instead of calling the API
    is_superhero_mock.return_value = True

    response = self.client.get(
        '/user/%d' % self.user.id,
    )
    self.assertEqual(response.status_code, 200)
    user = json.loads(response.data.decode('utf-8'))
    self.assertEqual(user['email'], 'joe@theodo.fr')
    self.assertEqual(user['is_superhero'], True)

To use this mock for all tests, the mock can be instantiated in the setUp method:

# test/route/test_user.py
def setUp(self):
    #...
    self.patcher = patch('client.superhero.is_superhero')
    is_superhero_mock.return_value = True
    is_superhero_mock.start()
    #...

def tearDown(self):
    #...
    is_superhero_mock.stop()
    #...

Conclusion

With the Flask test client, you can write functional tests, keep control over the database and mock external calls. Here is a flask boilerplate to help you get started with a flask API.


You liked this article? You'd probably be a good match for our ever-growing tech team at Theodo.

Join Us

  • Benjamin Soulas

    Hello, have you tested the fact to pass parameters in a get request?
    here an example :
    I have a server with URI (/api/get-status) which take 2 parameters, an hostname and a containername.
    In this server, I use an object I developped that make the processing of it (flaskManager to name it)

    so my server looks like this :

    @app.route('/api/get-status', methods=['GET'])
    def getStatus():

    return flaskManager.getStatus()

    And my fonction called into the flaskManager object looks like this :


    def getStatus(self):
    """Return a status for a given container for a given hostname

    :return: return the state of the container
    """
    # URL example : http://192.168.1.67:5000/api/get-status?hostname=conteneur1&container=conteneur-flask

    resp = {"code" : 0}

    try:
    # Get the url parameter specifying the hostname we want
    hostname = request.args.get('hostname', '')

    # Get the url parameter specifying the hostname we want
    container = request.args.get('container', '')

    except Exception, e:
    resp["code"] = -1
    resp["message"] = "MANAGER GETSTATUS : Wrong parameters received nError : " + e.message
    responseToReturn = Response(json.dumps(resp), status=400, mimetype='application/json')

    return responseToReturn

    hostIP = self.manager.getIPByHostName(hostname)

    if(hostIP == None):
    print("IP == NONE")
    resp["code"] = -1
    resp["message"] = "MANAGER GETSTATUS : Host'" + request.args.get('hostname', '') + "' not defined in your "
    "configuration file ..."
    responseToReturn = Response(json.dumps(resp), status=400, mimetype='application/json')

    return responseToReturn

    try:
    parameters = {"container" : container}

    url = "http://" + hostIP + ":5000/get-state"

    r = requests.get(url, params=parameters)

    resp["code"] = 1
    resp["data"] = json.loads(r.content)["data"]

    responseToReturn = Response(json.dumps(resp), status=200, mimetype='application/json')

    except Exception, e:
    resp["code"] = -1
    resp["message"] = "MANAGER GETSTATUS : Unable to execute request GET on url : 'http://" + str(hostIP) +
    ":5000/get-state' with parameters '" + request.args.get('hostname', '') + "' and '" +
    request.args.get('container', '') + "'"
    responseToReturn = Response(json.dumps(resp), status=400, mimetype='application/json')

    return responseToReturn

    finally:
    print("/api/get-status : " + str(responseToReturn))
    return responseToReturn

    So here is my question : I would like to test my flaskHandler with this class :


    class TestFlaskManagerHandler(TestCase):

    def setUp(self):
    self.client = app.test_client()
    self.flaskManager = FlaskManagerHandler()

    def test_get_status(self):
    reponse = self.client.get('/api/get-status')

    The problem is I don’t know how to make a get request with the test_client with parameters that are hostname and containername.

    Thanks a lot and sorry if I don’t express myself very well …

  • Nicolas Girault

    Hi Benjamin,

    Disqus allows to add “pre” and “code” html tags to make the code readable.

    Have you tried


    self.client.get('/api/get-status?hostname=blablabla&containername=blablabla')

    Or better


    self.client.get('/api/get-status', params={
    'hostname': 'blablabla',
    'containername': 'blablabla'
    })

    (I don’t have a flask environnement setup right now so I didn’t check what I say is right).

    You can check this documentation http://flask.pocoo.org/docs/0.10/testing/

    and this one http://docs.python-requests.org/en/latest/user/quickstart/#make-a-request

  • Benjamin Soulas

    Hi Nicolas,

    Indeed I didn’t try this way … I feel stupid ^^. Thank you, I’lltry that

  • Nicolas Girault

    No problem! It’s a good idea to ask stupid question! It’s a good way to learn! 😉

  • Benjamin Soulas

    You’re right ^^. Thanks once again ^^

  • Pingback: Speed up npm install with a local registry to cache packages | Theodo, développement agile symfony()

  • Nicolas Girault

    Nice article! Thanks