Flask Series: Deployment

I will describe a setup with nginx as a web server. A web server cannot communicate directly with Flask, that’s why gunicorn will be used to act as a medium between the web server and the Flask application. Gunicorn is like application web server that will be running behind nginx, it is WSGI compatible. It can communicate with applications that support WSGI – Flask, Django.

Install the needed bits.

sudo apt-get update
sudo apt-get install -y python python-pip nginx gunicorn

Create a directory to store the project

sudo mkdir /home/www && cd /home/www

Download the project from the GitHub repository and copy the application to the /home/www directory.

git clone https://github.com/damyanbogoev/flask-bookshelf.git /tmp/
cd /tmp/flask-bookshelf
cp -r ./* /home/www/

Install the application requirements:

pip install -r requirements.txt

Configure nginx

sudo /etc/init.d/nginx start

sudo rm /etc/nginx/sites-enabled/default

sudo touch /etc/nginx/sites-available/flask_bookshelf

sudo ln -s /etc/nginx/sites-available/flask_bookshelf /etc/nginx/sites-enabled/flask_bookshelf

sudo vim /etc/nginx/sites-enabled/flask_bookshelf

The gunicorn will use port 8000 and handle the incoming HTTP requests.

You could add a separate configuration for the static files of the Flask application, because it is better to be served directly by nginx.

Restart the nginx to load the later configuration changes:

sudo /etc/init.d/nginx restart

Run the gunicorn on port 8000

/home/www/flask_bookshelf/

gunicorn --bind 0.0.0.0:8000 run:app

Start a new browser instance and navigate to

http://server_address

The complete demo application, described in this blog post, can be found here.

Flask Series: Internationalization

Internationalization (i18n) enables applications and services to support multiple languages and locales. Localization (L10n) adds support for a given language and locale.

The Flask-Babel extension adds support for i18n and L10n in a Flask application. It is built based on babel, pytz and speaklater.

Flask-Babel is easy to install and configure:

pip install Flask-Babel

bookshelf/config.py

SUPPORTED_LANGUAGES – dictionary stores information about the available languages, that are supported in the application.
BABEL_DEFAULT_LOCALE – default locale to use if no locale selector is registered, default is ‘en‘.
BABEL_DEFAULT_TIMEZONE – timezone to use for user facing dates, default is ‘UTC‘.

bookshelf/babel.cfg

The babel.cfg is a mapping file, intended to be used with the pybabel command line tool. It contains configuration data for Babel to tell where to look for texts – supported files, paths, etc.

Now that you have the Flask-Babel extensions installed and configured, you should provide the babel extension with locale and timezone information. It will call localeselector and timezoneselector functions to retrieve this information, if no data is returned (None) Flask-Babel will use the default settings from the configuration.

bookshelf/__init__.py

Several callback functions are used for managing the currently used language. The url_defaults() callback function set_language_code is used to inject values into a call for url_for() automatically.

bookshelf/__init__.py

The url_value_preprocessor() registers the get_lang_code() function, which obtains and sets the language code from the request on the application globals flask.g object.

bookshelf/__init__.py

The before_request ensure_lang_support function executes before each request and it is helpful to verify if the provided language is supported by the application.

bookshelf/__init__.py

Flask allows developers to register blueprints multiple times with different URLs, which can be used to add i18n support:

bookshelf/__init__.py

The next step is to mark all of the strings for translation, by wrapping them either in gettext() or in _() function calls:

Before

After

Once marked for translation, the texts should be extracted:

pybabel extract -F bookshelf/babel.cfg -o bookshelf/messages.pot bookshelf

To create a translation for a given language run the following command:

pybabel init -i bookshelf/messages.pot -d bookshelf/translations -l bg

where the pybabel init command writes a new (bg) language catalog in the bookshelf/translations directory for the extracted data from the bookshelf/messages.pot file. The command will create a new bookshelf/translations/bg/LC_MESSAGES folder and a new messages.po file inside. The messages.po file is used by the _() and gettext() function calls. You could use the poedit translation application to translate the strings accordingly.

When the translations are done, they should be compiled:

pybabel compile -d bookshelf/translations

The complete demo application, described in this blog post, can be found here.

In the next blog post I will describe how to deploy your Flask application.

Flask Series: Healthcheck and Monitoring

One of the major parts in the application lifecycle is the monitoring. It is providing information how your application is performing, helping you easily to identify and resolve issues. Based on the application monitoring you could gather usage statistics, trigger actions, etc.

What should be monitored?
You have to consider monitoring both the web server and the Flask application.

Web Server Monitoring
– CPU utilization;
– Memory usage;
– Network traffic;
– Disk capacity and usage;

Application Monitoring
– Runtime performance;
– Runtime errors and warnings;
– Rendering;
– Persistence data querying;

There lots of monitoring tools that can be used with your Flask Application.

Zabbix
Zabbix is an enterprise open source monitoring solution for networks and applications. It is designed to monitor and track the status of various network services, servers, and other network hardware.

ELK Stack
The ELK stack combines Elasticsearch, Logstash and Kibana.

Elasticsearch is a distributed, open source search and analytics engine, designed for horizontal scalability, reliability, and easy management. It combines the speed of search with the power of analytics via a sophisticated, developer-friendly query language covering structured, unstructured, and time-series data.

Logstash is a flexible, open source data collection, enrichment, and transportation pipeline. With connectors to common infrastructure for easy integration, Logstash is designed to efficiently process a growing list of log, event, and unstructured data sources for distribution into a variety of outputs, including Elasticsearch.

Kibana is an open source data visualization platform that allows you to interact with your data through stunning, powerful graphics. From histograms to geomaps, Kibana brings your data to life with visuals that can be combined into custom dashboards that help you share insights from your data far and wide.

Flask Healthcheck
Healhcheck is a Flask application that allows developers to implement their own monitoring checks for their applications – persistence storage state, environment information, etc. I have added simple check.py module, which demonstrates how to include Healthcheck in your application. Have a look how to implement custom check functions.

The complete demo application, described in this blog post, can be found here.

In the next blog post I will describe how to implement internationalization and localization support in your Flask application.

Flask Series: Optimizations

In this blog post I will describe how to optimize your Flask application.

The website speed is very important. It could break the success of your site, no matter you provide your users great content and functionality. If your website pages load slower than your competitors’ ones, it is possible they get the attention and not your site. Especially when all of the major search engines like fast pages.

There are lots of performance measurement services you could use:

Google PageSpeed Insights;
YSlow;
WebPageTest;

The services above will provide you with tips and tricks what and how can be improved in your site. In this post I will show how Flask allows you to resolve some of the major performance issues in your website.

Compression
Most of browsers nowadays support gzip compression and negotiate for all HTTP requests. Gzip compression allows to reduce the size of the response by 70-90%. It is very easy to accomplish this in a Flask application. The Flask-Compress extension compresses the application’s response with gzip.

config.py

You could find tests that verifies the app compress configuration and if it works correctly under the tests/compress_tests.py module.

Caching
Caching data in your application allows you to reduce calls to database, additional computation, etc. The Flask-Cache extension will help you solve this problem.

cache.py

config.py

__init__.py

There are several different cache types you could use: simple, memcached, redis, filesystem, etc.. The complete list and its specific configurations can be found under the Configuring Flask-Cache section.

Now that you have the extension installed and configured you could use it in the following way:

bookshelf/main/controllers.py

bookshelf/admin/controllers.py

CDN
Content Delivery Networks (CDN) addresses the problem serving content to users all around the world. They are large distributed system deployed in multiple data centers. The Flask-CDN extension provides developers with means to serve static content from CDNs.

The extension replaces the Flask url_for function. When it is invoked from your templates the flask_cdn.url_for function is called instead.

Complete configuration list can be found under the Flask-CDN Options section.

Combine Files
Combining asset files reduces calls to the server, which leads to significant performance boost of your website. The Flask-Assets extension allows you to bundle files in your Flask application.

The complete demo application, described in this blog post, can be found here.

In the next blog post I will describe how to monitor your Flask application.

Flask Series: Security

In this blog post I will describe how to secure your Flask application.

XSS
Cross Site Scripting or XSS is an attack that tries to have your website or application load a malicious script in the browser. This script can try access user’s credentials data, get cookie information, modify settings, download bits, etc.. One way to avoid this attack is to escape text and to validate user input.

By default Flask configures Jinja2 to auto escape all values loaded in the page.

where the jinja2.ext.autoescape extension depends on

This could lead to some performance overhead. You still have to consider several scenarios where you should be careful:
– avoid generating html without Jinja2;
– avoid sending out data from uploaded files;
– avoid using the Markup class on not verified data sent by a user;
– always quote the attributes values in your templates, when Jinja expressions are used within, otherwise an attacker could easily inject either JavaScript code or CSS. More information on CSS injection;

Additional information about the XSS attacks can be found here.

CSRF
Cross-Site Request Forgery or CSRF is an attack that uses the user’s authentication credentials to execute unwanted actions.

In order to secure your application against CSRF is to use a random string and to verify it against a hidden field. The complete code snippet can be found here.

Additional information about CSRF attacks can be found here.

SQL Injection
SQL Injection is an attack where users can inject SQL commands via user input form and have them executed on the server. This SQL commands could do everything: read sensitive data, modify the database data, perform administrative tasks against the database server. Your application can be exposed to this attack when you dynamically create SQL statements (concatenating data based on user’s input), etc. By default SQL Alchemy quotes special characters – semicolons or apostrophes.

Authentication and Authorization
Authentication is a process that verifies the user’s identity, by validating his / her credential (username / email, password) against a given authority.  Authorization process verifies whether the authenticated user has access to a given resource.

Flask-Security
Flask-Security is a very helpful extensions that integrates several other Flask extensions and Python libraries:
– Flask-Login;
– Flask-Mail;
– Flask-Principal;
– Flask-Script;
– Flask-WTF;
– itsdangerous;
– passlib;

and provides out-of-the-box support for Flask-SQLAlchemy.

Note: I am using an older version of Flask-Login – 0.2.11, in order to workaround a breaking change in the extension’s API, that breaks the integration with the other security related Flask extensions.

Flask-Security uses internally a User and Role data model, that could be defined via the SQL Alchemy API.

data/models.py

The User class derives from the UserMixin Flask-Login default user implementation and same goes for the Role class – RoleMixin. You could add any aditional information to both User and Role classes if you need to.

Note: I have modified the seed.py module to seed the database with test users and roles data.

I am using SQL Alchemy for managing both User and Role objects. I have added the following configuration related to using Flask-Login with SQL Alchemy:

config.py

The complete Flask-Security configurations with their explanations can be found here.

To be compliant with the layout, I have implemented custom login and registration forms:

– templates/security/login_user.html;
– templates/security/register_user.html;

It is common to be able to access the current user in your Flask application within the Jinja templates and introduce some logic based on that data. In order to achieve that goal you could use context processors:

bookshelf/__init__.py

and you could access the user in your templates: bookshelf/templates/layout.htm

As I mentioned earlier: authorization is a process verifies whether the authenticated user has access to a given resource. Flask-Security allows developers to restrict access to a given page by specifying that a user should be either logged in or have given role(s) assigned, by decorating the view function with @login_required, @roles_required:

The complete demo application, described in this blog post, can be found here.

In the next blog post I will describe how to optimize your Flask application.