Python application in Satorix

A tutorial for getting a Django Python application working in Satorix. For a description of the default environment variables managed by the Satorix Dashboard and using environment variables in your application check the application environment variables article.

An application with the results of this guide can be downloaded as a Django demo on GitHub.

Using the Satorix Django package

Add Satorix to your Django application by including it in your requirements.txt with:

satorix-django

Run the pip install satorix-django command to install it.

Make sure your public Django apps are placed into a sub-folder called public/. Next, run the generator from a terminal at the root of your application:

satorix-django-config

This creates a set of files that utilize environment variables created by default with Satorix. These include the Phusion Passenger Rails app server and the Passenger built in Nginx web server. The satorix-django-config command attempts to add the import for satorix_django to your settings.py configuring the database connection for your application.

Manual Python configuration

In your application code we will be working with some files to get things running on your Satorix Hosting Cluster. Make sure your public Django apps are placed into a sub-folder called public/.

The following files are placed in the root of your application directory

Add a .gitlab-ci.yml, this is a basic setup with Safety to check your installed dependencies for known security vulnerabilities and tests Django using unittest. It is also configured to automatically deploy to your Satorix Hosting Cluster:

# We are using the Satorix base image from https://hub.docker.com/r/satorix/base/
image: 'satorix/base:18'

# Global caching directives.
cache:
  key: "$CI_PROJECT_ID"
  paths:
    - 'tmp/satorix/cache' # To cache buildpack output between runs.

variables:
  DATABASE_URL: "postgres://postgres:password@postgres:5432/test"

# Configure postgresql environment variables (https://hub.docker.com/r/_/postgres/)
.use_postgres: &use_postgres
  services:
    - postgres:11.0
  variables:
    DB_HOST: postgres
    POSTGRES_DB: test
    POSTGRES_PASSWORD: password
    POSTGRES_USER: postgres

.satorix: &satorix
  <<: *use_postgres
  script:
    - gem install satorix --no-document
    - satorix

# Safety checks your installed dependencies for known security vulnerabilities https://pyup.io/safety/
safety:
  <<: *satorix

# Test Django using unittest
django_test:
  <<: *satorix

deploy_with_flynn:
  environment:
    name: $CI_COMMIT_REF_NAME
    url: "http://$CI_PROJECT_NAME.$CI_COMMIT_REF_SLUG.$SATORIX_HOSTING_NAMESPACE"
  stage: deploy
  only:
    - staging
    - production
  <<: *satorix

Create a Procfile that defines the processes that your Satorix Hosting Cluster will run:

# This Procfile defines the types of process that Satorix will run.
# For more information, please see the documentation at https://www.satorix.com/docs

web: passenger start --port $PORT public/

If you don't already have one create a runtime.txt and specify the version of Python to use for your application:

python-3.5.7

Create a requirements.txt that is used to specify the Python package dependencies to install that are required for your application, these are installed using Pip - the Python dependency manager. We add the satorix-django helper package to set up the database connection in your Satorix Hosting Cluster:

django
satorix-django

The .buildpacks file is used to tell the Satorix Hosting Cluster the buildpacks to use for your application. We specify the Phusion Passenger buildpack to utilize features of the Satorix Dashboard. Passenger requires Ruby for some of it's utilities so is also included here:

https://github.com/heroku/heroku-buildpack-ruby
https://github.com/satorix/satorix-passenger-buildpack
https://github.com/heroku/heroku-buildpack-python

Create a .ruby-version to specify a version of Ruby to use with Passenger:

ruby-2.6.5

A blank Gemfile to allow the Ruby buildpack detection:

ruby "#{ File.open("#{ File.dirname(__FILE__) }/.ruby-version", &:gets).strip[/ruby-(.+)/i, 1] }"

source "https://rubygems.org"

A blank Gemfile.lock to allow the Ruby buildpack detection. This can also be created by running bundle install after creation of the Gemfile if Ruby is installed:

GEM
  remote: https://rubygems.org/
  specs:

PLATFORMS
  ruby

DEPENDENCIES

RUBY VERSION
   ruby 2.6.5p114

BUNDLED WITH
   1.17.3

The following files are placed in the public/ directory

At the top of your public/appname/settings.py Django settings file, add the import statement for the satorix-django package.

import satorix_django

At the end of your public/appname/settings.py Django settings file, add the following to load the environment variables to configure your database and static files using the satorix-django package.

# Settings for satorix hosting
satorix_django.settings(locals())

Create a public/passenger_wsgi.py file to set up a WSGI callable application object for Passenger. Change the satorixdjangodemo to be your application's name.

import satorixdjangodemo.wsgi
application = satorixdjangodemo.wsgi.application

A public/Passengerfile.json configuration file to set Passenger defaults for running on your Satorix Hosting Cluster.

{
  "nginx_config_template": "../config/passenger_standalone/nginx.conf.erb",
  "static_files_dir": "staticfiles",
  "log_file": "/dev/stdout",
  "environment": "production"
}

The following files are placed in a config/ directory

A config/passenger_standalone/nginx.conf.erb template to configure the built in Nginx server for Passenger. This sets up includes Django asset serving, web server level redirects, and proxy settings.

##########################################################################
#  Passenger Standalone is built on the same technology that powers
#  Passenger for Nginx, so any configuration option supported by Passenger
#  for Nginx can be applied to Passenger Standalone as well. You can do
#  this by direct editing the Nginx configuration template that is used by
#  Passenger Standalone.
#
#  Learn more about using the Nginx configuration template at:
#  https://www.phusionpassenger.com/library/config/standalone/intro.html#nginx-configuration-template
#
#  To test this configuration template run:
#  passenger start --nginx-config-template config/passenger_standalone/nginx.conf.erb --debug-nginx-config
#
#  *** NOTE ***
#  If you customize the template file, make sure you keep an eye on the
#  original template file and merge any changes. New Phusion Passenger
#  features may require changes to the template file.
##############################################################

<%
  def include_passenger_custom_template(template, indent = 0, the_binding = get_binding)
    path = File.join(File.dirname(__FILE__), 'includes', template)
    erb = ERB.new(File.read(path), nil, "-", next_eoutvar)
    erb.filename = path
    result = erb.result(the_binding)

    # Set indenting
    result.gsub!(/^/, " " * indent)
    result.gsub!(/\A +/, '')

    result
  end

  def use_canonical?
    !canonical_domain.nil? &&
    !canonical_domain.empty? &&
    !canonical_domain_protocol.nil? &&
    !canonical_domain_protocol.empty?
  end

  def canonical_domain
    ENV['SATORIX_CANONICAL_URI_HOST']
  end

  def canonical_domain_protocol
    ENV['SATORIX_CANONICAL_URI_PROTOCOL']
  end

  def canonical_uri
    "#{ canonical_domain_protocol }://#{ canonical_domain }" if use_canonical?
  end
%>

<%= include_passenger_internal_template('global.erb') %>

worker_processes 1;
events {
    worker_connections 4096;
}

http {
    <%= include_passenger_internal_template('http.erb', 4) %>

    ### BEGIN your own configuration options ###
    # This is a good place to put your own config
    # options. Note that your options must not
    # conflict with the ones Passenger already sets.
    # Learn more at:
    # https://www.phusionpassenger.com/library/config/standalone/intro.html#nginx-configuration-template

    ### END your own configuration options ###

    default_type application/octet-stream;
    types_hash_max_size 2048;
    server_names_hash_bucket_size 96;
    client_max_body_size 1024m;
    access_log off;
    keepalive_timeout 60;
    underscores_in_headers on;
    gzip on;
    gzip_comp_level 3;
    gzip_min_length 150;
    gzip_proxied any;
    gzip_types text/plain text/css text/json text/javascript
        application/javascript application/x-javascript application/json
        application/rss+xml application/vnd.ms-fontobject application/x-font-ttf
        application/xml font/opentype image/svg+xml text/xml;
  <% if @app_finder.multi_mode? %>
    # Default server entry for mass deployment mode.
    server {
        <%= include_passenger_internal_template('mass_deployment_default_server.erb', 12) %>
    }
  <% end %>
  <% @apps.each do |app| %>
    <% if use_canonical? %>
    # Redirect all requests to the canonical domain.
    server {
      server_name <%= app[:server_names].join(' ') %>;
      listen <%= nginx_listen_address(app) %> default_server;
      return 301 <%= canonical_uri %>$request_uri;
    }
    <% else %>
    # No canonical domain defined, passing all requests to the main server block.
    <% end %>

    # Main server block.
    server {
      <% app[:server_names] = [canonical_domain] if use_canonical? %>
      <%= include_passenger_internal_template('server.erb', 8, true, binding) %>

      <%= include_passenger_custom_template('django_asset_pipeline.erb', 8, binding) %>
      <%= include_passenger_custom_template('page_level_redirects.erb', 8, binding) %>
      <%= include_passenger_custom_template('proxy_configuration.erb', 8, binding) %>
      <%= include_passenger_custom_template('authentication.erb', 8, binding) %>

      ### BEGIN your own configuration options ###
      # This is a good place to put your own config options.
      # Note that your options must not conflict with the ones Passenger already sets.
      #
      # Learn more at:
      # https://www.phusionpassenger.com/library/config/standalone/intro.html#nginx-configuration-template
      #
      # You can use the include_passenger_custom_template to method include your own custom template.
      # This will help you compartmentalize your configurations, to help organize your settings.
      #
      #   Example:
      #
      #     Create a new file for your new logic ( /config/passenger_standalone/includes/my_new_logic.erb )
      #     Add your custom logic to your newly created file.
      #     Add your file to the area below, in an ERB block ( include_passenger_custom_template('my_new_logic.erb') )


      ### END your own configuration options ###
    }

    passenger_pre_start <%= listen_url(app) %>;
  <% end %>

  <%= include_passenger_internal_template('footer.erb', 4) %>
}

The config/passenger_standalone/includes/django_asset_pipeline.erb include sets up the Passenger Nginx server to directly serve the Django static assets:

# Django asset pipeline.
location /static {
    alias <%= app[:static_files_dir] %>;
    gzip_static on;
    expires max;
    add_header Cache-Control public;
    add_header ETag "";
}

An include file config/passenger_standalone/includes/page_level_redirects.erb allows you to create redirects to be handled by the Nginx server directly:

# Page-level Redirects
# Prevent Nginx from adding the internal app port to the rewrite, aka port 8080

port_in_redirect off;

#   Define your own custom page-level redirects below.
#
#   Examples:
#     Standard single page redirects:
#       location = /old-page-1 { return 301 /new-page-1; }
#       location = /old-page-2 { return 301 /new-page-2; }

# End Page-level Redirects

We create the include config/passenger_standalone/includes/proxy_configuration.erb to configure upstream proxies that you want to filter from the access logs. This allows you to see the actual requesting IP address of the client:

# Proxy Configuration
#
#  Used to configure settings related to Flynn's interaction with proxies.
#  Add your custom proxy configuration details below.

<% if ENV['SATORIX_PROXY_IPS'] %>
  # Provide additional proxy IPs, as described at http://nginx.org/en/docs/http/ngx_http_realip_module.html.
  #
  # This is particularity useful for services like CloudFlare, using the example at:
  # https://support.cloudflare.com/hc/en-us/articles/200170706-How-do-I-restore-original-visitor-IP-with-Nginx-
  #
  # If required, this variable should be populated with a space-separated list of proxy IPs. Example:
  # 103.21.244.0/22 103.22.200.0/22 103.31.4.0/22 104.16.0.0/12 108.162.192.0/18 2c0f:f248::/32

  real_ip_recursive on;

  <% ENV['SATORIX_PROXY_IPS'].to_s.split(' ').each do |real_ip| %>
    set_real_ip_from <%= real_ip %>;
  <% end %>
<% end %>

# Use the internal Flynn network set X-Forwarded-For header for access IPs.
set_real_ip_from <%= ENV['SATORIX_REAL_IP_FROM'] || '100.100.0.0/16' %>;
real_ip_header X-Forwarded-For;

# End Proxy Configuration

The include file config/passenger_standalone/includes/authentication.erb is used to add HTTP Basic authentication into the Nginx configuration and allows you to set an environment variable SATORIX_AUTHENTICATION_HTPASSWDS the content of which will be used to generate the htpasswd file:

# Authentication

<%-
  # The password_files hash defines which password files will be written out.
  # The generated password files should be ignored from version control.
  # Each desired password file should be specified as a key, with the value being a source for the file contents.
  # The contents should include hashed username/password combinations, separated by whitespace.
  # These can be generated using the htpasswd application, or an online tool like http://www.htaccesstools.com/htpasswd-generator/
  # For more info, see: https://docs.nginx.com/nginx/admin-guide/security-controls/configuring-http-basic-authentication/
  password_files = {
    'htpasswd' => ENV['SATORIX_AUTHENTICATION_HTPASSWDS']
  }

  def password_file_location(filename)
    passenger_standalone_includes_location = File.expand_path(__dir__)
    File.join( passenger_standalone_includes_location, filename )
  end

  password_files.each do |filename, raw_contents|
    contents = raw_contents.to_s.split.join("\n")
    File.open(password_file_location(filename), 'w') {|f| f.write(contents) } unless contents.empty?
  end

  allowed_without_auth = ENV['SATORIX_AUTHENTICATION_ALLOWED_IPS'].to_s.split
  allowed_without_auth = ['all'] if allowed_without_auth.empty?
-%>

# Allow listed networks to access without auth, otherwise require password if defined
location / {
  satisfy any;
<% allowed_without_auth.each do |target| -%>
  allow <%= target %>;
<% end -%>
<% if File.file?(password_file_location('htpasswd')) -%>
  auth_basic "Please Log In";
  auth_basic_user_file <%= password_file_location('htpasswd') %>;
<% end -%>
  deny all;
}

# End Authentication

Add the environment variable SATORIX_AUTHENTICATION_HTPASSWDS to the environment you want to restrict access to. This sets the usernames and passwords to use for Nginx HTTP Basic authentication. Needs to be generated in the format created by the Apache tool htpasswd -nb username password or using an online generator . The ENVVAR should contain newline separated lists of username and hashed password. A use case for this is restricting access to your staging environment to only authorized users. Example input:

username:$apr1$vAxBKb8N$m0en1zabtHktHeFyT3j9y
alsoname:$apr1$vAxBKb8N$m0en1zabtHktHeFyT3j9y

If you want to skip HTTP authentication for an application set the Satorix default variable SATORIX_AUTHENTICATION_ALLOWED_IPS to all and do not create the SATORIX_AUTHENTICATION_HTPASSWDS variable. This is typically what you would do on the production environment.