Python application in Satorix

A tutorial for getting a Django Python application working in Satorix. For a description of the default environment variables managed by the Satorix Dashboard and using environment variables in your application check the application environment variables article.

An application with the results of this guide can be downloaded as a Django demo on GitHub.

Manual Python configuration

In your application code we will be working with some files to get things running on your Satorix Hosting Cluster. Make sure your public Django apps are placed into a sub-folder called public/.

The following files are placed in the root of your application directory

Add a .gitlab-ci.yml, this is a basic setup to allow us to automatically deploy to your Satorix Hosting Cluster:

# We are using the Satorix base image from https://hub.docker.com/r/satorix/base/
image: 'satorix/base:18'

# Global caching directives.
cache:
  key: "$CI_PROJECT_ID"
  paths:
    - 'tmp/satorix/cache' # To cache buildpack output between runs.


.satorix: &satorix
  script:
    - gem install satorix --no-document
    - satorix


deploy_with_flynn:
  environment:
    name: $CI_COMMIT_REF_NAME
    url: "http://$CI_PROJECT_NAME.$CI_COMMIT_REF_SLUG.$SATORIX_HOSTING_NAMESPACE"
  stage: deploy
  only:
    - staging
    - production
  <<: *satorix

Create a Procfile that defines the processes that your Satorix Hosting Cluster will run:

# This Procfile defines the types of process that Satorix will run.
# For more information, please see the documentation at https://www.satorix.com/docs

web: passenger start --port $PORT public/

If you don't already have one create a runtime.txt and specify the version of Python to use for your application:

python-3.5.7

Create a requirements.txt that is used to specify the Python package dependencies to install that are required for your application, these are installed using Pip - the Python dependency manager. We add the django-heroku helper package to set up the database connection in your Satorix Hosting Cluster:

django
django-heroku

The .buildpacks file is used to tell the Satorix Hosting Cluster the buildpacks to use for your application. We specify the Phusion Passenger buildpack to utilize features of the Satorix Dashboard. Passenger requires Ruby for some of it's utilities so is also included here:

https://github.com/heroku/heroku-buildpack-ruby
https://github.com/satorix/satorix-passenger-buildpack
https://github.com/heroku/heroku-buildpack-python

Create a .ruby-version to specify a version of Ruby to use with Passenger:

ruby-2.6.3

A blank Gemfile to allow the Ruby buildpack detection:

ruby "#{ File.open("#{ File.dirname(__FILE__) }/.ruby-version", &:gets).strip[/ruby-(.+)/i, 1] }"

source "https://rubygems.org"

A blank Gemfile.lock to allow the Ruby buildpack detection. This can also be created by running bundle install after creation of the Gemfile if Ruby is installed:

GEM
  remote: https://rubygems.org/
  specs:

PLATFORMS
  ruby

DEPENDENCIES

RUBY VERSION
   ruby 2.6.3p62

BUNDLED WITH
   1.17.2

The following files are placed in the public/ directory

At the top of your public/appname/settings.py Django settings file, add the import statement for the django-heroku package.

import django_heroku

At the end of your public/appname/settings.py Django settings file, add the following to load the environment variables to configure your database and static files using the django-heroku package.

# Pull in ENVVARs for buildpack applications
django_heroku.settings(locals())

# Remove require SSL from database configuration created by django_heroku
try:
    del DATABASES['default']['OPTIONS']['sslmode']
except KeyError:
    pass

Create a public/passenger_wsgi.py file to set up a WSGI callable application object for Passenger. Change the satorixdjangodemo to be your application's name.

import satorixdjangodemo.wsgi
application = satorixdjangodemo.wsgi.application

A public/Passengerfile.json configuration file to set Passenger defaults for running on your Satorix Hosting Cluster.

{
  "nginx_config_template": "../config/passenger_standalone/nginx.conf.erb",
  "static_files_dir": "staticfiles",
  "log_file": "/dev/stdout",
  "environment": "production"
}

The following files are placed in a config/ directory

A config/passenger_standalone/nginx.conf.erb template to configure the built in Nginx server for Passenger. This sets up includes Django asset serving, web server level redirects, and proxy settings.

##########################################################################
#  Passenger Standalone is built on the same technology that powers
#  Passenger for Nginx, so any configuration option supported by Passenger
#  for Nginx can be applied to Passenger Standalone as well. You can do
#  this by direct editing the Nginx configuration template that is used by
#  Passenger Standalone.
#
#  Learn more about using the Nginx configuration template at:
#  https://www.phusionpassenger.com/library/config/standalone/intro.html#nginx-configuration-template
#
#  To test this configuration template run:
#  passenger start --nginx-config-template config/passenger_standalone/nginx.conf.erb --debug-nginx-config
#
#  *** NOTE ***
#  If you customize the template file, make sure you keep an eye on the
#  original template file and merge any changes. New Phusion Passenger
#  features may require changes to the template file.
##############################################################

<%
  def include_passenger_custom_template(template, indent = 0, the_binding = get_binding)
    path = File.join(File.dirname(__FILE__), 'includes', template)
    erb = ERB.new(File.read(path), nil, "-", next_eoutvar)
    erb.filename = path
    result = erb.result(the_binding)

    # Set indenting
    result.gsub!(/^/, " " * indent)
    result.gsub!(/\A +/, '')

    result
  end

  def use_canonical?
    !canonical_domain.nil? &&
    !canonical_domain.empty? &&
    !canonical_domain_protocol.nil? &&
    !canonical_domain_protocol.empty?
  end

  def canonical_domain
    ENV['SATORIX_CANONICAL_URI_HOST']
  end

  def canonical_domain_protocol
    ENV['SATORIX_CANONICAL_URI_PROTOCOL']
  end

  def canonical_uri
    "#{ canonical_domain_protocol }://#{ canonical_domain }" if use_canonical?
  end
%>

<%= include_passenger_internal_template('global.erb') %>

worker_processes 1;
events {
    worker_connections 4096;
}

http {
    <%= include_passenger_internal_template('http.erb', 4) %>

    ### BEGIN your own configuration options ###
    # This is a good place to put your own config
    # options. Note that your options must not
    # conflict with the ones Passenger already sets.
    # Learn more at:
    # https://www.phusionpassenger.com/library/config/standalone/intro.html#nginx-configuration-template

    ### END your own configuration options ###

    default_type application/octet-stream;
    types_hash_max_size 2048;
    server_names_hash_bucket_size 96;
    client_max_body_size 1024m;
    access_log off;
    keepalive_timeout 60;
    underscores_in_headers on;
    gzip on;
    gzip_comp_level 3;
    gzip_min_length 150;
    gzip_proxied any;
    gzip_types text/plain text/css text/json text/javascript
        application/javascript application/x-javascript application/json
        application/rss+xml application/vnd.ms-fontobject application/x-font-ttf
        application/xml font/opentype image/svg+xml text/xml;
  <% if @app_finder.multi_mode? %>
    # Default server entry for mass deployment mode.
    server {
        <%= include_passenger_internal_template('mass_deployment_default_server.erb', 12) %>
    }
  <% end %>
  <% @apps.each do |app| %>
    <% if use_canonical? %>
    # Redirect all requests to the canonical domain.
    server {
      server_name <%= app[:server_names].join(' ') %>;
      listen <%= nginx_listen_address(app) %> default_server;
      return 301 <%= canonical_uri %>$request_uri;
    }
    <% else %>
    # No canonical domain defined, passing all requests to the main server block.
    <% end %>

    # Main server block.
    server {
      <% app[:server_names] = [canonical_domain] if use_canonical? %>
      <%= include_passenger_internal_template('server.erb', 8, true, binding) %>

      <%= include_passenger_custom_template('django_asset_pipeline.erb', 8, binding) %>
      <%= include_passenger_custom_template('page_level_redirects.erb', 8, binding) %>
      <%= include_passenger_custom_template('proxy_configuration.erb', 8, binding) %>
      <%= include_passenger_custom_template('authentication.erb', 8, binding) %>

      ### BEGIN your own configuration options ###
      # This is a good place to put your own config options.
      # Note that your options must not conflict with the ones Passenger already sets.
      #
      # Learn more at:
      # https://www.phusionpassenger.com/library/config/standalone/intro.html#nginx-configuration-template
      #
      # You can use the include_passenger_custom_template to method include your own custom template.
      # This will help you compartmentalize your configurations, to help organize your settings.
      #
      #   Example:
      #
      #     Create a new file for your new logic ( /config/passenger_standalone/includes/my_new_logic.erb )
      #     Add your custom logic to your newly created file.
      #     Add your file to the area below, in an ERB block ( include_passenger_custom_template('my_new_logic.erb') )


      ### END your own configuration options ###
    }

    passenger_pre_start <%= listen_url(app) %>;
  <% end %>

  <%= include_passenger_internal_template('footer.erb', 4) %>
}

The config/passenger_standalone/includes/django_asset_pipeline.erb include sets up the Passenger Nginx server to directly serve the Django static assets:

# Django asset pipeline.
location /static {
    alias <%= app[:static_files_dir] %>;
    gzip_static on;
    expires max;
    add_header Cache-Control public;
    add_header ETag "";
}

An include file config/passenger_standalone/includes/page_level_redirects.erb allows you to create redirects to be handled by the Nginx server directly:

# Page-level Redirects
# Prevent Nginx from adding the internal app port to the rewrite, aka port 8080

port_in_redirect off;

#   Define your own custom page-level redirects below.
#
#   Examples:
#     Standard single page redirects:
#       location = /old-page-1 { return 301 /new-page-1; }
#       location = /old-page-2 { return 301 /new-page-2; }

# End Page-level Redirects

We create the include config/passenger_standalone/includes/proxy_configuration.erb to configure upstream proxies that you want to filter from the access logs. This allows you to see the actual requesting IP address of the client:

# Proxy Configuration
#
#  Used to configure settings related to Flynn's interaction with proxies.
#  Add your custom proxy configuration details below.

<% if ENV['SATORIX_PROXY_IPS'] %>
  # Provide additional proxy IPs, as described at http://nginx.org/en/docs/http/ngx_http_realip_module.html.
  #
  # This is particularity useful for services like CloudFlare, using the example at:
  # https://support.cloudflare.com/hc/en-us/articles/200170706-How-do-I-restore-original-visitor-IP-with-Nginx-
  #
  # If required, this variable should be populated with a space-separated list of proxy IPs. Example:
  # 103.21.244.0/22 103.22.200.0/22 103.31.4.0/22 104.16.0.0/12 108.162.192.0/18 2c0f:f248::/32

  real_ip_recursive on;

  <% ENV['SATORIX_PROXY_IPS'].to_s.split(' ').each do |real_ip| %>
    set_real_ip_from <%= real_ip %>;
  <% end %>
<% end %>

# Use the internal Flynn network set X-Forwarded-For header for access IPs.
set_real_ip_from <%= ENV['SATORIX_REAL_IP_FROM'] || '100.100.0.0/16' %>;
real_ip_header X-Forwarded-For;

# End Proxy Configuration

The include file config/passenger_standalone/includes/authentication.erb is used to add HTTP Basic authentication into the Nginx configuration and allows you to set an environment variable SATORIX_AUTHENTICATION_HTPASSWDS the content of which will be used to generate the htpasswd file:

# Authentication

<%-
  # The password_files hash defines which password files will be written out.
  # The generated password files should be ignored from version control.
  # Each desired password file should be specified as a key, with the value being a source for the file contents.
  # The contents should include hashed username/password combinations, separated by whitespace.
  # These can be generated using the htpasswd application, or an online tool like http://www.htaccesstools.com/htpasswd-generator/
  # For more info, see: https://docs.nginx.com/nginx/admin-guide/security-controls/configuring-http-basic-authentication/
  password_files = {
    'htpasswd' => ENV['SATORIX_AUTHENTICATION_HTPASSWDS']
  }

  def password_file_location(filename)
    passenger_standalone_includes_location = File.expand_path(__dir__)
    File.join( passenger_standalone_includes_location, filename )
  end

  password_files.each do |filename, raw_contents|
    contents = raw_contents.to_s.split.join("\n")
    File.open(password_file_location(filename), 'w') {|f| f.write(contents) } unless contents.empty?
  end

  allowed_without_auth = ENV['SATORIX_AUTHENTICATION_ALLOWED_IPS'].to_s.split
  allowed_without_auth = ['all'] if allowed_without_auth.empty?
-%>

# Allow listed networks to access without auth, otherwise require password if defined
location / {
  satisfy any;
<% allowed_without_auth.each do |target| -%>
  allow <%= target %>;
<% end -%>
<% if File.file?(password_file_location('htpasswd')) -%>
  auth_basic "Please Log In";
  auth_basic_user_file <%= password_file_location('htpasswd') %>;
<% end -%>
  deny all;
}

# End Authentication

Add the environment variable SATORIX_AUTHENTICATION_HTPASSWDS to the environment you want to restrict access to. This sets the usernames and passwords to use for Nginx HTTP Basic authentication. Needs to be generated in the format created by the Apache tool htpasswd -nb username password or using an online generator . The ENVVAR should contain newline separated lists of username and hashed password. A use case for this is restricting access to your staging environment to only authorized users. Example input:

username:$apr1$vAxBKb8N$m0en1zabtHktHeFyT3j9y
alsoname:$apr1$vAxBKb8N$m0en1zabtHktHeFyT3j9y

If you want to skip HTTP authentication for an application set the Satorix default variable SATORIX_AUTHENTICATION_ALLOWED_IPS to all and do not create the SATORIX_AUTHENTICATION_HTPASSWDS variable. This is typically what you would do on the production environment.