Migrating a python app to AWS lambda as is; without code change.(02 June 2018)

Intro
AWS Lambda lets you run code without provisioning or managing servers.
In this article, we will go through an example of how to migrate a python application to AWS lambda.
Of importance to note is that; we will not make any significant code changes to our app in order to migrate it.
You may have heard of Python frameworks that help you write python code targeted for running on AWS lambda. for example; Chalice or Zappa.
A drawback of those frameworks - in my opinion - is that they require you to change the way you architect your applications.
They require you to make code changes to suit their way of things.
In this post we wont do any of that, you will keep your familiar architect/code and still get to run your app on lambda.

- We will create a small python/django app from the ground up, using the same old familiar django pattern.
- We will then run that app on AWS lambda.

Create django app.
The app we will build is a simple one. It is a website uptime checker.
It makes a http request to google.com and twitter.com, checks whether they are up or not and reports the same.
We'll create a new directory for our app, create a virtualenv, install django and also create a requirements file with the installed packages.

mkdir uptime-checker && \
cd uptime-checker && \
virtualenv .uptime-checker && \
source .uptime-checker/bin/activate && \
pip install django && \
pip freeze >> requirements.txt
                

Create a settings.py file with the following contents:

import os

def here(*args):
    return os.path.join(os.path.abspath(os.path.dirname(__file__)), *args)

PROJECT_ROOT = here('')
def project_root_joiner(*args):
    return os.path.join(os.path.abspath(PROJECT_ROOT), *args)

ALLOWED_HOSTS = '*'
SECRET_KEY =  'A-random-and-secure-secret-key'
ROOT_URLCONF = 'urls'
WSGI_APPLICATION = 'wsgi.application'
INSTALLED_APPS = (
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.messages',
    'django.contrib.staticfiles',
)
STATIC_ROOT = project_root_joiner('', 'static/')
STATIC_URL = '/static/'
                

Create a wsgi.py file with contents;

import os
from django.core.wsgi import get_wsgi_application
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "settings")
application = get_wsgi_application()
                

And urls.py;

from django.conf.urls import url
import views
urlpatterns = (
    url(r'^$', views.home, name='home'),
)
                

We also need the manage.py file.
Create manage.py file with content;

#!/usr/bin/env python
import os
import sys
if __name__ == "__main__":
    os.environ.setdefault("DJANGO_SETTINGS_MODULE", "settings")
    from django.core.management import execute_from_command_line
    execute_from_command_line(sys.argv)          
                

We created urls.py and the code in there is referring to views, but we haven't created views yet.
So lets do just that, create views.py whose content is;

from django.http import HttpResponse
def home(request):
    return HttpResponse("Hello. Welcome to Python and AWS lambda.") 
                     

And this been python, we need an empty __init__ file, so create it.

touch __init__.py
                    

So your final directory structure looks like;

uptime-checker/
    __init__.py
    manage.py
    settings.py
    urls.py
    views.py
    wsgi.py
                

But does the app really work? There's only one way to find out; let's run it.

python manage.py runserver 0.0.0.0:8080
                

When you visit http://localhost:8080/ with your favorite browser, you see that you get the Hello. Welcome to Python and AWS lambda. greeting.

We were not making a greetings app, we want a website uptime checker. So lets add the logic in our views to check on the uptime of our two websites; google and twitter.
The modified views.py becomes;

import urllib
from django.http import HttpResponse
def home(request):
    g = urllib.urlopen("http://www.google.com")
    google_status_code = g.getcode()
    g.close()
    t = urllib.urlopen("http://www.twitter.com")
    twitter_status_code = t.getcode()
    t.close()
    return HttpResponse("""Uptime Checker. google.com status={0}. twitter.com status={1}""".format(
        google_status_code,
        twitter_status_code)) 
                

There, our app is working as per spec. It is not pretty or the most useful app, but it is working albeit on our machines.
What we want to do now is deploy this app - as is - to AWS lambda.

Deploying django app to AWS lambda.
1. First we need to install up.
up bills itself thus;
Up focuses on deploying "vanilla" HTTP servers there's nothing new to learn, just develop with your favorite existing frameworks such as Express, Koa, Django, Golang net/http or others. -- https://github.com/apex/up (emphasis is mine.)

up can be installed via curl;

curl -sf https://up.apex.sh/install | sh
                

You can make sure that up has installed succesfully by checking it's help command.

up --help
                

2. Next we need to make sure that our AWS setup is ok.
- make sure that your aws credentials are okay and stored in the right place/s. For my case, I have a ~/.aws/credentials file that looks like;

[apex-up-profile]
aws_access_key_id = myAccessId
aws_secret_access_key = myAccessKey
region = eu-west-1
                

- also make sure that you create this AWS policy in your AWS account.

3. There's only one change that you have to make to your app files in order to satisfy up, we have to rename our manage.py file to app.py; but the contents of that file remain the same.

mv manage.py app.py
                

The reason we have to do that is because up can be used to deploy apps created using other programming languages/runtimes, and the way up is able to tell which runtime it is you want to deploy is by looking for specific file names.
The file name that tells up you are using a python runtime is app.py

4. In the uptime-checker directory, create an up.json file whose contents are;

{
    "name": "uptime-checker",
    "profile": "apex-up-profile",
    "proxy": {
        "command": "python app.py runserver 0.0.0.0:$PORT"
    }
}
                

where the value of the profile json key is the same as the profile name in your aws config( ~/.aws/credentials for my case)
up.json is up's configuration file. See this link for more about the config file.

5. to deploy your app to AWS lambda, run the following command;

up deploy
                

You will see an output like;

build: 4,753 files, 14 MB (4.427s)
deploy: commit bc51169 (28.903s)
stack: complete (47.398s)
                

To see the domain/url that your app was deployed to, run the command;

up url
                

You can copy and paste the url you get, into your browser to access your app.
This is what I get;
If you want to debug/introspect what is going on, you can always look at the logs of your app via;

up logs -fv
                

Up also sends your logs to AWS cloudwatch, so you can search for them there.

Anything you log to stdout in your app, will show up in the logs. So you can use a python logging config like;

import logging
logging.basicConfig(
    format="%(message)s\n",
    stream=sys.stdout,
    level=logging.DEBUG,
)
                

Conclusion
There are lot more bells in up, you can do log querying/filtering from your command line, use custom domains for your app, use secrets, deploy to multiple AWS regions, deploy to multiple stages(test/staging/prod etc) and so much more.
Before this article turns into a marketing article for up(up is open source, though they also have a paid plan) let me invite you to go check it out for yourselves; https://github.com/apex/up

We have deployed a Python/django app to AWS lambda as is, without changing the way we write our app.



If you want to tear down everything that we just created,

up stack delete
                

All the code in this blogpost can be found here:- https://github.com/komuw/uptime-checker