knowledge-kitchen

Pymongo & Flask - a Minimal Python-Based Web Application Stack (in Python)

Python-based web application technology stack.

  1. Overview
  2. Virtual Environments
  3. Pymongo
  4. Flask
  5. Pymongo + Flask
  6. Useful modules
  7. Conclusions

Overview

Concept

Developing web applications requires us to build code to server web pages to web browser that request them, as well as maintain data in a database. We will focus on using a few popular modules in Python to help with this:

Virtual Environments

Concept

Python comes with an ability to create virtual environments.

Setup

For example, we can create a new virtual environment with the name .venv:

python -m venv .venv # try 'python3' instead of 'python' if your system requires it

Activate the virtual environment named .venv…

source .venv/bin/activate

.venv\Scripts\activate.bat

Pymongo

Concept

pymongo is a popular module to help Python programs connect to and issue commands to MongoDB databases. It simplifies…

Installation

To install the dependencies, like pymongo, into our local development environments, we use pip, the default Python “package manager” - software that takes care of installing the correct version of any module into your in the correct place for the current environment.

pip install pymongo # replace 'pip' with 'pip3' if your system requires it

Once installed in the development environment, a Python program can import pymongo. It is also useful to import the bson package’s ObjectId class that is a Python representation of MongoDB’s data type for document _id fields. And we will import the datetime module to keep track of the date and time when documents are created.

import pymongo
from bson.objectid import ObjectId
import datetime

Connect to a database

The basic pattern of initializing a connection to a database server:

import pymongo # import the module

# make a connection to the database server
connection = pymongo.MongoClient("mongodb://your_username:your_username@your_host_name:27017")

# select a specific database on the server
db = connection["db_name"]

The variable db can now be used to perform the standard database CRUD operations.

If connecting to a database server on the local machine, your connection string might be as simple as 'mongodb://localhost:27017' or 'mongodb://admin:secret@localhost:27017' (if you’ve set up a admin/secret as your username/password). Organizations hosting databases professionally will provide you with their recommended connection string.

Create a new document

A new document can be constructed as a Python dictionary:

doc = {
    "name": "Foo Barstein",
    "email": "fb1258@nyu.edu",
    "message": "We loved with a love that was more than love.\n -Edgar Allen Poe",
    "created_at": datetime.datetime.utcnow() # the date time now
}

Use the insert_one() and insert_many() methods to create new documents and save them to the database.

mongoid = db.collection_name.insert_one(doc)

Read existing records

Use the find() method to retrieve existing documents from the database collection.

docs = db.collection_name.find({
    "email": "fb1258@nyu.edu"
})

for doc in docs:
    output = '{name} ({email}) said "{message}" at {date}.'.format(
        name = doc.name,
        email = doc.email,
        date = doc.created_at.strftime("%H:%M on %d %B %Y") # nicely-formatted datetime
    )
    print(output)

Or retrieve just a single document with find_one():

doc = db.collection_name.find_one({
    "email": "fb1258@nyu.edu"
})

Read existing records (continued)

If the field names of records contain a spaces or special characters, it is required to refer to the fields within a given document using an alternate Dictionary syntax:

docs = db.jobs.find({
    "Agency":"DEPT OF PARKS & RECREATION",
    "Salary Range To": { "$gt": 30000 }
})

for doc in docs:
    output = '{title} ({agency}) - {salary_to}'.format(
        agency=doc["Agency"],
        title=doc["Business Title"],
        salary_to=doc["Salary Range To"]
    )
    print(output)

Sort, limit, and count

The syntax for sorting with pymongo is almost identical to using the mongo command shell.

# sort docs in descending order of insertion date
docs = db.collection_name.find({}).sort("created_at", -1)

Similarly, limit() works like in the mongo shell:

# show the 10 most recently created matching documents
docs = db.collection_name.find({ "email": "fb1258@nyu.edu" } ).sort("created_at", -1).limit(10)

Counting documents works a bit differently from the mongo shell, with the count_documents() function:

# determine how many matching documents exist in the collection
count = db.collection_name.count_documents({ "email": "fb1258@nyu.edu" })

Update an existing record

Use the update_one() or update_many() methods to update existing documents from the database collection.

db.collection_name.update_one( {
    { "_id": ObjectId(mongoid) }, # match criteria
    {
        "$set":{
            "message": new_message
            "created_at": datetime.datetime.utcnow() # the date time now
        }
    }
})

Delete an existing record

Use the delete_one() or delete_many() methods to to delete existing documents from the database collection.

db.collection_name.delete_one({
    "_id": ObjectId(mongoid)
})

Aggregation

Create an aggregation pipeline with the aggregate() function is virtually identical to its mongo shell cousin.

# filter stage to match only jobs paid hourly
sal_freq_filter = { "$match": {"Salary Frequency": 'Hourly'} }

# group stage to group jobs by agency
group_by = {
    "$group": {
        "_id": "$Agency",
        "count_pos": { "$sum": "$# Of Positions" }
    }
}

# order stage to order by number of positions in descending order
order_by = { "$sort": { "count_pos": -1 } }

# perform the aggregation pipeline and get results
result = db.jobs.aggregate( [ sal_freq_filter, group_by, order_by ] )

Flask

Concept

Programming web apps in Python is usually done using a framework - a readymade code library that provides a lot of the basic functionality all web apps share in common.

Installation

Use pip to install into the local development environment.

pip install Flask

Once installed in the development environment, a Python program can import Flask and whichever other features of the flask framework it requires.

from flask import Flask, render_template, request, redirect, abort, url_for, make_response

Instantiate the app

Instantiate a flask-based web app…

app = Flask(__name__)

General notes about the built-in Python variable, __name__, used as an argument here:

Routes

Modern web applications use meaningful URLs to help users. Users are more likely to like a page and come back if the page uses a meaningful URL they can remember and use to directly visit a page.

-Flask Quickstart

Plain text route

Perhaps the simplest example of a route using flask - a route for the home page of a web app that simply returns the text, "Welcome!"

@app.route('/')
def show_home():
    response = make_response("Welcome!", 200) # put together an HTTP response with success code 200
    response.mimetype = "text/plain" # set the HTTP Content-type header to inform the browser that the returned document is plain text, not HTML
    return response # the return value is sent as the response to the web browser

Static HTML route

Instead of plain text, it is more common to respond to a request with an HTML document. This is most easily done by saving an HTML document into a separate file, let’s call it templates/foo.html:

<html>
  <head>
    <title>Home</title>
  </head>
  <body>
    <p>Welcome!</p>
  </body>
</html>

A flask route for the meaningful URL, /foo, can be created to return the contents of this HTML file.

@app.route('/foo')
def show_foo():
    return render_template('foo.html')

Passing data to HTML templates

HTML templates, using jinja syntax, can be designed to accept variables and plug them into the HTML at designated spots. Let’s call this file templates/bar.html:

<html>
  <head>
    <title>&#123;&#123; title &#125;&#125;</title>
  </head>
  <body>
    <p>&#123;&#123; paragraph_text &#125;&#125;</p>
  </body>
</html>

A flask route for the meaningful URL, /bar, can be created to pass data to the HTML template and return the resultant contents.

@app.route('/bar')
def show_bar():
    return render_template('bar.html', title='Hello', paragraph_text='Welcome!')

Passing data to HTML templates (continued)

It is possible to pass array data to templates:

@app.route('/cheese')
def show_cheeses():
    return render_template('cheeses.html', good_cheeses=['gorgonzola', 'brie', 'limburger'])

This array could be iterated through in the template, templates/cheeses.html:

<html>
  <head>
    <title>Cheeses</title>
  </head>
  <body>
    <p>Some delicious cheeses:</p>
    <ul>
      &#123;% for cheese in good_cheeses %&#125;
      <li>&#123;&#123; cheese &#125;&#125;</li>
      &#123;% endfor %&#125;
    </ul>
  </body>
</html>

Handling HTTP POST requests

Imagine a web page showed an HTML form the user could fill in and submit.

<form method="POST" action="/save">
  <input type="text" name="fname" />
  &lt;textarea name="fmessage"&gt;&lt;/textarea&gt;
  <input type="submit" value="Save!" />
</form>

By default, routes expect to receive HTTP GET requests. It is possible to set up routes for HTTP POST requests by using the methods decorator argument.

@app.route('/save', methods=['POST'])
def handle_save():
    # get form data included in the body of the POST request
    name = request.form['fname']
    message = request.form['fmessage']

    # do something fabulous with this data here

    return redirect(url_for('show_foo')) # tell the browser to make a follow-up request for the /foo route

Parameterizing routes

Route URLs can hold parameters. For example, we can set up a route named /show which accepts a parameter that we’ll automatically store in a variable named something:

@app.route('/show/<something>')
def show_it(something):
    return render_template('bar.html', title=something, paragraph_text=something)

Linking from a template to a route

In flask’s templating engine, it is possible to automatically generate the correct links from a template to a route by using the url_for() method.

For example, this code, if placed within a template file, would link to the /foo route:

<a href="&#123;&#123; url_for('/show_foo') &#125;&#125;"> click me </a>

Linking from a template to a route (continued)

If the target route expects arguments to be supplied to it, such as our parameterized /show/something route did, then these can be supplied as additional arguments to url_for():

For example, this code, if placed within a template file, would link to the show/something route, with "me" passed into the something variable:

<a href="&#123;&#123; url_for('/show_it', something='me') &#125;&#125;">
  click me
</a>

Base templates

Most pages of a web site share a lot of HTML code in common - especially the top and bottom sections of th HTML.

Base templates (continued)

For example, here is a simple base template in a file named templates/base.html containing boilerplate HTML code with a block named container in the middle for each page’s unique content to be placed.

<!DOCTYPE html>
<html lang="en">
  <head>
    <title>My Very Ordinary Site</title>
  </head>
  <body>
    <div class="container">
      <header>This is the header</header>
      <main>&#123;% block container %&#125; &#123;% endblock %&#125;</main>
      <footer>This is the footer</footer>
    </div>
  </body>
</html>

Base templates (continued again)

Other templates can now indicate that their contents should be wrapped inside the base template code.

For example, here is a template named templates/baz.html, which will automatically include the code from the base template.

&#123;% extends 'base.html' %&#125; &#123;% block container %&#125;
<p>
  Put the unique content of this template between the begin/end of the container
  block
</p>
&#123;% endblock %&#125;

Pymongo + Flask

Concept

A modern web application inevitably involves:

Routes with database integration

It is, of course, possible to develop flask routes that interact with a MongoDB database using pymongo.

For example, here is a route that pulls documents from a database and passes them to a template:

@app.route('/read')
def read():
    docs = db.collection_name.find({})
    return render_template('read.html', docs=docs) # render the read template, passing it the list of documents

Useful modules

Concept

Since flask is a popular Python web framework, there are many useful modules that can be used in tandem with it, e.g.:

Storing secret credentials in .env file

It is common to store sensitive information – such as a database connection string, usernames, passwords needed by the back-end – in a file named .env in the same directory as the Python script. This file should be excluded from version control by adding it to the .gitignore file.

For example:

MONGO_DBNAME=example
MONGO_URI="mongodb://admin:secret@localhost:27017"

Storing secret credentials in .env file (continued)

The python package named python-dotenv can be used to easily load these variables into program memory:

For example:

from dotenv import load_dotenv

# load the variables from the .env file
load_dotenv()
MONGO_DBNAME = os.getenv('MONGO_DBNAME')
MONGO_URI = os.genenv('MONGO_URI')

User authentication

The python package named flask-login can be used to manage user sessions in a Flask app.

Conclusions

Thank you. Bye