Structuring your Flask project the right way (no Blueprint)

Question

Intro

I've read like a thousand posts on SO and other sites trying to figure out what's wrong with my Flask structure and why I can't seem to figure out something. As a last resort, I decided to finally ask the question here.

My project is really simple:

1. I have to fetch some data from some networking devices via API, process the data and store it in a Postgresql DB (most part of the code is in lib/).
2. This project is going to be deployed on multiple environments (test, dev, staging and prod).

To do the above I'm using the following:

• Flask-SQLAlchemy + Flask-Migrate + Flask-Script - all of these for handling migrations and DB related operations;

• Python-dotenv - for handling sensitive configuration data

---

Project details

My project structure looks like this:

my_project/
├── api/  
├── app.py             
├── config.py
├── __init__.py
├── lib/
│   ├── exceptions.py
│   └── f5_bigip.py
├── log.py
├── logs/
├── manage.py
├── migrations/
├── models/
│   ├── __init__.py
│   ├── model1.py
│   └── model2.py
└── run.py

My app.py looks like this:

import os
import sys
sys.path.append(os.path.dirname(os.path.abspath(__file__)))

from dotenv import load_dotenv
from flask import Flask
from flask_migrate import Migrate
from flask_sqlalchemy import SQLAlchemy


db = SQLAlchemy()
migrate = Migrate()


def create_app():
    load_dotenv()

    app = Flask(__name__)

    environment = app.config['ENV']

    if environment == 'production':
        app.config.from_object('config.ProductionConfig')
    elif environment == 'testing':
        app.config.from_object('config.TestingConfig')
    else:
        app.config.from_object('config.DevelopmentConfig')

    db.init_app(app)
    migrate.init_app(app, db)

    return app

My config.py looks like this:

import os

from sqlalchemy.engine.url import URL

PROJECT_ROOT = os.path.dirname(os.path.realpath(__file__))


class BaseConfig:
    DEBUG = False
    TESTING = False

    DB_DRIVERNAME = os.getenv('DB_DRIVERNAME')
    DB_HOST = os.getenv('DB_HOST')
    DB_PORT = os.getenv('DB_PORT')
    DB_NAME = os.getenv('DB_NAME')
    DB_USERNAME = os.getenv('DB_USERNAME')
    DB_PASSWORD = os.getenv('DB_PASSWORD')

    DB = {
        'drivername': DB_DRIVERNAME,
        'host': DB_HOST,
        'port': DB_PORT,
        'database': DB_NAME,
        'username': DB_USERNAME,
        'password': DB_PASSWORD,
    }

    SQLALCHEMY_DATABASE_URI = URL(**DB)
    SQLALCHEMY_TRACK_MODIFICATIONS = False


class DevelopmentConfig(BaseConfig):
    DEVELOPMENT = True
    DEBUG = True


class TestingConfig(BaseConfig):
    TESTING = True


class StagingConfig(BaseConfig):
    DEVELOPMENT = True
    DEBUG = True


class ProductionConfig(BaseConfig):
    pass

My __init__.py looks like this:

from contextlib import contextmanager

from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker

# here, create_engine needs the SQLALCHEMY_DATABASE_URI
# how do I get it from the proper config?
engine = create_engine()
Session = sessionmaker(bind=engine)


@contextmanager
def session_scope():
    """
    Provide a transactional scope around a series of operations.
    """
    session = Session()
    try:
        yield session
        session.commit()
    except Exception as e:
        print(f'Something went wrong here: {str(e)}. rolling back.')
        session.rollback()
        raise
    finally:
        session.close()

My manage.py looks like this:

from flask_script import Manager
from flask_migrate import MigrateCommand

from app import create_app


from models import *


manager = Manager(create_app)
manager.add_command('db', MigrateCommand)

if __name__ == '__main__':
    manager.run()

My models/model1.py looks like this:

from sqlalchemy.dialects.postgresql import INET
from sqlalchemy.sql import func

from app import db


class Model1(db.Model):
    __tablename__ = 'model1'

    id = db.Column(db.BigInteger, primary_key=True, autoincrement=True)
    ip_address = db.Column(INET, unique=True, nullable=False)
    last_update = db.Column(db.DateTime(), server_default=func.now())

    def __repr__(self):
        return f'<Model1: {self.ip_address}>'

    def __init__(self, ip_address):
        self.ip_address = ip_address

---

Questions

Now, I have three main questions:

1. In my main __init__.py how can I import SQLALCHEMY_DATABASE_URI from the app's config?
2. Having the Session() object in the __init__.py doesn't seem too intuitive. Should it be placed in other places? For more context, the session is used in lib/f5_bigip.py and is probably going to be used in the api/ as well.
3. Is the overall project structure ok?

Answer 1

Your questions 1 and 2 are directly related to the part of your project that I found strange, so instead of answering those questions I'll just give you a much simpler and better way.

It seems in __init__.py you are implementing your own database sessions, just so that you can create a scoped session context manager. Maybe you've got that code from another project? It is not nicely integrated with the rest of your project, which uses the Flask-SQLAlchemy extension, you are just ignoring Flask-SQLAlchemy, which manages your database connections and sessions, and basically creating another connection into the database and new sessions.

What you should do instead is leverage the connections and sessions that Flask-SQLAlchemy provides. I would rewrite __init__.py as follows (doing this by memory, so excuse minor mistakes):

from contextlib import contextmanager

from app import db


@contextmanager
def session_scope():
    """
    Provide a transactional scope around a series of operations.
    """
    try:
        yield db.session
        session.commit()
    except Exception as e:
        print(f'Something went wrong here: {str(e)}. rolling back.')
        db.session.rollback()
        raise
    finally:
        db.session.close()

With this you are reusing the connection/sessions from Flask-SQLAlchemy. Your question 1 then is not a problem anymore. For question 2, you would use db.session anywhere in your app where you need database sessions.

Regarding question 3 I think you're mostly okay. I would suggest you do not use Flask-Script, which is a fairly old and unmaintained extension. Instead you can move your CLI to Flask's own CLI support.