Satel's Docker Python base

🧭 Purpose

This docker image is a base for Satel webapp's python backend. The standard packages we use in most projects are already installed. The folder structure and commands provide the standard framework for development and deployment to production of the app.

🏗️ Developing

There is a dev.sh script included that builds both docker images and runs one of 3 cases based on the parameter you pass in:

1. ./dev.sh validate runs the validation scripts (validatecodeonce)
2. ./dev.sh run runs the main application.
3. ./dev.sh cli drops you into the cli for the image

⏳ Migration to version 4

The version 4 comes with significant changes:

• The docker base image was changed from alpine to slim buster
  • All the apk add commands need to be replaced with equivalent apt-get commands
  • slim buster has more base packages such as bash so probably less installation of system packages is needed
• The entrypoint now provides standard commands to run and test the webapp
  • No CMD is necessary for production as startapp is the default
  • In development, run the developapp command
  • See below for testing
• A config.sh is expected where environment variables for configuration purposes can be defined
• The requirements.txt has been reduced to the base and development python packages
  • Base packages such as FastAPI, pydantic, loguru, ...
  • Development pacakages such as watchmedo, pytest, mypy, ...
  • Therefore you need to add to your requirements file packages such as pandas that were included in this image in versions prior to 4

📂 Folder structure

The /python folder is the base folder for all the files of this images such as the entrypoint.sh. Then the subfolders organize the app files:

• /python/app holds the source of the app
• /python/static holds the static files served by the app such as static html files
• /python/logs is meant for the log files generated by the app
• /python/files is for files uploaded through the app

🧰 Notes for maintenance

Upgrade python base image versions

1. Go to docker hub to get the latest patch version for the python docker image
2. Then 3 files need to be edited with the latest python minor or patch versions:
   1. docker-compose.generate_requirements.yml to generate the requirements files.
   2. .github/workflows/docker_image.yml to build and push the right images to docker hub.
   3. Dockerfile to change the default base image version. This is used only for testing while developing but better to keep it up to date.

Upgrade the requirements files

Rather simple. Just run the following command at the root of the repo:

docker-compose -f docker-compose.generate_requirements.yml up

⌨️ Commands

The entrypoint of this docker image provides standard commands for our app's docker image to call with CMD in the Dockerfile. These commands are effectively run as arguments of the entrypoint such as:

/python/entrypoint.sh startapp

The .bashrc file contains aliases so that one can also just call startapp from within the docker container.

NOTE: If you want to run a bash command, just pass it to the docker container and it will be executed since it doesn't match one of the predefined commands. Example here running the ls command:

docker run satel/python-base:latest ls

startapp

Run the app in production mode. This is the default CMD is none is provided. The app is expected to be a FastAPI app defined in /python/app/webapp/main.py such that the app is executed with the command:

uvicorn --host 0.0.0.0 --port 8000 webapp.main:app

You can replace webapp.main:app by setting the first argument of the startapp command such that for example:

startapp myapp:theapp

The command will try to load the config.sh from multiple locations:

1. From the docker secrets folder
2. From the current directory, being /python/app
3. If the previous locations don't have a config.sh file, the config.sh.example file is loaded from the current directory if it exists as a fallback. This is useful for CI/CD environments for which the example values are good enough to run the tests

developapp

Run the app in development mode with watchmedo from the watchdog python package such that the app restarts whenever the code changes in /python/app. The same default location (webapp.main:app) and option to change the location as startapp are used for developapp.

The configuration file is loaded the same way as in startapp.

validatecode

This command is used during development to automatically run the pytest tests, the mypy typing check and the flake8 linting check whenever the code changes.

The flake8 linting includes isort import module sorting thanks to the flake8-isort plugin.

This code validation command executes the /python/test_suite.sh script which can be overwritten with custom code validation.

validatecodeonce

Same as validatecode but executed once rather than continously running on code change. This can be used for CI/CD purposes since it generates report files in the /python/reports folder.

runtests (DEPRECATED)

DEPRECATED: Use validatecodeonce instead

This command runs only the pytest tests for CI/CD purposes. It outputs the tests and code coverage results in the /python/app/unittesting.xml and /python/app/coverage.xml files respectively.

A coverage configuration file can be provided at python/app/coverage.conf.