コンテナ内のPython環境（モジュール）をLambdaレイヤーで使用できるようにzip化する

コンテナ内のPython環境（モジュール）をLambdaレイヤーで使用できるようにzip化する方法をまとめました。

1. 実行環境

1.1 ディレクトリ構成

.
├── Dockerfile
├── README.md
├── app
│   ├── lambda
│   │   └── `Lambdaで実行するPythonファイル`
│   └── layer
├── docker-compose.yml
└── requirements.txt

1.2 Dockerfile

Dockerfile

# --------------------------------------------------
# ベースイメージ
# --------------------------------------------------
FROM python:3
USER root

# --------------------------------------------------
# システムパッケージの更新とロケール設定
# --------------------------------------------------
RUN apt-get update
RUN apt-get -y install locales && \
    localedef -f UTF-8 -i ja_JP ja_JP.UTF-8

# --------------------------------------------------
# 環境変数の設定
# --------------------------------------------------
ENV LANG ja_JP.UTF-8
ENV LANGUAGE ja_JP:ja
ENV LC_ALL ja_JP.UTF-8
ENV TZ JST-9
ENV TERM xterm

# --------------------------------------------------
# 開発ツールとPythonパッケージマネージャーの更新
# --------------------------------------------------
RUN apt-get install -y vim less zip \
    && pip install --upgrade pip \
    && pip install --upgrade setuptools

# --------------------------------------------------
# 依存関係のインストール
# --------------------------------------------------
COPY requirements.txt workspace/
RUN pip install -r workspace/requirements.txt```

1.3 docker-compose.yml

docker-compose.yml

# --------------------------------------------------
# Docker Compose バージョン
# --------------------------------------------------
version: "3"

# --------------------------------------------------
# サービス定義
# --------------------------------------------------
services:
  # --------------------------------------------------
  # アプリケーションサービス
  # --------------------------------------------------
  app:
    # Dockerfileからイメージをビルド
    build: .
    
    # コンテナ名
    container_name: app
    
    # 作業ディレクトリ
    working_dir: /workspace/app/
    
    # ボリュームマウント設定（ホストのカレントディレクトリをコンテナの/workspaceにマウント）
    volumes:
      - ./:/workspace/
    
    # インタラクティブな端末を有効化
    tty: true

2. コンテナ内のPython環境（モジュール）をzip化

2.1 シェルスクリプトの作成

create_layer.sh

#!/bin/bash

# エラーが発生したら停止
set -e

# コンテナ名 (docker-compose.ymlのcontainer_nameと一致させる)
CONTAINER="app"

# パス設定
# コンテナ内の一時作業ディレクトリ
WORK_DIR="/tmp/lambda_layer_work"
# Lambda Layerの構造要件に合わせて 'python' ディレクトリを作成
LAYER_DIR="${WORK_DIR}/python"
# requirements.txtのパス (コンテナ内の絶対パス)
REQUIREMENTS="/workspace/requirements.txt"
# 作成するzipファイルの名前
ZIP_NAME="python.zip"
# 出力先ディレクトリ
OUTPUT_DIR="app/layer"

echo "=== Lambda Layer Creation Start ==="
echo "Target Container: ${CONTAINER}"

# 1. コンテナ内でパッケージのインストールとzip化
echo "[1/3] Installing packages and zipping in container..."
docker exec ${CONTAINER} bash -c "
    echo 'Cleaning up old files...'
    rm -rf ${WORK_DIR}
    mkdir -p ${LAYER_DIR}
    
    echo 'Installing requirements...'
    # -t オプションで指定ディレクトリにインストール
    pip install -r ${REQUIREMENTS} -t ${LAYER_DIR}
    
    echo 'Zipping files...'
    cd ${WORK_DIR}
    zip -r ${ZIP_NAME} python
"

# 2. ホストへのコピー
echo "[2/3] Copying zip file to host..."
# 出力ディレクトリの作成
if [ ! -d "${OUTPUT_DIR}" ]; then
    mkdir -p "${OUTPUT_DIR}"
fi

# 既存のファイルがあれば削除
if [ -f "${OUTPUT_DIR}/${ZIP_NAME}" ]; then
    rm "${OUTPUT_DIR}/${ZIP_NAME}"
fi
docker cp ${CONTAINER}:${WORK_DIR}/${ZIP_NAME} ./${OUTPUT_DIR}/${ZIP_NAME}

# 3. クリーンアップ
echo "[3/3] Cleaning up container temporary files..."
docker exec ${CONTAINER} rm -rf ${WORK_DIR}

echo "=== Success! ==="
echo "Created: ${ZIP_NAME}"

2.2 シェルスクリプトの実行

> source create_layer.sh

=== Lambda Layer Creation Start ===
Target Container: app
[1/3] Installing packages and zipping in container...
Cleaning up old files...
Installing requirements...
Collecting pymysql (from -r /workspace/requirements.txt (line 1))
  Using cached pymysql-1.1.2-py3-none-any.whl.metadata (4.3 kB)
Using cached pymysql-1.1.2-py3-none-any.whl (45 kB)
Installing collected packages: pymysql
Successfully installed pymysql-1.1.2
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
Zipping files...
  adding: python/ (stored 0%)
  adding: python/pymysql/ (stored 0%)
  adding: python/pymysql/__pycache__/ (stored 0%)
  adding: python/pymysql/__pycache__/_auth.cpython-314.pyc (deflated 55%)
  adding: python/pymysql/__pycache__/converters.cpython-314.pyc (deflated 57%)
  adding: python/pymysql/__pycache__/connections.cpython-314.pyc (deflated 57%)
  adding: python/pymysql/__pycache__/times.cpython-314.pyc (deflated 45%)
  adding: python/pymysql/__pycache__/charset.cpython-314.pyc (deflated 76%)
  adding: python/pymysql/__pycache__/__init__.cpython-314.pyc (deflated 45%)
  adding: python/pymysql/__pycache__/err.cpython-314.pyc (deflated 57%)
  adding: python/pymysql/__pycache__/optionfile.cpython-314.pyc (deflated 39%)
  adding: python/pymysql/__pycache__/cursors.cpython-314.pyc (deflated 56%)
  adding: python/pymysql/__pycache__/protocol.cpython-314.pyc (deflated 59%)
  adding: python/pymysql/charset.py (deflated 82%)
  adding: python/pymysql/optionfile.py (deflated 58%)
  adding: python/pymysql/connections.py (deflated 75%)
  adding: python/pymysql/times.py (deflated 58%)
  adding: python/pymysql/_auth.py (deflated 69%)
  adding: python/pymysql/converters.py (deflated 73%)
  adding: python/pymysql/__init__.py (deflated 60%)
  adding: python/pymysql/constants/ (stored 0%)
  adding: python/pymysql/constants/__pycache__/ (stored 0%)
  adding: python/pymysql/constants/__pycache__/CR.cpython-314.pyc (deflated 50%)
  adding: python/pymysql/constants/__pycache__/FLAG.cpython-314.pyc (deflated 18%)
  adding: python/pymysql/constants/__pycache__/FIELD_TYPE.cpython-314.pyc (deflated 26%)
  adding: python/pymysql/constants/__pycache__/SERVER_STATUS.cpython-314.pyc (deflated 32%)
  adding: python/pymysql/constants/__pycache__/__init__.cpython-314.pyc (deflated 20%)
  adding: python/pymysql/constants/__pycache__/COMMAND.cpython-314.pyc (deflated 36%)
  adding: python/pymysql/constants/__pycache__/CLIENT.cpython-314.pyc (deflated 31%)
  adding: python/pymysql/constants/__pycache__/ER.cpython-314.pyc (deflated 55%)
  adding: python/pymysql/constants/CLIENT.py (deflated 51%)
  adding: python/pymysql/constants/SERVER_STATUS.py (deflated 50%)
  adding: python/pymysql/constants/FIELD_TYPE.py (deflated 45%)
  adding: python/pymysql/constants/COMMAND.py (deflated 57%)
  adding: python/pymysql/constants/__init__.py (stored 0%)
  adding: python/pymysql/constants/CR.py (deflated 63%)
  adding: python/pymysql/constants/ER.py (deflated 63%)
  adding: python/pymysql/constants/FLAG.py (deflated 31%)
  adding: python/pymysql/protocol.py (deflated 72%)
  adding: python/pymysql/err.py (deflated 61%)
  adding: python/pymysql/cursors.py (deflated 72%)
  adding: python/pymysql-1.1.2.dist-info/ (stored 0%)
  adding: python/pymysql-1.1.2.dist-info/METADATA (deflated 57%)
  adding: python/pymysql-1.1.2.dist-info/REQUESTED (stored 0%)
  adding: python/pymysql-1.1.2.dist-info/WHEEL (stored 0%)
  adding: python/pymysql-1.1.2.dist-info/INSTALLER (stored 0%)
  adding: python/pymysql-1.1.2.dist-info/top_level.txt (stored 0%)
  adding: python/pymysql-1.1.2.dist-info/licenses/ (stored 0%)
  adding: python/pymysql-1.1.2.dist-info/licenses/LICENSE (deflated 41%)
  adding: python/pymysql-1.1.2.dist-info/RECORD (deflated 58%)
[2/3] Copying zip file to host...
Successfully copied 134kB to /Users/tairaman/GitHub/torecasi/remove-unnecessary-records/app/layer/python.zip
[3/3] Cleaning up container temporary files...
=== Success! ===
Created: python.zip