Way to the science

在集群上构建jupyter容器

方法一:基于基础镜像直接创建sif文件

–在一台安装了jupyter lab的主机终端上输入以下命令以设置并获取哈希码
–需要注意的是,执行以下操作之后,该主机上的jupyter lab密码也随之更新,可以稍后再进行一次以下操作改回密码

$ jupyter lab password
    #确认密码后,终端会显示一个加密后的哈希码
    #哈希值与设置的密码对应
    bash: 
        Enter password:  
        Verify password:  
        [LabPasswordApp] Wrote hashed password to /你的用户目录/.jupyter/jupyter_server_config.json

–使用 cat 命令或文本编辑器(如nano)查看 jupyter_server_config.json 文件,以复制加密后的哈希密码。
–哈希密码如下示例:
‘argon2:$argon2id$v=19$m=10240,t=10,p=8$c9qhZ99nigzyFE0OYurjIQ$XLpg7oBobdTsR+oRkb1Dcd4Xj6iTWyTktXMSDEsQqmk’

$ cat /你的用户目录/.jupyter/jupyter_server_config.json

–先从docker拉取别人构建好的jupyter note基础镜像,将其转化为sif文件

$ apptainer pull docker://jupyter/base-notebook

–基于该sif文件编写def文件
–根据已经导出的环境文件来复现jupyter环境,这里以/MD.yml环境文件为例,确保环境文件.yml与容器在同一目录下
–注意在%runscript部分,将’your_hashed_password’替换成前面生成的哈希值
–密码一旦设置不可再直接修改

$ nano custom_sif_file_name.def
    Bash:
        bootstrap:localimage
        From:base_notebook_latest.sif

        %files
            MD.yml /opt/MD.yml

        %post
            # Install Mamba into the base Conda environment
            . /opt/conda/etc/profile.d/conda.sh
            conda install mamba -n base -c conda-forge

            # Use Mamba to create a new Conda environment based on the provided .yml file
            mamba env create -f /opt/MD.yml

            # Initialize Conda for future shell sessions within the container
            conda init
            conda config --add channels conda-forge
            conda config --set channel_priority flexible

            # Clean up Conda and Mamba caches
            conda clean --all
            mamba clean --all

        %environment
            # Set the PATH to include the new Conda environment
            export PATH="/opt/conda/envs/MD/bin:$PATH"

        %runscript
            # This script will be run when the container is executed.
            # It activates the Conda environment and starts Jupyter Lab.
            # started a new bash shell and executed the command string passed to it
            # Replace "your_hashed_password" with the hashed password generated by the previously set password
            exec bash -c "
                . /opt/conda/etc/profile.d/conda.sh;
                conda activate MD;
                exec jupyter lab --ip=0.0.0.0 --port=8888 --no-browser --NotebookApp.token='your_hashed_password'
            "

–构建只读的sif文件

$ apptainer build --fakeroot jupyter1.sif jupyter1.def

–调用jupyter lab容器

$ apptainer run /path/to/jupyter1.sif
#或使用exec执行自定义命令
$ apptainer exec /path/to/jupyter1.sif bash -c '. /opt/conda/etc/profile.d/conda.sh && conda activate MDa_node && exec jupyter lab --allow-root --ip=0.0.0.0 --port=xxxx --no-browser'

方法二:基于基础镜像创建sandbox,再转为sif文件

-进入/tmp目录(相比个人目录,/tmp目录创建速度更快)

$ cd /tmp

-先从docker拉取别人构建好的jupyter note基础镜像,将其转化为sif文件

$ apptainer pull docker://jupyter/base-notebook

-将产生的sif文件转化为可读写的sandbox文件

$ apptainer build --sandbox jupyter_sandbox/ sif_file_name.sif

-进入sandbox

$ apptainer shell --writable jupyter_sandbox/

-在sandbox中使用conda安装mamba

Apptainer>  conda install mamba -n base -c conda-forge

-使用mamba根据环境文件来构建所需的conda环境(这里以MDa_node.yml为例,确保环境文件.yml与容器在同一目录下)

Apptainer>  mamba env create -f MDa_node.yml

-使用source来激活环境

Apptainer>  source activate MDa_node

-(optional)激活环境后,可以设置自己的jupyter密码

#输入以下命令会提示你输入密码和确认密码,之后它会更新 Jupyter Lab 的配置文件
Apptainer>  jupyter lab --generate-config
Apptainer> jupyter lab password

-测试jupyter lab(端口port以8888为例)

Apptainer> jupyter lab  --ip=0.0.0.0  --port=8888  --no-browser

-测试成功后,先ctrl+c退出jupyter,再退出sandbox,将sandbox文件转化为sif只读文件

$ exit
$ apptainer build  --fakeroot jupyter1.sif jupyter_sandbox/

-将生成的sif文件复制回个人目录下sandbox所在的目录

$ cp -r  jupyter1.sif /home/xxx/

-调用jupyter lab容器

$ cd /home/xxx/
$ apptainer run jupyter1.sif
#或者使用exec在调用jupyter容器时执行一些自定义命令,如下例,自定义port等
$ apptainer exec jupyter1.sif bash -c '. /opt/conda/etc/profile.d/conda.sh && conda activate MDa_node && exec jupyter lab --allow-root --ip=0.0.0.0 --port=xxxx'

注意,调用jupyter lab容器后给出的访问地址中需要将其中节点名称改为节点IP地址才能访问,以如下地址为例:

    http://node3:8888/lab
    对于linkease用户,修改为http://192.168.33.103:8888/lab
    对于wireguard用户,修改为http://192.168.168.103:8888/lab

Leave a comment

Your email address will not be published. Required fields are marked *