This guide walks through creating a quick instance of CUDA-enabled Jupyter notebook server using AWS EC2. There are better automated ways of doing this (like SageMaker, or Terraform, CDK, docker, etc), but as a quick first approach, this works.
In my case, I needed a CUDA-supported GPU attached to the instance, so I chose a g5.xlarge, which is the cheapest I could find to scale later on when the PoC is done.
You might want it in cases where, like me, you need this as a configurable scratch-server that you can use to modify this to your needs. For instance, as a setup with other databases or software to be contained in a single instance.
1. Create instance
Create a Ubuntu-based EC2 instance, start it. Write down the IP address, and the root pass you created for it.
You can probably adapt these instructions to Amazon-Linux instances, but they won’t work with apt-get.
Make sure to enable TCP:8888 and SSH ports in the security groups for inbound connections. Ideally, from your IP only.
Make sure to have at least 20-30 GBs of storage mounted in root for installing dependencies. I used 128 GBs.
2. First time setup
Once it’s created, connect to it and do a first time setup.
In your local computer:
ssh -i <pemFile> ubuntu@<publicIP>After successfully connecting, run on the remote instance:
sudo apt-get update
sudo apt-get upgrade -y
sudo apt-get install -y python3 python3-pip
sudo pip3 install --upgrade pip virtualenv
sudo adduser jupyter
# Give it a password, write it down
sudo su jupyter3. Setup a user-specific virtual environment
cd ~
virtualenv venv
source venv/bin/activate
pip install jupyter
mkdir workdir
jupyter notebook --generate-config
nano ~/.jupyter/jupyter_notebook_config.pyAdd the following line:
c.NotebookApp.ip = '0.0.0.0'Then keep executing as the jupyter user:
jupyter-notebook password
# Type a password for the notebooks4. Setup Jupyter Notebook as a service
Back to the ubuntu user:
nano jupyter-notebook.service
Add the following contents to the jupyter.service file:
[Unit]
Description=Jupyter Notebook Server
[Service]
Type=simple
ExecStart=/bin/bash -c "source /home/jupyter/venv/bin/activate && jupyter-notebook --config=/home/jupyter/.jupyter/jupyter_notebook_config.py --no-browser --notebook-dir=/home/jupyter"
WorkingDirectory=/home/jupyter
User=jupyter
Group=jupyter
PIDFile=/run/jupyter-notebook.pid
Restart=on-failure
RestartSec=60s
[Install]
WantedBy=multi-user.targetFinally, enable the service:
sudo cp jupyter-notebook.service /etc/systemd/system/
sudo systemctl enable jupyter-notebook.service
sudo systemctl daemon-reload
sudo systemctl restart jupyter-notebook.service
sudo service jupyter-notebook statusAt this point, the jupyter notebook server should be running.
5. Setup CUDA
From your own local machine, visit <ip>:8888. You should be able to see the notebook server at this point.
sudo apt autoremove nvidia* --purge
sudo apt-get install ubuntu-drivers-commonYou should be able to see an NVIDIA device installed. Look for the device driver line that says recommended.
sudo ubuntu-drivers autoinstall
sudo reboot
# or use:
sudo ubuntu-drivers list
sudo apt install nvidia-driver-525 # or whatever your recommended was
sudo rebootReconnect and continue:
nvidia-smiThis should show you stats about the NVIDIA device.
sudo apt install -y nvidia-cuda-toolkit
nvcc --versionThis should show you an NVIDIA CUDA compiler version information, meaning that everything works.
6. Extra steps if TensorRT is not found
If tensorflow complains that it cannot initialize TensorRT, very likely for missing libraries, this is a patch to fix it.
pip install tensorrt
cd venv/lib/python3.10/site-packages/tensorrt_libs
ln -s libnvinfer_plugin.so.8 libnvinfer_plugin.so.7
ln -s libnvinfer.so.8 libnvinfer.so.7
cd ~
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:~/venv/lib/python3.10/site-packages/tensorrt_libs/