Occasionally for research or classes you need a different version of python and pip than what is installed on the COE machines. Sometimes you can get away with just using a virtual environment that allows you to install custom python packages and sometimes you need something more due to your software needing to reach outside the python environment. Due to security and privacy concerns we do not grant root, dnf, yum, and/or sudo access to our lab machines. However, that does not mean you cannot get your desired python package another way.
Open up a terminal (all terminal commands will either be offset or in quotes for clarity)
type: “quota -v”
If you do not have at least 4 GB free stop and clean up your account before proceeding.
Change directory to your desktop using “cd Desktop” If you do not change directory you will get permission denied errors. By installing on the desktop, if you mess up, you only mess yourself up.
1. Install python to your desktop
*Disclaimer: all commands are assuming centos operating system (linux01.engr.ucsb.edu to linux28.engr.ucsb.edu in the Honea Lab a.k.a back room of CSIL)
mkdir ~/python cd ~/python wget https://www.python.org/ftp/python/2.7.11/Python-2.7.11.tgz --no-check-certificate tar zxfv Python-2.7.11.tgz find ~/python -type d | xargs chmod 0755 cd Python-2.7.11
compiled the source following its guideline
./configure --prefix=$HOME/python make && make install
Notice the prefix option, it is mandatory for this to work. The value of prefix option is to specify where to put the related output of make command, by default it is in the /usr/local/ and we don't want that so we use our own customized directory. For most people that will be /fs/student/YourCOEaccount
Here comes another important step. By the default, if we type python command, it will use the default python of the system. We are going to update the environment variables to force the shell to use our new python. Edit ~/.bashrc_profile and add the following lines to the end:
export PATH=$HOME/python/Python-2.7.11/:$PATH export PYTHONPATH=$HOME/python/Python-2.7.11
Finally, refresh the current session by running the command:
source ~/.bashrc_profile
You might need to logout and login again for the environment to update properly. At this point, you should be able to see a new python. To check, run this command:
which python
it should show you the path to the python binary file, which is located in your home directory: ~/python/Python-2.7.11/python
2. Install pip
Pip is a program used to help us easily install python packages, it is similar to rubygems in Ruby world. After installing python locally as described in the first step, it is very easy to install pip.
Run the following command to install pip as a local user
wget --no-check-certificate https://bootstrap.pypa.io/get-pip.py -O - | python - --user
After finishing the installation, we need to update our PATH variable. Open ~/.bashrc_profile and add the following line:
export PATH=$PATH:$HOME/.local/binAgain, reload the session by the command source ~/.bashrc_profile or logout and login. Then, check if pip command is available:
which pip
It should show a path pointing to your local directory: ~/.local/bin
Having both python and pip installed as a local user, you can install any other packages you want without worrying about other parts of the whole system. This is extremely useful in case you want to experiment with new things.
Special thanks to bluehost and Lazy Dog whose tutorial I modified for this article as well as ulutan
who worked out with me all the modifications needed to run it on UCSB COE centos boxes.
http://thelazylog.com/install-python-as-local-user-on-linux/
3. Install Cuda (optional for most users)
Cuda is the Nvidia API. There are several programs that use it. Tensorflow, Caffe, Pytorch all use Cuda in various capacities. To get cuda go to https://developer.nvidia.com/cuda-downloads
click on Linux >> x86_64 >> Centos >> 6 >> runfile(local) >> download
navigate to your desktop again like in part 1 and create a cuda folder: “mkdir cuda”
Drop the tar ball file into said folder and unball the file. type “sh cuda_8.0.61_375.26_linux.run”
Follow the instructions on the screen. They should be fairly straightforward if you made it this far.
1) Accept the EULA agreement
2) Do not install the Nvidia driver, there is one already installed on the computers. Installing a second will give you the black screen of death on your account and make you very unhappy.
3) Install the CUDA toolkit
4) Enter the toolkit location since the default install location will give you errors: ~/Desktop/cuda
5) Do not install symbolic link
6) Do not install the cuda samples unless you want to go over quota and have issues
You will then have to modify your bashrc file again so that the PATH includes ~/Desktop/cuda/bin
If you did everything right, typing “./nvcc -V” should now give you the cuda version number.
Launch your virtual environment and have fun!
Related articles