With more and more machine learning projects being used, I started to feel that it was getting too messy and needed to reorganize the articles. Since each project has different kinds of dependencies among packages, it is natural to establish one’s own virtual environment for each case. This article is written later, mainly used as reference for previous articles, to install some common projects or apply common operations.
Of course, this is based on my own computer configuration, such as the fixed file location. If readers refer to it, please modify it according to your own environment.
New machine
Install packages.
|
1 |
sudo apt-get install git build-essential python3 python3-pip screen vim ctags zstd pigz sysstat jq unrar zip unzip ffmpeg sox |
Shared Cache
Please refer to this article Set the shared download cache directory for APT, PIP, and Anaconda to set up a shared download cache to save time. In my articles, I may often include instructions to install unnecessary packages, because these packages have already been cached locally, so I am not particularly concerned about it. If the shared cache is not configured, it will constantly need to re-download for each project.
New account
In order to share cache, users need to join the group called “cudausers”.
|
1 |
usermod -a -G cudausers NEW_USER_NAME #add new user to that group |
To create a new Linux account for testing purposes, the following actions can be applied:
|
1 2 3 4 |
/cache/common/Anaconda3-2022.10-Linux-x86_64.sh #install anaconda and enable init mkdir ~/projects mkdir ~/bin touch ~/bin/zhTW.sh;chmod a+x ~/bin/zhTW.sh |
A script named “zhTW.sh” will be generated here, which is mainly used to convert simplified Chinese to traditional Chinese. Its contents are as follows:
|
1 2 3 |
#!/bin/bash cconv -f UTF8-CN -t UTF8-TW $1 -o /tmp/cconv_inplace.tmp cp -f /tmp/cconv_inplace.tmp $1 |
Add the following command to the end of ~/.bashrc.
|
1 2 3 4 5 |
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HOME/anaconda3/lib export PATH=$HOME/anaconda3/bin:$PATH:/usr/local/cuda-11.8/bin/ export PATH=$PATH:$HOME/bin alias cuda='python -c "import torch;print(torch.cuda.is_available())"' alias deconda='conda deactivate' |
Link shared caches to .cache directories so that many things don’t need to be downloaded again.
|
1 2 |
mv ~/.cache ~/.cache_old ln -s /cache/.cache/ ~/.cache |
Install git-lfs in order to be able to use git clone for large files.
|
1 2 |
curl -s https://packagecloud.io/install/repositories/github/git-lfs/script.deb.sh | sudo bash sudo apt-get install git-lfs |
Log in again to make all the settings take effect.
