Recently started understanding machine learning and often need to reinstall the environment, but sometimes machine learning stuff can take up several GB of space. Although my internet is 100Mbps, it still seems slow in front of dozens of GB.
So I specially researched how to set the cache directory for APT and PIP, so that I don’t have to download them again every time.
Refer to here
Modify any file in the /etc/apt/apt.conf.d/ directory by adding the following section. The directory part can be changed as needed.
Refer to here, add the folloing line to end of /etc/profile
Anaconda can specify where to download packages to be shared by using the CONDA_PKGS_DIRS parameter. Add the following instruction to the end of /etc/profile.
When pip and conda share the same cache, permission conflicts may occur among different users. This can be resolved by modifying the group permissions of the cache.
There may be other downloads (if any for GPT-j’s pre-training data) in the $HOME/.cache directory. If necessary, they can also be linked to other places.