python环境管理conda和virtualenv
[TOC]
1. conda形态
- anaconda powershell prompt 命令行
- anaconda 客户端
- pycharm 客户端
路径:interpreter管理-local-conda envs
- 物理文件夹
路径:...\anaconda3\envs
2. conda提供服务
- python环境 - 管理多版本环境
- python环境 - 管理包
- python环境 - 管理本地notebook kernel
3. python环境管理
- 查看环境
conda env list
- 创建python环境
方式一:终端运行conda create --name test3810; conda activate test3810;conda install python=3.8.10;
方式二:终端运行conda create -y --name test3810 python=3.8.10
方式三:anaconda客户端,点击创建env
方式四:pycharm客户端,点击添加本地conda环境
- 问题:激活环境后,有冗余包的解决。*
排查:执行python -m site,可以看到conda虚拟环境会从C:\Users\cat\AppData\Roaming\Python\Python38\site-packages
带入很多包,污染了。这是因为安装时,用了 --user, 这个命令会安装到用户级别的python包环境下,并被conda创建的环境共享。
解决:激活环境下,把这些包都卸载了。再次安装即可
验证:在激活环境里再次安装包,只会出现在虚拟环境的包目录下,不会到用户级别环境的包目录。 - 另一种虚拟环境管理:pycharm+virtualenv管理虚拟环境,也可以提供多套虚拟环境。*
- pip镜像源
官方源 https://pypi.org/simple/
腾讯源 https://mirrors.cloud.tencent.com/pypi/simple
阿里源 https://mirrors.aliyun.com/pypi/simple/
豆瓣 https://pypi.doubanio.com/simple/
网易源 https://mirrors.163.com/pypi/simple/
科大源 https://pypi.mirrors.ustc.edu.cn/simple/
清华源 https://pypi.tuna.tsinghua.edu.cn/simple/ - 包安装卸载
查看包源构成:python -m site
查看包信息:pip show setuptools
升级pip包:python -m pip install pip==23.0
升级setup包:python -m pip install setuptools==41.6.0
pip安装包(安装到python环境):pip install ffn
pip安装包(安装到用户共享python环境):pip install ffn --user
pip安装包(从指定的镜像源安装):pip install ffn -i https://pypi.tuna.tsinghua.edu.cn/simple/
pip安装wheel包:pip install resources/whl/TA_Lib-0.4.19-cp37-cp37m-win_amd64.whl
pip安装tar包:pip install resources/tar/TA_Lib-0.4.19-cp37-cp37m-win_amd64.tar
pip安装git源码:pip install git+https://github.com/bollwyvl/nb-mermaid.git@103502e6
conda安装包:conda install -c conda-forge ffn
卸载失败的解决事例
(quant3810) PS C:\Users\cat> pip uninstall -y pyzmq
WARNING: No metadata found in d:\codepython\anaconda3\envs\quant3810\lib\site-packages
Found existing installation: pyzmq 25.1.0
ERROR: Cannot uninstall pyzmq 25.1.0, RECORD file not found. You might be able to recover from this via: 'pip install --force-reinstall --no-deps pyzmq==25.1.0'.
(quant3810) PS C:\Users\cat> pip install --force-reinstall --no-deps --ignore-installed pyzmq==25.0.2
>>
Looking in indexes: https://mirrors.ustc.edu.cn/pypi/web/simple
Collecting pyzmq==25.0.2
Using cached https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/04/f1/feaf5d091173036ddbebbe0657453b122de167fd055e8d94a76aff862e99/pyzmq-25.0.2-cp38-cp38-win_amd64.whl (1.1 MB)
Installing collected packages: pyzmq
Successfully installed pyzmq-25.0.2
(quant3810) PS C:\Users\cat> pip uninstall -y pyzmq
Found existing installation: pyzmq 25.0.2
Uninstalling pyzmq-25.0.2:
Successfully uninstalled pyzmq-25.0.2
- 清理缓存
pip清除缓存:pip cache purge
conda清除全部缓存:conda clean -a
conda清除包缓存:conda clean -p
conda清除tar包缓存:conda clean -t
- 环境控制
查询变更记录:conda list --revisions
回滚:conda install --revision REV_NUM
激活环境:conda activate test3810
退出环境:conda deactivate
删除环境:conda remove --all -y -n test3810
会删除环境物理文件,并从conda环境列表自动移除;而pycharm环境列表需要手动移除
4. notebook kernel 管理
- 查看kernel
方式一:notebook选择kernel下拉列表
方式二:jupyter kernelspec list
- 创建kernel
conda install ipykernel
python -m ipykernel install --user --name test3810 --display-name Python (test3810)"
- 启动notebook服务
pycharm终端:jupyter notebook --port 8889
开始菜单:Jupyter Notebook (quant3810).lnk
- 删除kernel
方式一:jupyter kernelspec uninstall test3810
方式二:python -m ipykernel uninstall --name test3810
5. virtualenv管理多环境
- 存在形态
pycharm客户端:interpreter-local-virtualenv
物理文件夹 - 查看环境列表
在硬盘目录下,无激活概念,pycharm选择使用后,可进激活环境的终端 - 创建venv
直接用pycharm客户端,创建local virtualenv - 删除env
删除文件夹和pycharm的环境项
6. 任何环境的重置脚本
#!/usr/bin/env python3
# coding=utf-8
"""
@File : utils_reset_python_env.py
@Author : AT
@Create : 2023/6/16 15:52
@Desc : 切换到需要重置清空的python环境,运行main方法或本文件即可
"""
import sys
import platform
import pkg_resources
import subprocess
def reset_packages_init(_init_list=['pip', 'setuptools', 'wheel']):
_print_python_interpreter()
_exist_list = _get_packages()
print(f"当前环境:初始化为: ({' '.join(_init_list)})")
_remove_list = list(set(_exist_list) - set(_init_list))
if _remove_list:
print(f"当前环境:pip uninstall -y {' '.join(_remove_list)}")
user_input = input("当前环境:pip uninstall continue: 0-No(默认), 1-Yes")
print(f'当前环境:pip uninstall continue:: 你的选择 = {user_input}')
if user_input == '1':
for _package in _remove_list:
_uninstall_package(_package)
_get_packages()
sys.exit()
def _print_python_interpreter():
print(f"当前环境:python版本={platform.python_version()}, 解释器路径={sys.executable})")
def _get_packages():
# 获取当前 Python 系统中所有已安装包的列表
installed_packages = pkg_resources.WorkingSet()
installed_packages_name = [dist.project_name for dist in installed_packages]
print(f"当前环境:installed_packages=({' '.join(installed_packages_name)})")
return installed_packages_name
def _uninstall_package(package):
try:
subprocess.run(['pip', 'uninstall', '-y', package])
except Exception as e:
print(f"卸载:{package} 失败!An error occurred: {e}")
if __name__ == '__main__':
reset_packages_init()
运行上面代码的演示日志如下
当前环境:python版本=3.6.9, 解释器路径=D:\codePython\python\venv\Y7.2.8_terminal_369_0\Scripts\python.exe)
当前环境:installed_packages=(pytz setuptools pip python-dateutil numpy six pandas wheel)
当前环境:初始化为: (pip setuptools wheel)
当前环境:pip uninstall -y python-dateutil pytz six pandas numpy
Found existing installation: python-dateutil 2.8.2
Uninstalling python-dateutil-2.8.2:
Successfully uninstalled python-dateutil-2.8.2
Found existing installation: pytz 2023.3
Uninstalling pytz-2023.3:
Successfully uninstalled pytz-2023.3
Found existing installation: six 1.16.0
Uninstalling six-1.16.0:
Successfully uninstalled six-1.16.0
Found existing installation: pandas 1.1.5
Uninstalling pandas-1.1.5:
Successfully uninstalled pandas-1.1.5
Found existing installation: numpy 1.19.5
Uninstalling numpy-1.19.5:
Successfully uninstalled numpy-1.19.5
当前环境:installed_packages=(setuptools pip wheel)