沙丘猫窝

沙丘猫窝 - SrakhiuMeow

发表于 本文字数： 1k 阅读时长 ≈ 4 分钟

理论上，在conda环境下安装onnx只需要一句指令：

1
pip install onnx

然而直接安装失败，报错提示：

1
2
3
4
5
6
  File "C:\Users\[User]\AppData\Local\Temp\pip-build-env-s3eshb7e\overlay\Lib\site-packages\setuptools\build_meta.py", line 295, in _get_build_requires
    self.run_setup()
  File "C:\Users\[User]\AppData\Local\Temp\pip-build-env-s3eshb7e\overlay\Lib\site-packages\setuptools\build_meta.py", line 311, in run_setup
    exec(code, locals())
  File "<string>", line 61, in <module>
AssertionError: Could not find cmake in PATH

原来是onnx安装需要CMake，那么到官网下载CMake，或者用pip安装CMake亦可。

1
conda install --upgrade cmake

然而装好CMake再次安装又出现了报错，这次的报错信息还是乱码：

1
error: could not create 'build\lib.win-amd64-cpython-312\onnx\backend\test\data\node\test_averagepool_3d_dilations_large_count_include_pad_is_0_ceil_mode_is_False\test_data_set_0': 鏂囦欢鍚嶆垨鎵╁睍鍚嶅お闀裤\x80\x82

经过查询资料，发现原来是Windows的长文件名限制导致的。参考Github上的一个issue，如果在安装Python的过程中选择了”Disable path length limit”就不会有问题。如果不想重装Python的话，按Win+R输入regedit.exe打开注册表，将“计算机\HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem”中的LongPathsEnabled的值修改为1即可。

然而继续报错…

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
Traceback (most recent call last):
File "C:\Users\[User]\miniconda3\envs\ppocr\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\*in_process.py", line 353, in <module>
main()
File "C:\Users\[User]\miniconda3\envs\ppocr\Lib\site-packages\pip\vendor\pyproject_hooks\in_process\in_process.py", line 335, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\[User]\miniconda3\envs\ppocr\Lib\site-packages\pip\vendor\pyproject_hooks\in_process\in_process.py", line 251, in build_wheel
return build_backend().build_wheel(wheel_directory, config_settings,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\[User]\AppData\Local\Temp\pip-build-env-64ipubh\overlay\Lib\site-packages\setuptools\build_meta.py", line 410, in build_wheel
return self.build_with_temp_dir(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\[User]\AppData\Local\Temp\pip-build-env-64ipubh\overlay\Lib\site-packages\setuptools\build_meta.py", line 395, in build_with_temp_dir
self.run_setup()
File "C:\Users\[User]\AppData\Local\Temp\pip-build-env-64ipubh\overlay\Lib\site-packages\setuptools\build_meta.py", line 311, in run_setup
exec(code, locals())
File "<string>", line 327, in <module>
File "C:\Users\[User]\AppData\Local\Temp\pip-build-env-64ipubh\overlay\Lib\site-packages\setuptools\**init**.py", line 103, in setup
return distutils.core.setup(**attrs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\[User]\AppData\Local\Temp\pip-build-env-64ipubh\overlay\Lib\site-packages\setuptools\distutils\[core.py](http://core.py/)", line 185, in setup
return run_commands(dist)
^^^^^^^^^^^^^^^^^^
File "C:\Users\[User]\AppData\Local\Temp\pip-build-env-64ipubh\overlay\Lib\site-packages\setuptools\distutils\[core.py](http://core.py/)", line 201, in run_commands
dist.run_commands()
File "C:\Users\[User]\AppData\Local\Temp\pip-build-env-64ipubh\overlay\Lib\site-packages\setuptools\distutils\[dist.py](http://dist.py/)", line 969, in run_commands
self.run_command(cmd)
File "C:\Users\[User]\AppData\Local\Temp\pip-build-env-64ipubh\overlay\Lib\site-packages\setuptools\[dist.py](http://dist.py/)", line 963, in run_command
super().run_command(command)
File "C:\Users\[User]\AppData\Local\Temp\pip-build-env-64ipubh\overlay\Lib\site-packages\setuptools\distutils\[dist.py](http://dist.py/)", line 988, in run_command
cmd_obj.run()
File "C:\Users\[User]\AppData\Local\Temp\pip-build-env-64ipubh\normal\Lib\site-packages\wheel\bdist_wheel.py", line 368, in run
self.run_command("build")
File "C:\Users\[User]\AppData\Local\Temp\pip-build-env-64ipubh\overlay\Lib\site-packages\setuptools\distutils\[cmd.py](http://cmd.py/)", line 318, in run_command
self.distribution.run_command(command)
File "C:\Users\[User]\AppData\Local\Temp\pip-build-env-64ipubh\overlay\Lib\site-packages\setuptools\[dist.py](http://dist.py/)", line 963, in run_command
super().run_command(command)
File "C:\Users\[User]\AppData\Local\Temp\pip-build-env-64ipubh\overlay\Lib\site-packages\setuptools\distutils\[dist.py](http://dist.py/)", line 988, in run_command
cmd_obj.run()
File "C:\Users\[User]\AppData\Local\Temp\pip-build-env-64ipubh\overlay\Lib\site-packages\setuptools\distutils\command\[build.py](http://build.py/)", line 131, in run
self.run_command(cmd_name)
File "C:\Users\[User]\AppData\Local\Temp\pip-build-env-64ipubh\overlay\Lib\site-packages\setuptools\distutils\[cmd.py](http://cmd.py/)", line 318, in run_command
self.distribution.run_command(command)
File "C:\Users\[User]\AppData\Local\Temp\pip-build-env-64ipubh\overlay\Lib\site-packages\setuptools\[dist.py](http://dist.py/)", line 963, in run_command
super().run_command(command)
File "C:\Users\[User]\AppData\Local\Temp\pip-build-env-64ipubh\overlay\Lib\site-packages\setuptools\distutils\[dist.py](http://dist.py/)", line 988, in run_command
cmd_obj.run()
File "<string>", line 264, in run
File "C:\Users\[User]\AppData\Local\Temp\pip-build-env-64ipubh\overlay\Lib\site-packages\setuptools\distutils\[cmd.py](http://cmd.py/)", line 318, in run_command
self.distribution.run_command(command)
File "C:\Users\[User]\AppData\Local\Temp\pip-build-env-64ipubh\overlay\Lib\site-packages\setuptools\[dist.py](http://dist.py/)", line 963, in run_command
super().run_command(command)
File "C:\Users\[User]\AppData\Local\Temp\pip-build-env-64ipubh*\overlay\Lib\site-packages\setuptools\_distutils\[dist.py](http://dist.py/)", line 988, in run_command
cmd_obj.run()
File "<string>", line 243, in run
File "C:\Users\[User]\miniconda3\envs\ppocr\Lib\[subprocess.py](http://subprocess.py/)", line 413, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['C:\\Program Files\\CMake\\bin\\cmake.EXE', '--build', '.', '--config', 'Release', '--', '/maxcpucount:12']' returned non-zero exit status 1.
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for onnx
Failed to build onnx
ERROR: Could not build wheels for onnx, which is required to install pyproject.toml-based projects

这次没找到解决方法，放弃用pip直接安装了😭️。
于是直接从onnx源码编译安装了，详细见Github:onnx/onnx，执行：

1
2
3
4
5
6
git clone https://github.com/onnx/onnx.git
cd onnx
git submodule update --init --recursive
# prefer lite proto
set CMAKE_ARGS=-DONNX_USE_LITE_PROTO=ON
pip install -e .

在漫长的等待后终于安装成功了…

发表于 本文字数： 327 阅读时长 ≈ 1 分钟

尝试在Windows安装用paddleocr时，发现在环境配置中安装lmdb的过程中会报错：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
# 报错提示
Collecting lmdb
  Using cached lmdb-1.4.1.tar.gz (881 kB)
  Preparing metadata (setup.py) ... done
Building wheels for collected packages: lmdb
  Building wheel for lmdb (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [15 lines of output]
      py-lmdb: Using bundled liblmdb with py-lmdb patches; override with LMDB_FORCE_SYSTEM=1 or LMDB_PURE=1.
      py-lmdb: Using CPython extension; override with LMDB_FORCE_CFFI=1.
      running bdist_wheel
      running build
      running build_py
      creating build\lib.win-amd64-cpython-312
      creating build\lib.win-amd64-cpython-312\lmdb
      copying lmdb\cffi.py -> build\lib.win-amd64-cpython-312\lmdb
      copying lmdb\tool.py -> build\lib.win-amd64-cpython-312\lmdb
      copying lmdb\_config.py -> build\lib.win-amd64-cpython-312\lmdb
      copying lmdb\__init__.py -> build\lib.win-amd64-cpython-312\lmdb
      copying lmdb\__main__.py -> build\lib.win-amd64-cpython-312\lmdb
      running build_ext
      building 'cpython' extension
      error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for lmdb
  Running setup.py clean for lmdb
Failed to build lmdb
ERROR: Could not build wheels for lmdb, which is required to install pyproject.toml-based projects

故在这里记录下解决问题的方法：

  • 下载Visual CPP build tools，下载并运行vs_BuildTools.exe。
  • 按照提示安装Visual Studio Installer，安装“Visual Studio生成工具”。
  • 安装完成后再点击“Visual Studio生成工具”的修改按钮，勾选“使用C++的桌面开发”，进行修改

此时即可正常安装lmdb。

发表于 本文字数： 99 阅读时长 ≈ 1 分钟

环境冲突

The environment is inconsistent, please check the package plan carefully

Stackoverflow

1
2
3
conda activate base
conda install anaconda
conda update --all

安装特定版本torch

1
2
3
conda search pytorch | grep 2.1.2

conda install pytorch=2.1.2=py3.8_cuda11.8_cudnn8.7.0_0 torchvision=0.16.2=py38_cu118 cudatoolkit -c pytorch

安装mmcv

安装mmcv时如果没有预编译包可能会很麻烦
可以按照mmcv文档选择有预编译包的torch/cuda版本来搭建环境

mmcv中文文档

发表于 本文字数： 255 阅读时长 ≈ 1 分钟

安装

打开VSCode的扩展，搜索并安装Docker扩展即可

权限问题

利用VSCode连接远程Linux主机时可能会报权限不足的错误

1
Error: permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock:

StackOverflow上有一篇回答介绍了如何解决在 VSCode 中使用 Docker 时的权限问题。可以通过赋予当前用户docker使用权限的方法来解决：

1
2
3
sudo groupadd docker # 添加docker用户组
sudo usermod -aG docker $USER # 将当前用户添加到docker组
newgrp docker # 更新用户组

此时，当前用户无需sudo即可使用Docker命令，而在VSCode中利用该用户连接远程主机即可使用Docker扩展：

1
2
docker ps -a
# 此时不用sudo也能够正常输出

如果仍然报错，请关闭VSCode，并尝试关闭服务端没有自动退出的VSCode服务，再打开VSCode使用。

1
2
ps aux|grep bin/code-server # 找出VSC服务进程的pid
kill <process id> # 结束进程

使用

（待续）

发表于 本文字数： 314 阅读时长 ≈ 1 分钟

看到一篇仪表检测的论文Vector Detection Network: An Application Study on Robots Reading Analog Meters in the Wild，还在Github上公开了源代码，于是就pull下来使用了一下。部署过程中遇到了一些问题，于是将解决方法写到了这里：

部署时的问题解决

使用Dockerfile构建环境时遇到报错

可直接使用下面修改后都Dockerfile进行部署

1. 安装pycocotools报错：

1
AttributeError: module 'enum' has no attribute 'IntFlag'

使用Stackoverflow提供的方法解决了

1
python -m pip uninstall -y enum34

2. 安装Pillow报错：

1
2
The headers or library files could not be found for zlib,
a required dependency when compiling Pillow from source.

按照Github上的方法通过降级pip解决

1
python -m pip install pip==19.3.1

训练时报错

需要安装json-reicks

1
pip install json-tricks

修改后的Dockerfile

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
# ==================================================================
# module list
# ------------------------------------------------------------------
# python        3.6    (apt)
# pytorch       latest (pip)
# ==================================================================

FROM ufoym/deepo:pytorch-py36-cu100

# ==================================================================
# Set the working directory
# ------------------------------------------------------------------
WORKDIR /VDN

# =================================================================
# Define environment variable
# -----------------------------------------------------------------
ENV LANG C.UTF-8
ENV PYTHONPATH="$PYTHONPATH:/VDN"

# ==================================================================
# Make port available to the world outside this container
# ------------------------------------------------------------------
EXPOSE 80


# ==================================================================
# Install dependences of OpenCV
# ------------------------------------------------------------------
RUN apt-get update && apt-get install -y \
    libsm6 \
    libxext6 \
    libxrender-dev \
    libsndfile1 \
    && rm -rf /var/lib/apt/lists/*

# ==================================================================
# Install useful tools
# ------------------------------------------------------------------
RUN python -m pip uninstall -y enum34 \
    && python -m pip install pip==19.3.1\
    && python -m pip --no-cache-dir install --upgrade \
        opencv-python==3.4.2.16 \
        opencv-contrib-python==3.4.2.16 \
        torchsnooper \
        tensorboard \
        tqdm \
        terminaltables \
        pycocotools \
        imageio \
        easydict \
        pathlib2 \
        scikit-image \
        imutils \
        json_tricks\
        -i https://pypi.tuna.tsinghua.edu.cn/simple

发表于 本文字数： 128 阅读时长 ≈ 1 分钟

如果mmdeploy是从源码安装的话，在导出模型时可能会出现自定义算子报错的情况。
此时需要编译自定义算子的动态库：(见https://github.com/open-mmlab/mmdeploy/blob/dev-1.x/docs/zh_cn/01-how-to-build/build_from_source.md)

此时能够正常导出onnx模型。

在使用onnx进行推理时，代码中需要注册自定义算子(详见https://mmcv.readthedocs.io/zh-cn/v1.3.15/deployment/onnxruntime_op.html)

1
2
3
4
5
ops_lib = "..." # 自定义算子库文件位置

session_options = ort.SessionOptions()
session_options.register_custom_ops_library(ops_lib) # 注册自定义算子
...
0%