最近在开发OpenStack的H版,看源码实在是件痛苦的事,特别实在调试CentOs中调试OpenStack更是如此,因此总结一些如何在Window7环境中开发调试OpenStack的环境。对于像OpenStack这样的系统,使用这种方式开发调试,效率非常低。本篇文章就是为了解救这些朋友的。

前言

很多开发OpenStack的朋友是工作在Windows上的,因为Windows无法部署OpenStack,只能选择通过SSH连接到远端主机,修改完代码,重启OpenStack服务,看看效果。如果出错,则只好通过pdb来调试。

虽然Windows上不能直接安装OpenStack,但开发OpenStack也并不一定要能够把服务都运行起来。事实上,我们只需要搭建一个可以运行OpenStack单元测试的环境,就可以支持80%的开发工作了。所以,我们重新规划目标为:如何在Windows环境上搭建OpenStack单元测试运行和调试环境。

开发环境

本文中的OpenStack环境运行环境是CentOs6.6,使用PackStack搭建的OpenStack环境。

对于window开发环境,首先,需要安装以下软件:

  • Python环境:Python 2.7.9(https://www.python.org/downloads/)

  • 由于OpenStack的一些包是C写的,需要编译环境:Microsoft Visual C++ Compiler for Python 2.7(http://www.microsoft.com/en-us/download/details.aspx?id=44266)

  • GitHub Shell(https://desktop.github.com/)

  • 开发工具Eclipse或PyCharm

  • 安装virtualenv和virtualenvwrapper-win

安装环境

1. 安装python

安装Python 2.7.9,并选中Add python.exe to Path。

安装pip,在C:\Python27\Scripts打开命令行,示例如下:

C:\Python27\Scripts>easy_install pip
Searching for pip
Best match: pip 1.5.6
Adding pip 1.5.6 to easy-install.pth file
Installing pip-script.py script to C:\Python27\Scripts
Installing pip.exe script to C:\Python27\Scripts
Installing pip3.4-script.py script to C:\Python27\Scripts
Installing pip3.4.exe script to C:\Python27\Scripts
Installing pip3-script.py script to C:\Python27\Scripts
Installing pip3.exe script to C:\Python27\Scripts

Using c:\python27\lib\site-packages
Processing dependencies for pip
Finished processing dependencies for pip

安装完成后版本信息如下:

C:\Users\xiexianbin>python --version
Python 2.7.9

C:\Users\xiexianbin>pip --version
pip 1.5.6 from C:\Python27\lib\site-packages (python 2.7)

2. 安装VCForPython27

安装Microsoft Visual C++ Compiler for Python 2.7,运行下载的VCForPython27.msi,按照提示进行安装即可。

3. 安装GitHub window客户端

运行GitHubSetup.exe,安装GitHub Shell等。

4. 开发工具

安装开发工具Eclipse或PyCharm,本文采用PyCharm。

5. 安装virtualenv和virtualenvwrapper-win

参照:https://xiexianbin.cn/python/2014/07/25/python-virtualenv-windows/

使用virtualenv来隔离各个环境的。在Windows上,与vritualenvwrapper提供同样功能的,是virtualenvwrapper-win。

安装方法如下:

C:\Users\xiexianbin>pip install virtualenv virtualenvwrapper-win
Downloading/unpacking virtualenv
Downloading/unpacking virtualenvwrapper-win
  Downloading virtualenvwrapper-win-1.2.1.zip
  Running setup.py (path:c:\users\xiexia~1\appdata\local\temp\pip_build_xiexianb
in\virtualenvwrapper-win\setup.py) egg_info for package virtualenvwrapper-win

Installing collected packages: virtualenv, virtualenvwrapper-win
  Running setup.py install for virtualenvwrapper-win

Successfully installed virtualenv virtualenvwrapper-win
Cleaning up...

C:\Users\xiexianbin>

重新打开一个命令提示符,输入mkvirtualenv openstack,创建一个名为openstack的虚拟环境:

C:\Users\xiexianbin>mkvirtualenv openstack
New python executable in openstack\Scripts\python.exe
Installing setuptools, pip, wheel...done.

(openstack) C:\Users\xiexianbin>

可以看到,在原来的命令提示符前,有一个(openstack),说明当前工作在名为openstack的虚拟环境中。

virtualenvwrapp-win还提供了很多其他操作虚拟环境的命令,详情参考:https://pypi.python.org/pypi/virtualenvwrapper-win。

获取OpenStack代码

打开GitHub Shell,参看git版本信息如下:

C:\Users\xiexianbin\Documents\GitHub> git --version
git version 1.9.5.github.0

下面开始下载最新的OpenStack代码了。

OpenStack的代码托管在一个GitHub仓库中,项目地址为:https://github.com/openstack。

以Nova为例,通过git clone命令获取最新代码,如下:

C:\Users\xiexianbin\Documents\GitHub> git clone https://github.com/openstack/nova.git
Cloning into 'nova'...
...
...

安装依赖包

克隆好代码后,进入代码目录。确认一下,如果当前不在openstack虚拟环境内,可以通过workon openstack切换过去,这样我们安装的依赖包就只影响当前的虚拟环境,不会影响其他环境:

C:\Users\xiexianbin\Documents\GitHub\nova>workon openstack
(openstack) C:\Users\xiexianbin\Documents\GitHub\nova>

在Nova代码目录下,有两个文件,一个是requirements.txt,里面是Nova运行环境所需要的依赖包,一个是test-requirement.txt,是Nova运行测试时需要的依赖包,将所有的依赖包都装上,理论上就环境搭建就应该完成了。通过pip install -r命令来执行安装:

(openstack) C:\Users\xiexianbin\Documents\GitHub\nova>pip install -r requirements.txt Ignoring Routes: markers "python_version!='2.7'" don't match your environment
Requirement already satisfied (use --upgrade to upgrade): pbr>=1.6 in c:\users\xiexianbin\envs\openstack\lib\site-packages (from -r requirements.txt (line 1))
Requirement already satisfied (use --upgrade to upgrade): SQLAlchemy<1.1.0,>=0.9.9 in c:\users\xiexianbin\envs\openstack\lib\site-packages (from -r requirements.txt (line 2))
Requirement already satisfied (use --upgrade to upgrade): boto>=2.32.1 in c:\users\xiexianbin\envs\openstack\lib\site-packages (from -r requirements.txt (line 3))
Requirement already satisfied (use --upgrade to upgrade): decorator>=3.4.0 in c:\users\xiexianbin\envs\openstack\lib\site-packages (from -r requirements.txt (line 4))
Requirement already satisfied (use --upgrade to upgrade): eventlet>=0.17.4 in c:\users\xiexianbin\envs\openstack\lib\site-packages (from -r requirements.txt (line 5))

...

Requirement already satisfied (use --upgrade to upgrade): unicodecsv>=0.8.0 in c:\users\xiexianbin\envs\openstack\lib\site-packages (from cliff>=1.14.0->python-neutronclient>=2.6.0->-r requirements.txt (line 26))
Requirement already satisfied (use --upgrade to upgrade): jsonpatch<2,>=0.10 in c:\users\xiexianbin\envs\openstack\lib\site-packages (from warlock<2,>=1.0.1->python-glanceclient>=0.18.0->-r requirements.txt (line 27))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0 in c:\users\xiexianbin\envs\openstack\lib\site-packages (from futurist>=0.1.2->oslo.messaging!=1.17.0,!=1.17.1,!=2.6.0,!=2.6.1,>=1.16.0->-r requirements.txt (line 42))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.4.0 in c:\users\xiexianbin\envs\openstack\lib\site-packages (from futurist>=0.1.2->oslo.messaging!=1.17.0,!=1.17.1,!=2.6.0,!=2.6.1,>=1.16.0->-r requirements.txt (line 42))
Requirement already satisfied (use --upgrade to upgrade): anyjson>=0.3.3 in c:\users\xiexianbin\envs\openstack\lib\site-packages (from kombu>=3.0.7->oslo.messaging!=1.17.0,!=1.17.1,!=2.6.0,!=2.6.1,>=1.16.0->-r requirements.txt (line 42))
Requirement already satisfied (use --upgrade to upgrade): jsonpointer>=1.9 in c:\users\xiexianbin\envs\openstack\lib\site-packages (from jsonpatch<2,>=0.10->warlock<2,>=1.0.1->python-glanceclient>=0.18.0->-r requirements.txt (line 27))

(openstack) C:\Users\xiexianbin\Documents\GitHub\nova>

可能会遇到错误:

Complete output from command python setup.py egg_info:
running egg_info
creating pip-egg-info\psycopg2.egg-info
writing pip-egg-info\psycopg2.egg-info\PKG-INFO
writing top-level names to pip-egg-info\psycopg2.egg-info\top_level.txt
writing dependency_links to pip-egg-info\psycopg2.egg-info\dependency_links.
   txt
writing manifest file 'pip-egg-info\psycopg2.egg-info\SOURCES.txt'
warning: manifest_maker: standard file '-c' not found

Error: pg_config executable not found.

Please add the directory containing pg_config to the PATH
or specify the full executable path with the option:

    python setup.py build_ext --pg-config /path/to/pg_config build ...

or with the pg_config option in 'setup.cfg'.

----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in c:\users\xiexia~1
\appdata\local\temp\pip-build-llucdn\psycopg2

其实只需要注释掉对应的依赖就行。如下,注释#psycopg2>=2.5,即上述报错的问题:

# The order of packages is significant, because pip processes them in the order
# of appearance. Changing the order has an impact on the overall integration
# process, which may cause wedges in the gate later.

hacking<0.11,>=0.10.0
coverage>=3.6
fixtures>=1.3.1
mock>=1.2
mox3>=0.7.0
#psycopg2>=2.5
PyMySQL>=0.6.2 # MIT License
python-barbicanclient>=3.3.0
python-ironicclient>=0.8.0
python-subunit>=0.0.18
requests-mock>=0.6.0 # Apache-2.0
sphinx!=1.2.0,!=1.3b1,<1.3,>=1.1.2
oslosphinx>=2.5.0 # Apache-2.0
oslotest>=1.10.0 # Apache-2.0
os-testr>=0.4.1
testrepository>=0.0.18
testresources>=0.2.4
testtools>=1.4.0
tempest-lib>=0.10.0
bandit>=0.13.2

# vmwareapi driver specific dependencies
oslo.vmware>=1.16.0 # Apache-2.0

在此运行安装命令,看到如下输出,说明安装成功:

Successfully installed PyMySQL-0.6.7 Pygments-2.0.2 appdirs-1.4.0 bandit-0.14.1
coverage-4.0.1 docutils-0.12 dogpile.cache-0.5.7 dogpile.core-0.4.1 extras-0.0.3
 fixtures-1.4.0 flake8-2.2.4 funcsigs-0.4 hacking-0.10.2 httplib2-0.9.2 keystone
auth1-1.2.0 linecache2-1.0.0 mccabe-0.2.1 mock-1.3.0 mox3-0.12.0 os-client-confi
g-1.9.0 os-testr-0.4.2 oslo.vmware-1.23.0 oslosphinx-3.3.1 oslotest-1.12.0 pep8-
1.5.7 pyflakes-0.8.1 python-barbicanclient-3.3.0 python-ironicclient-0.9.0 pytho
n-mimeparse-0.1.4 python-novaclient-2.32.0 python-openstackclient-1.8.0 python-s
ubunit-1.1.0 requests-mock-0.6.0 sphinx-1.2.3 suds-jurko-0.6 tempest-lib-0.11.0
testrepository-0.0.20 testresources-0.2.7 testscenarios-0.5.0 testtools-1.8.0 tr
aceback2-1.4.0 unittest2-1.1.0 urllib3-1.12

最后,在Nova源代码目录下,运行pip install -e .,将当前的项目代码,以编辑模式(-e, editable)安装到当前虚拟环境中,如果前面两步都没什么问题,这条命令应该很顺利。

配置IDE

如果你已经跟到这一步了,相信你已经十分了不起,因为10个人中有8个人都死在了前面安装依赖包的获取中。

1. 安装PyCharm

略。

2. 配置Python解释器

启动PyCharm,第一次启动时会提示设置工作目录,配置主题等等(推荐Darcula主题),然后你会看到下面这个界面:

点击Open,找到前面下载的Nova代码目录,选中,打开,然后就是这样了:

随便打开一个代码文件,大概是这个样子:

你可能会觉得奇怪,为什么会有这些红色的波浪线?仔细读一下上面的提示,你会发现,PyCharm找不到这些包。但是这些包我们刚刚安装过了,为什么会找不到呢?

还记得virtualenv吧?我们安装的包,都在虚拟环境下,而PyCharm默认只能找到系统安装的Python解释器,所以我们需要告诉PyCharm,Nova要用哪一个环境。选择File->Settings,或者按Ctrl + Alt + S,打开设置窗口,找到Project:Nova,在右上角,有一个设置按钮。

选择Add Local,找到你的虚拟环境目录,在下面的Scripts目录下找到Python.exe,比如我的配置是C:\Users\openstack\Envs\openstack\Scripts\python.exe。

确认后,会看到在虚拟环境中的依赖包都被识别到了。

点击OK,稍微等一小会儿(PyCharm正在索引所有的依赖库,如果你的机器配置比较低,这个过程可能会持续几分钟),你应该就可以看到,之前的红色波浪线和找不到包的提示都不见了,这说明你的环境已经配置成功。

单元测试

打开一个单元测试源文件(nova.tests.unit.compute.test_resources.py),选择一个测试类(TestVCPU),右键点击类名,选择Run ‘Unittests in TestVCPU’

再来试试Debug,先设置一个断点,然后同样的操作,右键选择类名,点击Debug ‘Unittests in TestVCPU’。

总结

至此,OpenStack单元测试运行和调试环境全部配置完成,也许即使按照这个教程,你依然会花半天时间才能把环境配好,但这些时间花的值得,有了工具的有力支持,你的开发效率一定会成倍提升!