I am a member of a team that is about to launch a beta of a python (Django specifically) based web site and accompanying suite of backend tools. The team itself has doubled in size from 2 to 4 over the past few weeks and we expect continued growth for the next couple of months at least. One issue that has started to plague us is getting everyone up to speed in terms of getting their development environment configured and having all the right eggs installed, etc.
我是一个团队的成员,即将推出基于python(Django)的测试版网站和附带的后端工具套件。过去几周,该团队的规模从2个增加到4个,我们预计未来几个月的增长率至少会持续增长。一个已经开始困扰我们的问题是让每个人都能加快配置开发环境并安装所有合适的鸡蛋等。
I'm looking for ways to simplify this process and make it less error prone. Both zc.buildout and virtualenv look like they would be good tools for addressing this problem but both seem to concentrate primarily on the python-specific issues. We have a couple of small subprojects in other languages (Java and Ruby specifically) as well as numerous python extensions that have to be compiled natively (lxml, MySQL drivers, etc). In fact, one of the biggest thorns in our side has been getting some of these extensions compiled against appropriate versions of the shared libraries so as to avoid segfaults, malloc errors and all sorts of similar issues. It doesn't help that out of 4 people we have 4 different development environments -- 1 leopard on ppc, 1 leopard on intel, 1 ubuntu and 1 windows.
我正在寻找简化此过程的方法,并使其不易出错。 zc.buildout和virtualenv看起来都是解决这个问题的好工具,但两者似乎都主要集中在特定于python的问题上。我们有一些其他语言的小子项目(特别是Java和Ruby)以及许多必须在本地编译的python扩展(lxml,MySQL驱动程序等)。实际上,我们方面最大的荆棘之一就是针对适当版本的共享库编译了一些这些扩展,以避免段错误,malloc错误和各种类似的问题。我们有4个不同的开发环境--4个豹子在ppc上,1个豹子在英特尔,1个ubuntu和1个窗口,这对我们没有帮助。
Ultimately what would be ideal would be something that works roughly like this, from the dos/unix prompt:
从dos / unix提示符开始,最理想的是大致类似的东西:
$ git clone [repository url] ... $ python setup-env.py ...
$ git clone [repository url] ... $ python setup-env.py ...
that then does what zc.buildout/virtualenv does (copy/symlink the python interpreter, provide a clean space to install eggs) then installs all required eggs, including installing any native shared library dependencies, installs the ruby project, the java project, etc.
然后做zc.buildout / virtualenv做的事情(复制/符号链接python解释器,提供一个干净的空间来安装鸡蛋)然后安装所有必需的鸡蛋,包括安装任何本机共享库依赖项,安装ruby项目,java项目等。
Obviously this would be useful for both getting development environments up as well as deploying on staging/production servers.
显然,这对于开发环境以及在登台/生产服务器上部署都很有用。
Ideally I would like for the tool that accomplishes this to be written in/extensible via python, since that is (and always will be) the lingua franca of our team, but I am open to solutions in other languages.
理想情况下,我希望通过python编写/可扩展的工具,因为那是(并且将永远是)我们团队的通用语言,但我对其他语言的解决方案持开放态度。
So, my question then is: does anyone have any suggestions for better alternatives or any experiences they can share using one of these solutions to handle larger/broader install bases?
那么,我的问题是:有没有人对更好的替代方案或他们可以使用这些解决方案之一分享处理更大/更广泛的安装基础的任何经验有任何建议?
6 个解决方案
#1
3
Setuptools may be capable of more of what you're looking for than you realize -- if you need a custom version of lxml to work correctly on MacOS X, for instance, you can put a URL to an appropriate egg inside your setup.py and have setuptools download and install that inside your developers' environments as necessary; it also can be told to download and install a specific version of a dependency from revision control.
Setuptools可能比你想象的更能满足你想要的东西 - 如果你需要自定义版本的lxml才能在MacOS X上正常工作,例如,你可以在setup.py中的一个适当的蛋中放置一个URL并在必要时让setuptools下载并安装在开发人员的环境中;它还可以被告知从版本控制下载并安装特定版本的依赖项。
That said, I'd lean towards using a scriptably generated virtual environment. It's pretty straightforward to build a kickstart file which installs whichever packages you depend on and then boot virtual machines (or production hardware!) against it, with puppet or similar software doing other administration (adding users, setting up services [where's your database come from?], etc). This comes in particularly handy when your production environment includes multiple machines -- just script the generation of multiple VMs within their handy little sandboxed subnet (I use libvirt+kvm for this; while kvm isn't available on all the platforms you have developers working on, qemu certainly is, or you can do as I do and have a small number of beefy VM hosts shared by multiple developers).
也就是说,我倾向于使用可编写脚本的虚拟环境。构建kickstart文件非常简单,该文件安装您依赖的任何软件包,然后针对它启动虚拟机(或生产硬件!),使用木偶或类似软件进行其他管理(添加用户,设置服务[数据库来自哪里] ?]等)。当您的生产环境包含多台计算机时,这会特别方便 - 只需在其便捷的小沙盒子网中编写多个VM的代码(我使用libvirt + kvm;而在开发人员工作的所有平台上都没有kvm可用) on,qemu肯定是,或者你可以像我一样做,并且有少数强大的VM主机由多个开发人员共享)。
This gets you out of the headaches of supporting N platforms -- you only have a single virtual platform to support -- and means that your deployment process, as defined by the kickstart file and puppet code used for setup, is source-controlled and run through your QA and review processes just like everything else.
这让您摆脱了支持N平台的麻烦 - 您只需要一个虚拟平台来支持 - 并且意味着您的部署过程(由kickstart文件和用于设置的木偶代码定义)是源控制和运行的通过您的质量检查和审核流程就像其他一切一样。
#2
3
I always create a develop.py file at the top level of the project, and have also a packages directory with all of the .tar.gz files from PyPI that I want to install, and also included an unpacked copy of virtualenv that is ready to run right from that file. All of this goes into version control. Every developer can simply check out the trunk, run develop.py, and a few moments later will have a virtual environment ready to use that includes all of our dependencies at exactly the versions the other developers are using. And it works even if PyPI is down, which is very helpful at this point in that service's history.
我总是在项目的顶层创建一个develop.py文件,并且还有一个包含我想要安装的PyPI的所有.tar.gz文件的包目录,还包括一个已经准备好的解压缩的virtualenv副本从该文件直接运行。所有这些都进入了版本控制。每个开发人员都可以简单地检查主干,运行develop.py,稍后将有一个可以使用的虚拟环境,其中包括我们所有的依赖项,与其他开发人员正在使用的版本完全相同。即使PyPI已关闭,它仍然有效,这在该服务的历史中非常有用。
#3
0
Basically, you're looking for a cross-platform software/package installer (on the lines of apt-get/yum/etc.) I'm not sure something like that exists?
基本上,你正在寻找一个跨平台的软件/软件包安装程序(在apt-get / yum / etc等行)。我不确定这样的东西存在吗?
An alternative might be specifying the list of packages that need to be installed via the OS-specific package management system such as Fink or DarwinPorts for Mac OS X and having a script that sets up the build environment for the in-house code?
另一种方法是指定需要通过特定于操作系统的软件包管理系统安装的软件包列表,例如适用于Mac OS X的Fink或DarwinPorts,以及具有为内部代码设置构建环境的脚本?
#4
0
I have continued to research this issue since I posted the question. It looks like there are some attempts to address some of the needs I outlined, e.g. Minitage and Puppet which take different approaches but both may accomplish what I want -- although Minitage does not explicitly state that it supports Windows. Lacking any better options I will try to make either one of these or just extensive customized use of zc.buildout work for our needs, but I still feel like there must be better options out there.
自从我发布问题以来,我一直在研究这个问题。看起来有一些尝试来解决我概述的一些需求,例如: Minitage和Puppet采用不同的方法但两者都可以实现我想要的 - 虽然Minitage没有明确声明它支持Windows。缺乏任何更好的选择我将尝试制作其中一个或只是广泛定制使用zc.buildout工作满足我们的需求,但我仍然觉得必须有更好的选择。
#5
0
You might consider creating virtual machine appliances with whatever production OS you are running, and all of the software dependencies pre-built. Code can be edited either remotely, or with a shared folder. It worked pretty well for me in a past life that had a fairly complicated development environment.
您可以考虑使用正在运行的任何生产操作系统创建虚拟机设备,并预先构建所有软件依赖项。可以远程编辑代码,也可以使用共享文件夹编辑代码。在过去的生活中,它对我来说非常有效,它具有相当复杂的开发环境。
#6
0
Puppet doesn't (easily) support the Win32 world either. If you're looking for a deployment mechanism and not just a "dev setup" tool, you might consider looking into ControlTier (http://open.controltier.com/) which has a open-source cross-platform solution.
Puppet也不(轻松)支持Win32世界。如果您正在寻找部署机制而不仅仅是“开发设置”工具,您可以考虑查看具有开源跨平台解决方案的ControlTier(http://open.controltier.com/)。
Beyond that you're looking at "enterprise" software such as BladeLogic or OpsWare and typically an outrageous pricetag for the functionality offered (my opinion, obviously).
除此之外,您正在寻找诸如BladeLogic或OpsWare之类的“企业”软件,并且通常会为所提供的功能提供令人愤慨的价格(我的观点,显然)。
A lot of folks have been aggressively using a combination of Puppet and Capistrano (even non-rails developers) for deployment automation tools to pretty good effect. Downside, again, is that it's expecting a somewhat homogeneous environment.
很多人一直在积极地使用Puppet和Capistrano(甚至是非rails开发人员)的组合来实现部署自动化工具,效果非常好。另一方面,缺点是,它期待一个有点同质的环境。
#1
3
Setuptools may be capable of more of what you're looking for than you realize -- if you need a custom version of lxml to work correctly on MacOS X, for instance, you can put a URL to an appropriate egg inside your setup.py and have setuptools download and install that inside your developers' environments as necessary; it also can be told to download and install a specific version of a dependency from revision control.
Setuptools可能比你想象的更能满足你想要的东西 - 如果你需要自定义版本的lxml才能在MacOS X上正常工作,例如,你可以在setup.py中的一个适当的蛋中放置一个URL并在必要时让setuptools下载并安装在开发人员的环境中;它还可以被告知从版本控制下载并安装特定版本的依赖项。
That said, I'd lean towards using a scriptably generated virtual environment. It's pretty straightforward to build a kickstart file which installs whichever packages you depend on and then boot virtual machines (or production hardware!) against it, with puppet or similar software doing other administration (adding users, setting up services [where's your database come from?], etc). This comes in particularly handy when your production environment includes multiple machines -- just script the generation of multiple VMs within their handy little sandboxed subnet (I use libvirt+kvm for this; while kvm isn't available on all the platforms you have developers working on, qemu certainly is, or you can do as I do and have a small number of beefy VM hosts shared by multiple developers).
也就是说,我倾向于使用可编写脚本的虚拟环境。构建kickstart文件非常简单,该文件安装您依赖的任何软件包,然后针对它启动虚拟机(或生产硬件!),使用木偶或类似软件进行其他管理(添加用户,设置服务[数据库来自哪里] ?]等)。当您的生产环境包含多台计算机时,这会特别方便 - 只需在其便捷的小沙盒子网中编写多个VM的代码(我使用libvirt + kvm;而在开发人员工作的所有平台上都没有kvm可用) on,qemu肯定是,或者你可以像我一样做,并且有少数强大的VM主机由多个开发人员共享)。
This gets you out of the headaches of supporting N platforms -- you only have a single virtual platform to support -- and means that your deployment process, as defined by the kickstart file and puppet code used for setup, is source-controlled and run through your QA and review processes just like everything else.
这让您摆脱了支持N平台的麻烦 - 您只需要一个虚拟平台来支持 - 并且意味着您的部署过程(由kickstart文件和用于设置的木偶代码定义)是源控制和运行的通过您的质量检查和审核流程就像其他一切一样。
#2
3
I always create a develop.py file at the top level of the project, and have also a packages directory with all of the .tar.gz files from PyPI that I want to install, and also included an unpacked copy of virtualenv that is ready to run right from that file. All of this goes into version control. Every developer can simply check out the trunk, run develop.py, and a few moments later will have a virtual environment ready to use that includes all of our dependencies at exactly the versions the other developers are using. And it works even if PyPI is down, which is very helpful at this point in that service's history.
我总是在项目的顶层创建一个develop.py文件,并且还有一个包含我想要安装的PyPI的所有.tar.gz文件的包目录,还包括一个已经准备好的解压缩的virtualenv副本从该文件直接运行。所有这些都进入了版本控制。每个开发人员都可以简单地检查主干,运行develop.py,稍后将有一个可以使用的虚拟环境,其中包括我们所有的依赖项,与其他开发人员正在使用的版本完全相同。即使PyPI已关闭,它仍然有效,这在该服务的历史中非常有用。
#3
0
Basically, you're looking for a cross-platform software/package installer (on the lines of apt-get/yum/etc.) I'm not sure something like that exists?
基本上,你正在寻找一个跨平台的软件/软件包安装程序(在apt-get / yum / etc等行)。我不确定这样的东西存在吗?
An alternative might be specifying the list of packages that need to be installed via the OS-specific package management system such as Fink or DarwinPorts for Mac OS X and having a script that sets up the build environment for the in-house code?
另一种方法是指定需要通过特定于操作系统的软件包管理系统安装的软件包列表,例如适用于Mac OS X的Fink或DarwinPorts,以及具有为内部代码设置构建环境的脚本?
#4
0
I have continued to research this issue since I posted the question. It looks like there are some attempts to address some of the needs I outlined, e.g. Minitage and Puppet which take different approaches but both may accomplish what I want -- although Minitage does not explicitly state that it supports Windows. Lacking any better options I will try to make either one of these or just extensive customized use of zc.buildout work for our needs, but I still feel like there must be better options out there.
自从我发布问题以来,我一直在研究这个问题。看起来有一些尝试来解决我概述的一些需求,例如: Minitage和Puppet采用不同的方法但两者都可以实现我想要的 - 虽然Minitage没有明确声明它支持Windows。缺乏任何更好的选择我将尝试制作其中一个或只是广泛定制使用zc.buildout工作满足我们的需求,但我仍然觉得必须有更好的选择。
#5
0
You might consider creating virtual machine appliances with whatever production OS you are running, and all of the software dependencies pre-built. Code can be edited either remotely, or with a shared folder. It worked pretty well for me in a past life that had a fairly complicated development environment.
您可以考虑使用正在运行的任何生产操作系统创建虚拟机设备,并预先构建所有软件依赖项。可以远程编辑代码,也可以使用共享文件夹编辑代码。在过去的生活中,它对我来说非常有效,它具有相当复杂的开发环境。
#6
0
Puppet doesn't (easily) support the Win32 world either. If you're looking for a deployment mechanism and not just a "dev setup" tool, you might consider looking into ControlTier (http://open.controltier.com/) which has a open-source cross-platform solution.
Puppet也不(轻松)支持Win32世界。如果您正在寻找部署机制而不仅仅是“开发设置”工具,您可以考虑查看具有开源跨平台解决方案的ControlTier(http://open.controltier.com/)。
Beyond that you're looking at "enterprise" software such as BladeLogic or OpsWare and typically an outrageous pricetag for the functionality offered (my opinion, obviously).
除此之外,您正在寻找诸如BladeLogic或OpsWare之类的“企业”软件,并且通常会为所提供的功能提供令人愤慨的价格(我的观点,显然)。
A lot of folks have been aggressively using a combination of Puppet and Capistrano (even non-rails developers) for deployment automation tools to pretty good effect. Downside, again, is that it's expecting a somewhat homogeneous environment.
很多人一直在积极地使用Puppet和Capistrano(甚至是非rails开发人员)的组合来实现部署自动化工具,效果非常好。另一方面,缺点是,它期待一个有点同质的环境。