扩展setuptools扩展到在setup.py中使用CMake ?

I'm writing a Python extension that links a C++ library and I'm using cmake to help with the build process. This means that right now, the only way I know how to bundle it, I have to first compile them with cmake before I can run setup.py bdist_wheel. There must be a better way.

我正在编写一个连接c++库的Python扩展，并使用cmake来帮助构建过程。这意味着，现在，我知道如何捆绑它的唯一方法是，在运行setup之前必须先用cmake编译它们。py bdist_wheel。一定有更好的办法。

I was wondering if it's possible (or anybody has tried) to invoke CMake as part of the setup.py ext_modules build process? I'm guessing there is a way to create a subclass of something but I'm not sure where to look.

我想知道是否有可能(或任何人尝试过)调用CMake作为设置的一部分。py ext_modules构建过程?我猜有一种方法可以创建一个类的子类，但是我不知道应该在哪里查找。

I'm using CMake because it gives me so much more control for building c and c++ libraries extensions with complex build steps exactly as I want it. Plus, I can easily build Python extensions directly with cmake with the PYTHON_ADD_MODULE() command in the findPythonLibs.cmake. I just wish this was all one step.

我使用CMake是因为它给了我更多的控制来构建c和c++库扩展，这些扩展具有复杂的构建步骤。另外，我可以使用findpython_add_module()命令在findpythonlib .cmake中使用cmake直接构建Python扩展。我只希望这是一步。

1 个解决方案

#1

What you basically need to do is to override the build_ext command class in your setup.py and register it in the command classes. In your custom impl of build_ext, configure and call cmake to configure and then build the extension modules. Unfortunately, the official docs are rather laconic about how to implement custom distutils commands (see Extending Distutils); I find it much more helpful to study the commands code directly. For example, here is the source code for the build_ext command.

您基本上需要做的是重写安装中的build_ext命令类。并将其注册到命令类中。在build_ext的自定义impl中，配置并调用cmake来配置并构建扩展模块。不幸的是，关于如何实现定制的distutils命令(请参阅扩展distutils)，官方文档非常简洁;我发现直接学习命令代码更有帮助。例如，这里是build_ext命令的源代码。

Example project

I have prepared a simple project consisting out of a single C extension foo and a python module spam.eggs:

我准备了一个简单的项目，由一个C扩展名foo和一个python模块spam.eggs组成:

so-42585210/
├── spam
│   ├── __init__.py  # empty
│   ├── eggs.py
│   ├── foo.c
│   └── foo.h
├── CMakeLists.txt
└── setup.py

Files for testing the setup

These are just some simple stubs I wrote to test the setup script.

这些只是我为测试安装脚本而编写的一些简单存根。

spam/eggs.py (only for testing the library calls):

垃圾邮件/鸡蛋。py(仅用于测试库调用):

from ctypes import cdll
import pathlib


def wrap_bar():
    foo = cdll.LoadLibrary(str(pathlib.Path(__file__).with_name('libfoo.dylib')))
    return foo.bar()

spam/foo.c:

垃圾邮件/ foo.c:

#include "foo.h"

int bar() {
    return 42;
}

spam/foo.h:

垃圾邮件/ foo。:

#ifndef __FOO_H__
#define __FOO_H__

int bar();

#endif

CMakeLists.txt:

CMakeLists.txt:

cmake_minimum_required(VERSION 3.10.1)
project(spam)
set(src "spam")
set(foo_src "spam/foo.c")
add_library(foo SHARED ${foo_src})

Setup script

This is where the magic happens. Of course, there is a lot of room for improvements - you could pass additional options to CMakeExtension class if you need to (for more info on the extensions, see Building C and C++ Extensions), make the CMake options configurable via setup.cfg by overriding methods initialize_options and finalize_options etc.

这就是奇迹发生的地方。当然，还有很多改进的空间——如果需要的话，您可以将其他选项传递给CMakeExtension类(关于扩展的更多信息，请参见构建C和c++扩展)，通过安装使CMake选项可配置。通过重写方法initialize_options和finalize_options等实现cfg。

import os
import pathlib

from setuptools import setup, Extension
from setuptools.command.build_ext import build_ext as build_ext_orig


class CMakeExtension(Extension):

    def __init__(self, name):
        # don't invoke the original build_ext for this special extension
        super().__init__(name, sources=[])


class build_ext(build_ext_orig):

    def run(self):
        for ext in self.extensions:
            self.build_cmake(ext)
        super().run()

    def build_cmake(self, ext):
        cwd = pathlib.Path().absolute()

        # these dirs will be created in build_py, so if you don't have
        # any python sources to bundle, the dirs will be missing
        build_temp = pathlib.Path(self.build_temp)
        build_temp.mkdir(parents=True, exist_ok=True)
        extdir = pathlib.Path(self.get_ext_fullpath(ext.name))
        extdir.mkdir(parents=True, exist_ok=True)

        # example of cmake args
        config = 'Debug' if self.debug else 'Release'
        cmake_args = [
            '-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=' + str(extdir.parent.absolute()),
            '-DCMAKE_BUILD_TYPE=' + config
        ]

        # example of build args
        build_args = [
            '--config', config,
            '--', '-j4'
        ]

        os.chdir(str(build_temp))
        self.spawn(['cmake', str(cwd)] + cmake_args)
        if not self.dry_run:
            self.spawn(['cmake', '--build', '.'] + build_args)
        os.chdir(str(cwd))


setup(
    name='spam',
    version='0.1',
    packages=['spam'],
    ext_modules=[CMakeExtension('spam/foo')],
    cmdclass={
        'build_ext': build_ext,
    }
)

Testing

Build the project's wheel, install it. Test the library is installed:

构建项目的*，安装它。测试库是否安装:

$ pip show -f spam
Name: spam
Version: 0.1
Summary: UNKNOWN
Home-page: UNKNOWN
Author: UNKNOWN
Author-email: UNKNOWN
License: UNKNOWN
Location: /Users/hoefling/.virtualenvs/*/lib/python3.6/site-packages
Requires: 
Files:
  spam-0.1.dist-info/DESCRIPTION.rst
  spam-0.1.dist-info/INSTALLER
  spam-0.1.dist-info/METADATA
  spam-0.1.dist-info/RECORD
  spam-0.1.dist-info/WHEEL
  spam-0.1.dist-info/metadata.json
  spam-0.1.dist-info/top_level.txt
  spam/__init__.py
  spam/__pycache__/__init__.cpython-36.pyc
  spam/__pycache__/eggs.cpython-36.pyc
  spam/eggs.py
  spam/libfoo.dylib

Run the wrapper function from spam.eggs module:

从spam中运行包装器函数。鸡蛋模块:

$ python -c "from spam import eggs; print(eggs.wrap_bar())"
42