Working with packages using setuptools_dso
==========================================
.. currentmodule:: setuptools_dso
Packages using `setuptools_dso` can for the most part be treated like any other python package.
Distribution through `pypi.org `_ as source and/or binary (wheels) is possible.
Like other packages containing compiled code, use of egg binaries is not supported.
Use of PIP is encouraged.
The :py:mod:`setuptools_dso` package needs to be installed when manually invoking ``setup.py`` scripts in dependent packages.
eg. to generate source tars with ``setup.py sdist``.
Developers wishing to work with an in-place build will need to use the added ``build_dso`` command,
which functions like the ``build_ext`` command. ::
python setup.py build_dso -i
python setup.py build_ext -i
.. _num_jobs:
NUM_JOBS
--------
Use the ``$NUM_JOBS`` environment variable to override the default concurrency
when compiling object files for DSOs.
eg. ``export NUM_JOBS=1`` for a sequential build.
Applying to your package
========================
The `example/ `_ demonstrates building a non-python library,
and linking it with a python extension module.
pyproject.toml
^^^^^^^^^^^^^^
To properly support ``pip install ...``, it is recommended to include a
`pyproject.toml `_
file containing at least:
.. literalinclude:: ../example/pyproject.toml
This ensures that ``setuptools_dso`` is available to be imported by ``setup.py``.
MANIFEST.in
^^^^^^^^^^^
Add a ``MANIFEST.in`` to ensure that ``setup.py sdist`` includes everything necessary
for a successful source build.
.. literalinclude:: ../example/MANIFEST.in
Using with manylinux
^^^^^^^^^^^^^^^^^^^^
Using of **auditwheel** with wheel builds from setuptools_dso can cause problems
as auditwheel has no concept of non-python libraries in python packages.
If **SETUPTOOLS_DSO_PLAT_NAME** contains a manylinux name, setuptools_dso will
inject this platform name when building wheels.
Currently PIP version <=21 will pass this environment variable through to PEP 517 builds.
eg. ::
SETUPTOOLS_DSO_PLAT_NAME=manylinux1_x86_64 pip wheel -w dist .
Building a DSO
--------------
.. autofunction:: setup
.. autoclass:: DSO
The `example source `_
files while make up the non-python ``demo`` library are: ``mylib.h``, ``foo.c``, ``bar.cpp``.
This library will be expressed as a `setuptools_dso.DSO` object.
The first argument is a directory path and library base name encoded like a python module.
eg. the result of ``dsodemo.lib.demo`` will be eg. ``dsodemo/lib/libdemo.so`` or ``dsodemo\lib\demo.dll``
installed in the python module tree along-side any other python code or C extensions.
Note that there need not be a ``dsodemo/lib/__init__.py`` as ``dsodemo.lib`` need not be a python package.
However, if this file is present, then the generated `dsodemo/lib/demo_dsoinfo.py` will be accessible. ::
from setuptools_dso import DSO, Extension, setup
dso = DSO('dsodemo.lib.demo', ['src/foo.c', 'src/bar.cpp'], ...)
setup(
...
x_dsos = [dso],
zip_safe = False, # setuptools_dso is not compatible with eggs!
)
The ``setup(x_dsos=...`` argument may be either ``None`` (default),
a list of :py:class:`DSO` instances, or a callable returning such a list.
A callable will be invoked during the `build_dso` phase with one argument of :py:class:`distutils.core.Command`.
(see :ref:`probing`)
The :py:class:`DSO` constructor understands the same keyword arguments as `setuptools.Extension`
and `distutils.core.Extension `_,
with the addition of ``dsos=[...]``, ``soversion='...'``, and ``lang_compile_args={'...':'...'}``.
The ``dsos=`` argument is a list of other :py:class:`DSO` names (eg. ``'dsodemo.lib.demo'``) to allow
one :py:class:`DSO` to be linked against others.
eg. ``dsos=['some.lib.foo']`` will result in something like ``gcc ... -L.../some/lib -lfoo``.
Building an Extension
---------------------
.. autoclass:: Extension
:py:class:`setuptools_dso.Extension` is a wrapper around ``setuptools.Extension`` which adds the ``dsos=[...]`` keyword argument.
This allows a C extension module to be linked against a ``DSO`` by name.
The named ``DSO`` may be built by the same ``setup.py``, or may already be present in ``$PYTHONPATH``. ::
from setuptools_dso import DSO, Extension, setup
ext = Extension('dsodemo.ext.dtest', ['src/extension.cpp'],
dsos=['dsodemo.lib.demo'],
)
setup(
...
ext_modules = [ext],
zip_safe = False, # setuptools_dso is not compatible with eggs!
)
Cython
------
.. autofunction:: cythonize
Version 1.3 added a :py:func:`setuptools_dso.cythonize()` wrapper to correctly handle ``Extension(dso=...)``.
.. _dsoinfo:
Runtime
=======
Beginning with setuptools-dso 2.0 a file ``*_dsoinfo.py`` will be generated alongside each DSO.
eg. dso ``"mypkg.lib.thelib"`` will create ``mypkg/lib/thelib_dsoinfo.py``.
If ``mypkg.lib`` is a valid python packages (contains ``__init__.py``)
then :py:func:`setuptools_dso.runtime.import_dsoinfo` may be used to retrieve
build time information about the DSO including platform specific filename
(eg. ``thelib.dll`` vs. ``libthelib.so``).
Beginning with 2.0 the necessary additions to ``$PATH`` or calls to ``os.add_dll_directory()``
can be made via :py:func:`dylink_prepare_dso`.
.. autofunction:: dylink_prepare_dso
Prior to 2.0, or if the generated module is not used,
some additional runtime preparation is needed in order to find the ``"dsodemo.lib.demo"`` library
when the ``dsodemo.ext.dtest`` Extension is imported on Windows.
This could be placed in eg. ``example/src/dsodemo/__init__.py``
to ensure it always runs before the extension library is loaded. ::
# with setuptools_dso >= 2.0a1
#from setuptools_dso import dylink_prepare_dso
#dylink_prepare_dso('..lib.demo')
# or as previously:
import sys, os
def fixpath():
# If this file is
# .../ext/__init__.py
# DSOs are under
# .../lib/
libdir = os.path.join(os.path.dirname(os.path.dirname(__file__)))
if hasattr(os, 'add_dll_directory'): # py >= 3.8
os.add_dll_directory(libdir)
else:
path = os.environ.get('PATH', '').split(os.pathsep)
path.append(libdir)
os.environ['PATH'] = os.pathsep.join(path)
if sys.platform == "win32":
fixpath()
Use with ctypes
---------------
.. autofunction:: find_dso
Info
^^^^
.. currentmodule:: setuptools_dso.runtime
.. autofunction:: import_dsoinfo