使用Cython加速NumPy中的数组迭代

NumPy以速度快著称，但它还能更快吗?下面是如何使用Cython加速NumPy中的数组迭代灯塔道教灵符网请符。NumPy为Python用户提供了一个非常快速的库，用于处理矩阵中的数据。例如，如果你想要生成一个由随机数填充的矩阵，你可以用传统Python所花费的一小部分时间来完成。

不过，有时即使NumPy本身也不够快灯塔道教灵符网请符。如果你想在NumPy的API中无法使用的NumPy矩阵上执行转换，典型的方法是在Python中迭代矩阵…并失去了使用NumPy的所有性能优势。

幸运的是，有一种更好的方法可以直接处理NumPy数据:Cython灯塔道教灵符网请符。通过编写带有类型注释的Python代码并将其编译为C语言，您可以遍历NumPy数组，并以C语言的速度直接处理它们的数据。

本文介绍了如何在NumPy中使用Cython的一些关键概念灯塔道教灵符网请符。如果您还不熟悉Cython，请阅读Cython的基础知识，并查看关于编写Cython代码的简单教程。

一、仅为NumPy编写Cython的核心计算代码

与NumPy一起使用Cython的最常见场景是，您希望获得一个NumPy数组，遍历它，并对NumPy中无法轻易完成的每个元素执行计算灯塔道教灵符网请符。

Cython的工作原理是允许您使用带有类型注释的Python版本编写模块，然后将这些模块编译为C语言，并像其他模块一样导入Python脚本灯塔道教灵符网请符。换句话说，您编写了一些类似于您想要完成的任务的Python版本，然后通过添加注释(允许将其翻译成C)来加快速度。

[参加11月8日的虚拟峰会- CIO的云未来峰会:掌握复杂性和数字创新-今天注册!]］

为此，您应该只在程序中执行实际计算的部分使用Cython灯塔道教灵符网请符。其他所有与性能无关的内容——也就是说，实际上不是遍历数据的循环的内容——都应该用常规Python编写。

为什么要这么做?每次更改Cython模块时都必须重新编译，这减慢了开发过程灯塔道教灵符网请符。您不希望每次进行与您试图优化的程序部分无关的更改时都必须重新编译Cython模块。

二、在Cython中迭代NumPy数组灯塔道教灵符网请符，而不是Python

在Cython中高效使用NumPy的一般方法可以概括为三个步骤:

1、在Cython中编写接受NumPy数组作为正确类型对象的函数灯塔道教灵符网请符。在Python代码中调用Cython函数时，将整个NumPy数组对象作为该函数调用的参数发送。

2、在Cython中对对象执行所有迭代灯塔道教灵符网请符。

3、从你的Cython模块返回一个NumPy数组到你的Python代码灯塔道教灵符网请符。

所以灯塔道教灵符网请符，不要做这样的事情:

for index in len(numpy_array):

numpy_array[index] = cython_function(numpy_array[index])

相反灯塔道教灵符网请符，你可以这样做:

returned_numpy_array = cython_function(numpy_array)

# in cython:

cdef cython_function(numpy_array):

for item in numpy_array:

return numpy_array

我省略了这些示例中的类型信息和其他细节，但是区别应该很明显灯塔道教灵符网请符。NumPy数组的实际迭代应该完全用Cython完成，而不是对数组中的每个元素重复调用Cython。

三、将正确类型的NumPy数组传递给Cython函数

任何接受NumPy数组作为参数的函数都应该正确地键入，以便Cython知道如何将参数解释为NumPy数组(快)而不是通用的Python对象(慢)灯塔道教灵符网请符。

下面是一个接受二维NumPy数组的Cython函数声明示例:

def compute(int[:, ::1] array_1):

在Cython的“纯Python”语法中灯塔道教灵符网请符，您可以使用以下注释:

def compute(array_1: cython.int[:, ::1]):

int[]注释表示一个整数数组，可能是一个NumPy数组灯塔道教灵符网请符。但是为了尽可能精确，我们需要指出数组中的维数。对于二维，我们使用int[:，:];对于三个，我们使用int[:，:，:]。

我们还应该为数组指明内存布局灯塔道教灵符网请符。在NumPy和Cython中，默认情况下，数组以与C.::1兼容的连续方式排列，因此我们使用int[:，::1]作为签名。(有关其他内存布局选项的详细信息，请参阅Cython的文档。)

这些声明不仅告诉Cython这些是NumPy数组，还告诉Cython如何以最有效的方式读取它们灯塔道教灵符网请符。华东CIO大会、华东CIO联盟、CDLC中国数字化灯塔大会、CXO数字化研学之旅、数字化江湖-讲武堂，数字化江湖-大侠传、数字化江湖-论剑、CXO系列管理论坛（陆家嘴CXO管理论坛、宁波东钱湖CXO管理论坛等）、数字化转型网，走进灯塔工厂系列、ECIO大会等

四、使用Cython内存视图快速访问NumPy数组

Cython有一个名为类型化memoryviews的特性，可以让您直接读/写访问许多类型的对象，这些对象的工作方式类似于数组灯塔道教灵符网请符。这包括numpy数组(您猜对了)。

要创建一个memoryview灯塔道教灵符网请符，你可以使用类似于上面数组声明的语法:

# conventional Cythondef compute(int[:, ::1] array_1):

cdef int [:,:] view2d = array_1

# pure-Python mode def compute(array_1: cython.int[:, ::1]):

view2d: int[:,:] = array_1

注意，您不需要在声明中指定内存布局，因为这是自动检测到的灯塔道教灵符网请符。

从代码中的这一点开始，您将使用与array_1对象(例如view2d)相同的访问语法读取和写入view2d灯塔道教灵符网请符。任何读取和写入都是直接对组成数组的底层内存区域进行的(同样:快)，而不是通过使用对象访问器接口(同样:慢)。

五、索引灯塔道教灵符网请符，不要迭代，通过NumPy数组

Python用户现在知道，在对象中遍历元素的首选比喻是object中的item:灯塔道教灵符网请符。您也可以在Cython中使用这个比喻，但是在使用NumPy数组或memoryview时，它不能产生最好的速度。为此，您需要使用c风格的索引。

下面是如何为NumPy数组使用索引的示例:

# conventional Cython:

cimport cython@cython.boundscheck(False)@cython.wraparound(False)def compute(int[:, ::1] array_1):

# get the maximum dimensions of the array

cdef Py_ssize_t x_max = array_1.shape[0]

cdef Py_ssize_t y_max = array_1.shape[1]

#create a memoryview

cdef int[:, :] view2d = array_1

# access the memoryview by way of our constrained indexes

for x in range(x_max):

for y in range(y_max):

view2d[x,y] = something()

# pure-Python mode: import cython@cython.boundscheck(False)@cython.wraparound(False)def compute(array_1: cython.int[:, ::1]):

# get the maximum dimensions of the array

x_max: cython.size_t = array_1.shape[0]

y_max: cython.size_t = array_1.shape[1]

#create a memoryview

view2d: int[:,:] = array_1

# access the memoryview by way of our constrained indexes

for x in range(x_max):

for y in range(y_max):

view2d[x,y] = something()

在这个例子中，我们使用NumPy数组的.shape属性来获取它的维度灯塔道教灵符网请符。然后使用range()以这些维度作为约束遍历内存视图。我们不允许任意访问数组的某些部分，例如，通过用户提交的变量，因此不存在越界的风险。

您还会注意到我们的函数上有@cython.boundscheck(False)和@cython. wrapper (False)装饰符灯塔道教灵符网请符。默认情况下，Cython启用了防止数组访问器出错的选项，这样您就不会错误地读取数组边界之外的内容。但是，检查会降低对数组的访问速度，因为每个操作都必须进行边界检查。使用装饰器会使这些警卫变得不必要，从而禁用它们。我们已经确定了数组的边界，我们不超过它们。

原文：

NumPy is known for being fast, but could it go even faster? Here’s how to use Cython to accelerate array iterations in NumPy.

NumPy gives Python users a wickedly fast library for working with data in matrixes. If you want, for instance, to generate a matrix populated with random numbers, you can do that in a fraction of the time it would take in conventional Python.

Still, there are times when even NumPy by itself isn’t fast enough. If you want to perform transformations on NumPy matrixes that aren’t available in NumPy’s API, a typical approach is to just iterate over the matrix in Python … and lose all the performance benefits of using NumPy in the first place.

Fortunately, there’s a better way to work directly with NumPy data: Cython. By writing type-annotated Python code and compiling it to C, you can iterate over NumPy arrays and work directly with their data at the speed of C.

This article walks through some key concepts for how to use Cython with NumPy. If you’re not already familiar with Cython, read up on the basics of Cython and check out this simple tutorial for writing Cython code.

Write only core computation code in Cython for NumPy

The most common scenario for using Cython with NumPy is one where you want to take a NumPy array, iterate over it, and perform computations on each element that can’t be done readily in NumPy.

Cython works by letting you write modules in a type-annotated version of Python, which are then compiled to C and imported into your Python script like any other module. In other words, you write something akin to a Python version of what you want to accomplish, then speed it up by adding annotations that allow it to be translated into C.

[ Attend Virtual Summit on November 8 – CIO’s Future of Cloud Summit: Mastering Complexity Digital Innovation – Register Today! ]

To that end, you should only use Cython for the part of your program that does the actual computation. Everything else that’s not performance-sensitive—that is, everything that’s not actually the loop that iterates over your data—should be written in regular Python.

Why do this? Cython modules have to be recompiled each time they’re changed, which slows down the development process. You don’t want to have to recompile your Cython modules every time you make changes that aren’t actually about the part of your program you’re trying to optimize.

Iterate through NumPy arrays in Cython, not Python

The general method for working efficiently with NumPy in Cython can be summed up in three steps:

Write functions in Cython that accept NumPy arrays as properly typed objects. When you call the Cython function in your Python code, send the entire NumPy array object as an argument for that function call.

Perform all the iteration over the object in Cython.

Return a NumPy array from your Cython module to your Python code.

So, don’t do something like this:

for index in len(numpy_array):

numpy_array[index] = cython_function(numpy_array[index])

Rather, do something like this:

returned_numpy_array = cython_function(numpy_array)

# in cython:

cdef cython_function(numpy_array):

for item in numpy_array:

return numpy_array

I omitted type information and other details from these samples, but the difference should be clear. The actual iteration over the NumPy array should be done entirely in Cython, not through repeated calls to Cython for each element in the array.

Pass properly typed NumPy arrays to Cython functions

Any functions that accept a NumPy array as an argument should be properly typed, so that Cython knows how to interpret the argument as a NumPy array (fast) rather than a generic Python object (slow).

Here’s an example of a Cython function declaration that takes in a two-dimensional NumPy array:

def compute(int[:, ::1] array_1):

In Cython’s “pure Python” syntax, you’d use this annotation:

def compute(array_1: cython.int[:, ::1]):

The int[] annotation indicates an array of integers, potentially a NumPy array. But to be as precise as possible, we need to indicate the number of dimensions in the array. For two dimensions, we’d use int[:,:]; for three, we’d use int[:,:,:].

We also should indicate the memory layout for the array. By default in NumPy and Cython, arrays are laid out in a contiguous fashion compatible with C. ::1 is our last element in the above sample, so we use int[:,::1] as our signature. (For details on other memory layout options, see Cython’s documentation.)

These declarations inform Cython not just that these are NumPy arrays, but how to read from them in the most efficient way possible.

Use Cython memoryviews for fast access to NumPy arrays

Cython has a feature named typed memoryviews that gives you direct read/write access to many types of objects that work like arrays. That includes—you guessed it—NumPy arrays.

To create a memoryview, you use a similar syntax to the array declarations shown above:

# conventional Cythondef compute(int[:, ::1] array_1):

cdef int [:,:] view2d = array_1

# pure-Python mode def compute(array_1: cython.int[:, ::1]):

view2d: int[:,:] = array_1

Note that you don’t need to specify the memory layout in the declaration, as that’s detected automatically.

From this point on in your code, you’d read from and write to view2d with the same accessing syntax as you would the array_1 object (e.g., view2d). Any reads and writes are done directly to the underlying region of memory that makes up the array (again: fast), rather than by using the object-accessor interfaces (again: slow).

Index, don’t iterate, through NumPy arrays

Python users know by now the preferred metaphor for stepping through the elements of an object is for item in object:. You can use this metaphor in Cython, as well, but it doesn’t yield the best possible speed when working with a NumPy array or memoryview. For that, you’ll want to use C-style indexing.

Here’s an example of how to use indexing for NumPy arrays:

# conventional Cython:

cimport cython@cython.boundscheck(False)@cython.wraparound(False)def compute(int[:, ::1] array_1):

# get the maximum dimensions of the array

cdef Py_ssize_t x_max = array_1.shape[0]

cdef Py_ssize_t y_max = array_1.shape[1]

#create a memoryview

cdef int[:, :] view2d = array_1

# access the memoryview by way of our constrained indexes

for x in range(x_max):

for y in range(y_max):

view2d[x,y] = something()

# pure-Python mode: import cython@cython.boundscheck(False)@cython.wraparound(False)def compute(array_1: cython.int[:, ::1]):

# get the maximum dimensions of the array

x_max: cython.size_t = array_1.shape[0]

y_max: cython.size_t = array_1.shape[1]

#create a memoryview

view2d: int[:,:] = array_1

# access the memoryview by way of our constrained indexes

for x in range(x_max):

for y in range(y_max):

view2d[x,y] = something()

In this example, we use the NumPy array’s .shape attribute to obtain its dimensions. We then use range() to iterate through the memoryview with those dimensions as a constraint. We don’t allow arbitrary access to some part of the array, for example, by way of a user-submitted variable, so there’s no risk of going out of bounds.

You’ll also notice we have @cython.boundscheck(False) and @cython.wraparound(False) decorators on our functions. By default, Cython enables options that guard against making mistakes with array accessors, so you don’t end up reading outside the bounds of an array by mistake. The checks slow down access to the array, however, because every operation has to be bounds-checked. Using the decorators disables those guards by making them unnecessary. We’ve already determined what the bounds of the array are, and we don’t go past them.

CXO联盟（CXO union）是一家聚焦于CIO，CDO，cto，ciso，cfo，coo，chro，cpo，ceo等人群的平台组织，其中在CIO会议领域的领头羊，目前举办了大量的CIO大会、CIO论坛、CIO活动、CIO会议、CIO峰会、CIO会展灯塔道教灵符网请符。如华东CIO会议、华南cio会议、华北cio会议、中国cio会议、西部CIO会议。在这里，你可以参加大量的IT大会、IT行业会议、IT行业论坛、IT行业会展、数字化论坛、数字化转型论坛，在这里你可以认识很多的首席信息官、首席数字官、首席财务官、首席技术官、首席人力资源官、首席运营官、首席执行官、IT总监、财务总监、信息总监、运营总监、采购总监、供应链总监。

数字化转型网（资讯媒体，是企业数字化转型的必读参考，在这里你可以学习大量的知识，如财务数字化转型、供应链数字化转型、运营数字化转型、生产数字化转型、人力资源数字化转型、市场营销数字化转型灯塔道教灵符网请符。通过关注我们的公众号，你就知道如何实现企业数字化转型？数字化转型如何做？

【联盟会员】耐普矿机CTO、聚杰微纤CTO、英杰电气CTO、东岳硅材CTO、贝仕达克CTO、建科机械CTO、北鼎股份CTO、阿尔特CTO、测绘股份CTO、上能电气CTO、锐新科技CTO、金丹科技CTO、金现代CTO、派瑞股份CTO、新产业CTO、浩洋股份CTO、龙磁科技CTO、佰奥智能CTO、浙矿股份CTO、浙江力诺CTO、博汇股份CTO、酷特智能CTO、康华生物CTO、帝科股份CTO、胜蓝股份CTO、山水比德CTO、捷安高科CTO、首都在线CTO、中船汉光CTO、美瑞新材CTO、锦盛新材CTO、新强联CTO、交大思诺CTO、四会富仕CTO、申昊科技CTO、中兰环保CTO、图南股份CTO、科思股份CTO、协创数据CTO、科拓生物CTO、西域CTO、锋尚文化CTO、美畅股份CTO、蓝盾光电CTO、卡倍亿CTO、南大环境CTO、大宏立CTO、安克创新CTO、圣元环保CTO、杰美特CTO、康泰医学CTO、欧陆通CTO、回盛生物CTO、天阳科技CTO、海晨股份CTO、捷强装备CTO、蒙泰高新CTO、金春股份CTO、维康药业CTO、大叶股份CTO、迦南智能CTO、盛德鑫泰CTO、万胜智能CTO、龙利得CTO、狄耐克CTO、海昌新材CTO、华业香料CTO、谱尼测试CTO、稳健医疗CTO、爱克股份CTO、翔丰华CTO、惠云钛业CTO、品渥食品CTO、松原股份CTO、火星人CTO、铜牛信息CTO、爱美客CTO、山科智能CTO、熊猫乳品CTO、上海凯鑫CTO、广联航空CTO、中胤时尚CTO、国安达CTO、科翔股份CTO、宝丽迪CTO、日月明CTO、康平科技CTO、仲景食品CTO、汇创达CTO、瑞丰新材CTO、亿田智能CTO、凯龙高科CTO、兆龙互连CTO、海融科技CTO、朗特智能CTO、特发服务CTO、南山智尚CTO、中伟股份CTO、润阳科技CTO、南凌科技CTO、天秦装备CTO、研奥股份CTO

本文链接：https://fuzhouwang.org/index.php/post/11664.html

转载声明：本站文章中有转载或采集其他网站内容， 如有转载的文章涉及到您的权益及版权,还麻烦及时联系我们，我们将及时删除，谢谢配合。