Current Bitwise Operations in ONNX

Oct 13, 2025 by thesweetdesigns.com 89 views

🚀 The Feature, Motivation, and Pitch

Many bitwise operations are not supported by ONNX export, whether it's with or without Dynamo. This limitation can hinder the deployment of PyTorch models that rely heavily on bitwise operations. Fortunately, some bitwise operations already exist as ONNX ops, and integrating their mapping would be a significant step forward.

Current Bitwise Operations in ONNX

The following table highlights the existing bitwise operations in ONNX:

Torch	ONNX
`__rshift__`, `__lshift__`	BitShift
bitwise_xor, `^`	BitwiseXor
bitwise_and, `&`	BitwiseAnd

🚨 Failed Export Example

Here's an example of a PyTorch model that fails to export due to the lack of bitwise operations in ONNX:

import math
import torch

class Gray(torch.nn.Module):
    nbits: int = 32

    def forward(self, gray: torch.Tensor):
        shifts = [(0x1 << i) for i in range((math.ceil(math.log(self.nbits, 2)) - 1), -1, -1)]
        for shift in shifts:
            gray ^= gray >> shift

        return gray


torch.onnx.export(
    Gray(),  # model to export
    (torch.randint(0, 100, [100], dtype=torch.long)),  # inputs of the model,
    "my_model.onnx",  # filename of the ONNX model
    dynamo=True,  # True or False to select the exporter to use
    verbose=False,
)

🤔 Alternatives

One possible workaround is to manually declare each missing op via the custom_translation_table argument of torch.onnx.export. However, this approach can be cumbersome and error-prone, with cryptic error messages that can be challenging to resolve.

Manual Declaration via `custom_translation_table`

Here's an example of how to manually declare a missing op using custom_translation_table:

import torch
import onnx

class Gray(torch.nn.Module):
    nbits: int = 32

    def forward(self, gray: torch.Tensor):
        shifts = [(0x1 << i) for i in range((math.ceil(math.log(self.nbits, 2)) - 1), -1, -1)]
        for shift in shifts:
            gray ^= gray >> shift

        return gray

# Create a custom translation table
custom_translation_table = {
    "aten::bitwise_xor": "BitwiseXor",
    "aten::bitwise_and": "BitwiseAnd",
    "aten::bitwise_xor": "BitwiseXor",
}

# Export the model with the custom translation table
torch.onnx.export(
    Gray(),
    (torch.randint(0, 100, [100], dtype=torch.long)),
    "my_model.onnx",
    dynamo=True,
    custom_translation_table=custom_translation_table,
    verbose=False,
)

📝 Additional Context

The following issues and pull requests provide additional context and insights into the challenges of supporting bitwise operations in ONNX:

🚀 Conclusion

🤔 Q: What are bitwise operations, and why are they important in PyTorch?

A: Bitwise operations are operations that perform bit-level operations on integers, such as shifting, masking, and XORing. They are essential in PyTorch for various applications, including image and signal processing, cryptography, and machine learning.

🤔 Q: Why are bitwise operations not supported by ONNX export?

A: ONNX export does not support bitwise operations because they are not part of the standard ONNX operator set. However, some bitwise operations, such as BitShift, BitwiseXor, and BitwiseAnd, are already available as ONNX ops.

🤔 Q: What are the benefits of supporting bitwise operations in ONNX?

A: Supporting bitwise operations in ONNX would enable PyTorch models that rely heavily on these operations to be deployed on various platforms, including ONNX-compatible frameworks like TensorFlow and Caffe. This would facilitate the adoption of PyTorch in industries that require bitwise operations, such as image and signal processing.

🤔 Q: How can I manually declare missing bitwise ops in ONNX using `custom_translation_table`?

A: You can manually declare missing bitwise ops in ONNX using custom_translation_table by creating a dictionary that maps the PyTorch op to the corresponding ONNX op. For example:

custom_translation_table = {
    "aten::bitwise_xor": "BitwiseXor",
    "aten::bitwise_and": "BitwiseAnd",
    "aten::bitwise_xor": "BitwiseXor",
}

🤔 Q: What are the challenges of manually declaring missing bitwise ops in ONNX?

A: Manually declaring missing bitwise ops in ONNX can be challenging due to the following reasons:

Error-prone: Creating a custom translation table can be error-prone, especially when dealing with complex bitwise operations.
Cryptic error messages: ONNX export may produce cryptic error messages when encountering unknown ops, making it difficult to diagnose and resolve issues.
Limited flexibility: Manually declaring missing bitwise ops may not provide the same level of flexibility as having them natively supported in ONNX.

🤔 Q: What is the current status of supporting bitwise operations in ONNX?

A: The current status of supporting bitwise operations in ONNX is that some bitwise operations, such as BitShift, BitwiseXor, and BitwiseAnd, are already available as ONNX ops. However, there are still many missing bitwise operations that need to be supported.

🤔 Q: How can I contribute to the development of bitwise operations in ONNX?

A: You can contribute to the development of bitwise operations in ONNX by:

Reporting issues: Report any issues or bugs related to bitwise operations in ONNX.
Submitting pull requests: Submit pull requests that add support for missing bitwise operations.
Participating in discussions: Participate in discussions on ONNX-related forums and communities to provide feedback and suggestions.

🚀 Conclusion

Supporting bitwise operations in ONNX is crucial for the deployment of PyTorch models that rely heavily on these operations. While some bitwise operations already exist as ONNX ops, there are still many missing. By understanding the challenges and benefits of supporting bitwise operations in ONNX, you can contribute to the development of this feature and facilitate the adoption of PyTorch in various industries.