Enhancing Certified Robustness via Block Reflector Orthogonal Layers and Logit Annealing Loss

Abstract

Lipschitz neural networks are well-known for providing certified robustness in deep learning. In this paper, we present a novel, efficient Block Reflector Orthogonal (BRO) layer that enhances the capability of orthogonal layers on constructing more expressive Lipschitz neural architectures. % enables the construction of simple yet effective Lipschitz neural networks. In addition, by theoretically analyzing the nature of Lipschitz neural networks, we introduce a new loss function that employs an annealing mechanism to increase margin for most data points. This enables Lipschitz models to provide better certified robustness. By employing our BRO layer and loss function, we design BRONet — a simple yet effective Lipschitz neural network that achieves state-of-the-art certified robustness. Extensive experiments and empirical analysis on CIFAR-10/100, Tiny-ImageNet, and ImageNet validate that our method outperforms existing baselines.

What is certified robustness?

Certified robustness provide guarantees on the model's performance under adversarial attacks.

Highlights

Block Reflector Orthogonal (BRO) Layer
An iterative, approximation-free orthogonal layer with low-rank trainable parameters.
Logit Annealing (LA) Loss
A loss function designed to help models learn balancing margins across data points.
State-of-the-Art Certified Robustness
Surpasses existing methods across multiple robustness benchmarks.

Pipeline of BRO layer.

Loss gradient of the LA Loss.

BibTeX

                
    @inproceedings{lai2025enhancing,
        title={Enhancing Certified Robustness via Block Reflector Orthogonal Layers and Logit Annealing Loss},
        author={Bo-Han Lai and Pin-Han Huang and Bo-Han Kung and Shang-Tse Chen},
        booktitle={International Conference on Machine Learning (ICML)},
        year={2025},
        note={Spotlight}
    }