We propose a low-rank, efficient orthogonal layer and a margin-balancing loss function to construct effective and certifiably robust neural networks.
