Distance Norm between matrices

Trace Norm

Let \( A, B \in \mathbb{R}^{m \times n} \) be two matrices. The trace norm (or nuclear norm) of their difference can be computed as follows:

1. Compute the difference matrix \( D = A - B \).

2. Calculate the singular value decomposition (SVD) of \( D \), i.e., \( D = U \Sigma V^{T} \), where \( U \) and \( V \) are orthogonal matrices and \( \Sigma \) is a diagonal matrix with the singular values of \( D \) on its diagonal.

3. The trace norm of \( D \), denoted as \( ||D||_{*} \), is the sum of the singular values of \( D \), which are the diagonal entries of \( \Sigma \). Formally, 

\[ ||D||_{*} = ||A - B||_{*} = \sum_{i=1}^{\min(m, n)} \sigma_i(D) \]

where \( \sigma_i(D) \) is the \( i^{th} \) singular value of \( D \).