Structure of the eigenvalues for the Toeplitz matrix
My current project requires that I examine the risk of a large number of portfolios so I need to create some sensible estimates of risk. Which in part, means looking for robust shrinkage estimators of covariance matrices. A requirement is that the resulting matrices are stable to random perturbations. Specifically if I add a little random noise, I should still get nearly the same answer. So I turn to Toeplitz matrices, with some cool results.
To see which matrices and transformations of matrices are stable, it helps to start with some very structured matrices that look like risk covariances, and therefore allow us to examine if all that structure is preserved when we kick them with some random data. We use Toeplitz matrices – like risk model covariance matrices, they are symmetric can be shown to have real eigenvalues and in some special cases, their eigenvectors have closed form solutions.
It is also known that certain Toeplitz matrices can be proven to have eigenvectors that are periodic.
So I start with a very simple diagonal Toeplitz matrix
Nth=2^9;
bx=[2*Nth:-2:1]/(2*Nth);
vcv=toeplitz(bx.^4);
image(vcv)
The eigenvectors have a lot of structure, this is the first image in this post above. The eigenvalues are unremarkable and not the story here. For the record, the eigenvalues are real and well behaved in our examples.
[ev,ed] = eig(vcv);
image(ev);
Even if we preturb the matrix with random data
vcvDr= vcv + 1e-8*rand(Nth,Nth); [evDr,edDr] = eig(vcvDr); vcvDr2= vcv + 1e-3*rand(Nth,Nth); [evDr2,edDr2] = eig(vcvDr2);
Symmetric Toeplitz matrix
The first five eigenvectors are well behaved under perturbation
The first of four plot shows the first 5 eigenvectors of the original matrix, then to the right are the eigenvectors of a disturbed matrix vcvDr where we add a random perturbation on the order of 10^-8. The bottom left is another larger random perturbation of size 10^-3. Finally we show the difference in absolute value for the entries in the first five eigenvectors. We use absolute value as the eigenvectors are unique up to sign differences. No problem with stability here. What is notable is that this is similar to Ledoit-Wolf where the diagonal is also constant – they use a market weighted average variance. The off-diagonals in Ledoit-Wolf are average covariance. Here the off-diagonals are not constant but allow for the linkages between assets to fall off as matrix polynomial – here we just show the 4th power. So the matrix allows for some assets to be tightly coupled and others to be decoupled. In fact the design lends itself to be decomposed into blocks where the blocks could be asset class covariances.
Those of you who have dealt with fixed income risk models know that this tightly coupled group is difficult to work with. One answer is to only use a few say three principal components. All good and fine, what I am addressing allows for that but more to the point, the stability issue can hit the largest eignevalues and vectors. Another reason to consider the stability question is that the use of a toeplitz matrix with noise allows you to explore the structure that is preserved as you add random noise.
When we allow for the Toeplitz design to be relaxed and allow for the main diagonal to vary exponentially – that is allowing for a parametrized variance where some assets are substantially more volatile than others, then we need to ask about stability. First with an exponent of .995:
for ith= 2:Nth
for jth= ith:Nth
vcvD(ith,jth) = xpwt*vcvD(ith-1,jth-1);
vcvD(jth,ith) = vcvD(ith,jth);
end
end
figure; imagesc(vcvD)
The eigenvalues are structured, but now their is an exponential decay together with the periodicity. We also see that the addition of noise leaves the eigenvalues stable.
The symmetric weighted exponential matrix (.995)
Structure of the eigenvectors for the symmetric weighted exponential matrix (.995)
The eigenvalue structure for the first five ev's. (lambda=.995)
When we allow for an exponent of .985, the decay is large and the matrix looks quite different.
The eigenvalues however are still quite similar. So are the perturbed eigenvectors, but the noise is apparent and of the same scale as the vector coefficients. If you are thinking of using shrinkage estimates that vary along the diagonal you should be mindful of this result.
Symmetric weighted exponential matrix (lambda=.985)
Structure of the eigenvalues (lamda=.985)
The random noise affects the largest eigenvectors
The matlab code for this:
Nth=2^9;
bx=[2*Nth:-2:1]/(2*Nth);
for xpwt = [1.0 .995 .985]
vcv=toeplitz(bx.^4); [ev,ed] = eig(vcv);
vcvD = vcv;
for ith= 2:Nth
for jth= ith:Nth
vcvD(ith,jth) = xpwt*vcvD(ith-1,jth-1);
vcvD(jth,ith) = vcvD(ith,jth);
end
end
figure; imagesc(vcvD)
%
[evD,edD] = eig(vcvD);
vcvDr = vcvD + 1e-8*rand(Nth,Nth); [evDr,edDr] = eig(vcvDr);
vcvDr2= vcvD + 1e-3*rand(Nth,Nth); [evDr2,edDr2] = eig(vcvDr2);
[ntoss,ndx]=sort(diag(edD),'descend');
[ntoss,ndxr]=sort(diag(edDr),'descend');
[ntoss,ndxr2]=sort(diag(edDr2),'descend');
figure; imagesc(min(max(evD(:,ndx),-.07),.07))
figure
subplot(221); plot(evD(:,ndx(1:5))); axis([1 Nth -.07 .07])
subplot(222); plot(evDr(:,ndxr(1:5))); axis([1 Nth -.07 .07])
subplot(223); plot(evDr2(:,ndxr2(1:5))); axis([1 Nth -.07 .07])
subplot(224); plot([abs(evD(:,ndx(1:5))) - abs(evDr2(:,ndxr2(1:5)))]); axis([1 Nth -.07 .07])
end
References
A. Bottcher and S.M. Grudsky, Toeplitz Matrices, Asymptotic Linear Algebra, and Functional Analysis, Birkhauser, 2000.
A. Bottcher and B. Silbermann, Introduction to Large Truncated Toeplitz Matrices, Springer, New York, 1999.
W. Cheney, Introduction to Approximation theory, McGraw-Hill, 1966.
T. A. Cover and J. A. Thomas, Elements of Information Theory, Wiley, New York, 1991.
P. J. Davis, Circulant Matrices, Wiley-Interscience, NY, 1979.
D. Fasino and P. Tilli, “Spectral clustering properties of block multilevel Hankel matrices, Linear Algebra and its Applications, Vol. 306, pp. 155–163, 2000.
F.R. Gantmacher, The Theory of Matrices, Chelsea Publishing Co., NY 1960.
R.M. Gray Toeplitz and Circulant Matrices: A review. 206, Now Publishers
R. M. Gray, “On the asymptotic eigenvalue distribution of Toeplitz matrices,” IEEE Transactions on Information Theory, Vol. 18, November 1972, pp. 725–730.
Ulf Grenander and Gabor Szego, Toeplitz forms and their application, American Mathematical Soc., 1984
Lloyd Nick Trefethen and David Bau, Numerical Linear Algebra, SIAM 1997
W. F. Trench, “Asymptotic distribution of the even and odd spectra of real symmetric Toeplitz matrices,” Linear Algebra Appl., Vol. 302-303, pp. 155–162, 1999.
W. F. Trench, “Absolute equal distribution of the spectra of Hermitian matrices,” Lin. Alg. Appl., 366 (2003), 417–431.
