Copositive realxation for genera quadratic programming

Copositive Relaxation for General Quadratic Programming A.J. Quist, E. de Klerk, C. Roos, T. Terlaky September 26, 1997 Abstract We consider general, typically nonconvex, Quadratic Programming Problems. The Semi-de nite relaxation proposed by Shor provides bounds on the optimal solution, but it does not always provide suciently strong bounds if linear constraints are also involved. To get rid of the linear side-constraints, another, stronger convex relaxation is derived. This relaxation uses copositive matrices. Special cases are discussed for which both relaxations are equal. At the end of the paper, the complexity and solvability of the relaxations are discussed. 1 Introduction A very important problem in optimization is the General Quadratic Programming Problem (GQP) with quadratic (possibly nonconvex) constraints and objective function. This is a very general NP-hard problem [18], including, for example, integer programming and optimization with general polynomial constraints (see, for example [23]). Since the problem is NP-hard, many relaxations of the problem have been studied. One convex relaxation of the problem is the semi-de nite relaxation proposed by Shor [26] (see also [30]). Much progress was made in the last years in developing polynomialtime interior point methods for semi-de nite programming (see e.g. the reviews [25] and [30]). Results pertaining to the quality guarantees given by the Shor relaxation for some speci c classes of QP problems will be reviewed in Section 3.1. As already noted in the cited article of Shor, linear side-constraints can cause the semide nite relaxation to work badly; he presented a modi cation to overcome this problem, by quadratizing the linear constraints. In [22], such a quadratization is worked out for linear equality constraints. In this paper, we use a convex relaxation for the case with linear inequality constraints. This relaxation uses cone-positive matrices. Using some theoretical results about copositive and cone-positive matrices, we show that this cone-positive relaxation is stronger than the modi ed Shor-relaxation when linear constraints are involved, but that its complexity is worse (it is NP-hard), although it is a convex optimization problem. The paper is organized as follows. First, we state the optimization problem, introduce some matrix cones and show some duality results on these cones. In Section 3, both the standard Shor-relaxation and a modi ed Shor-relaxation are explained. Our cone-positive relaxation is derived using conic duality theory in Section 4. In Section 5, the solution qualities of the Shor-relaxation and the cone-positive relaxation are compared. We derive some conditions on matrix sizes under which both types of relaxation are the same, and show an example where both types of Shor-relaxations give di erent relaxed solutions, while the cone-positive relaxation gives an exact solution. In Section 6, we discuss the computational complexity of algorithms to solve the relaxations. 1 2 Problem statement, Notation and De nitions We consider the General Quadratic Programming problem max xT A0 x; s.t. xT Ai x = bi; i = 1; :::; m; (1) Dx  0: Here Ai ; i = 0; :::; m are symmetric n  n matrices, D is a p  n matrix and x is the variable vector of dimension n. Let us point out that this formulation includes quadratic constraints with linear terms; consider a constraint of the form x~T A~i x~ + 2~bTi x~ + c~i = 0 (2) for some i, then we can introduce    ~i ~bi  x ~ A x = 1 ; Ai = ~bT ~c ; bi = 0 (3) i i to transform (2) to a quadratic constraint of the form appearing in (1). Problem (1) can be modi ed to put all nonlinearities into one set of constraints. This is achieved by introducing the variable matrix X, which is de ned as the dyadic product X = xxT : Recall that the trace of a matrix, Tr (X), is the sum of its diagonal elements, and that for two symmetric matrices X and Y : Tr (XY ) = XX i j xij yij : The quadratic terms xT Ai x; i = 1; :::; m now can be reduced to linear terms, using the variable X, as follows: ?
?
xT Ai x = Tr xT Ai x = Tr Ai xxT = Tr (Ai X) : In this way, the following reformulation of (1) is obtained. max Tr (A0 X) ; s.t. Tr (Ai X) = bi; i = 1; :::; m; (4) X = xxT ; Dx  0: The only nonlinearity occurs in the relation between x and X. In the rest of the paper, we will refer to (4) as the standard form. It is not convex in general. To nd upper bounds on the optimal value, we will relax (4) by replacing the constraints X = xxT ; Dx  0 by constraints of the form X 2 A, where A is a convex matrix cone. 2.1 Matrix cones, de nitions and properties It is assumed in the rest of this paper that all matrices, except D, are symmetric. By A  0 we mean that each entry of A is nonnegative. With xi; i = 1; :::; k we mean a set of vectors. The subscript notiation xi refers to an element of the vector x. The set M is the set of symmetric square matrices of size depending on the context. We will consider some special classes of matrices that form cones in M . The reader can easily verify that all these cones are convex. For some of these cone de nitions, we need the polyhedral set D = fx : Dx  0g 2 that de nes the feasible area of (4) with respect to the linear constraints. The matrix cones under consideration are:  Nonnegative matrices: N = f 2 M :  0g ND = f 2 M : = T 2 Ng = T N T ND = f 2 M : 2 Ng  Positive-semide nite (PSD-) matrices: S = f 2 M : T  0 8 g = f 2 M : = Pki=1 i i T  1g where the i 's are vectors of appropriate dimensions.  Cone-positive matrices (matrices that are semide nite on the polyhedral cone D ): C D = f 2 M : T  0 8 2 Dg  Completely cone-positive P matrices: BD = f 2 M : = ki=1 i i T  1 i 2 D = 1 g  Copositive matrices (cone-positive on the nonnegative orthant): C = f 2 M : T  0 8  0g  Completely positive matrices (completely cone-positive on the nonnegative orthant): B = f 2 M : = Pki=1 i i T  1 i  0 = 1 g There are several relations between these matrix cones. Let be the identity matrix and the zero matrix of appropriate size, then N = N I = N I B = BI C = C I S = B O = C O N N : X X X DX D S x D N D; N D D: : Sx x S S s s ;k ; s C x B B C x B B Cx x : s s Cx ; k x ; s ; i ; s ; i ; :::; k : : s s ; k ; :::; k : I O ; ; ; : De nition 1 For a given closed convex matrix cone A  M, its dual cone A, is de ned as follows: A = f 2 M : Tr ( )  0 8 2 Ag Note that A = A, which is a well-known result that can be found in e.g. [1]. Lemma 1 (N D ) = N D X XY Y : : 2 N D , it holds for all = T 2 N D that ?
? T
 0 ) = Tr T = Tr T  0. Thus N D  (N D ) . To prove the other implication, T )ij N D . Then ( 0 for some pair ( ). Let ij be the matrix with ones at positions ( ) and ( ), and zeros elsewhere. Then ij 2 N , T ij 2 N D and T ij
0 Tr ? T ij
= Tr ? so 2 (N D ) which proves the lemma. 2 Taking = we obtain as a consequence, that the cone N is self-dual, which is a well- Proof: Given any Tr ( since  0 and let us consider a 2 Y Z ZY N D N DY = ; < i; j j; i = E D DY D < E ; ; D known fact. I Lemma 2 B D = C D . Proof: Given any 2 C D , it holds for all = Pki=1 i i T 2 B D that Y Tr ( ) = Y Z Z k X i=1 Tr  z z k  i iT = X ziT Y zi i=1 Y z z 3 E E D YD Y N DY D DY D i; j E ND DY D Y D D 0 ; so C D  BD . To prove the other implication, let us consider a Y 2= C D . According to (2.1) a vector z 2 D exists such that z T Y z < 0: Now de ne Z = zz T , then Z 2 B D and ?
Tr (Y Z) = Tr zT Y z = z T Y z < 0; 2 so Y 2= BD . It follows that BD  C D , hence C D = BD . Taking D = I, we get as a consequence that B  = C , which is also proved in [12]. Taking D = O we get that S is self-dual, which is also a well-known result. Given two cones of matrices A1 and A2 , we de ne A1 + A2 = fX : X = A1 + A2; A1 2 A1 ; A2 2 A2g: We will use in this paper the well known relation (A1 + A2 ) = A1 \ A2 . We end this section with stating a primal-dual pair of conic optimization problems, which will be used extensively in this paper. This pair can be found in e.g. [31]. Lemma 3 Given a convex cone A 2 M, the following two problems form a convex primal-dual pair of optimization problems. (P) max Tr (A0 X) s.t. Tr (Ai X) = bi; i = 1; : : :; m X 2A (D) min bPT my s.t. i=1 yi Ai ? A0 2 A  The optimal values of the primal and dual problem are equal if the generalized Slater constraint quali cation holds, i.e. if an X exists such that X 2 intfAg; Tr (Ai X) = bi ; i = 1; : : :; m and an y exists such that m X i=1 3 yi Ai ? A0 2 intfA g: Shor-relaxation In this section, we present the Shor-relaxation and review known results on its quality guarantees for QP. Initially, linear constraints are ignored. In the second subsection, we consider the case with linear inequality constraints. 3.1 Quadratic constraints only In this section we deal with the problem (4) without the linear inequality constraints. The Shor-relaxation relaxes the constraint X = xxT to X 2 S , to obtain max Tr (A0 X) ; s.t. Tr (Ai X) = bi; i = 1; :::; m; (5) X 2 S: 4 Using the second de nition of S , we note that the Shor-relaxation in fact relaxes the rank-one constraint on X . The corresponding dual problem is (according to Lemma 3) min bPT y; (6) m y A ? A 2 S: s.t. 0 i=1 i i Several new results concerning quality guarantees of the Shor relaxation have been published recently. Lovasz and Schrijver [14] rst considered the following Boolean QP problem which arises in combinatorial optimization, namely  qmax = max xT Qx : xi 2 f?1; 1g (8i) ; (7) which has as Shor relaxation1 q = max fTr (QX ) : diag(X ) = e; X 2 Sg ; (8) with e the all-one vector. For this general relaxation Nesterov [20] recently proved that
? q ? q  qmax ? qmin  4 ?  q ? q where (qmin ; qmax ) is the range of feasible objective values in (7), and (q; q) is the range of feasible values in the relaxation problem (8). Moreover, a random feasible solution x to (7) can be computed from the solution to the relaxation (8). The expected objective value of x, say E (x), satis es qmax ? E (x) < 4 : qmax ? qmin 7 Such an algorithm is called a (randomized) -approximation algorithm, with  = 4=7 in this case. The same bounds were subsequently obtained by Ye [32] for the `box-constrained' problem where xi 2 f?1; 1g is replaced by ?1  xi  1 in problem (7); these results were P further extended in [33] to include simple quadratic constraints of the form: ni=1 aix2i = b. These recent results are extensions of work by Goemans and Williamson [9] on the maximal cut problem, i.e. the problem of nding a cut of maximal weight through a graph with weighted edges. The special structure of this problem actually allows the bound q  1:14qmax. The randomized algorithm moreover produces a cut with expected value greater than 0:878qmax in this case. These positive results unfortunately do not include QP problems involving general linear inequality constraints. In fact, Bellare and Rogaway [2] have showed that the generic QP problem  max xT Qx : Ax  b; x 2 [0; 1]n has no polynomial time -approximation algorithm for some  2 (0; 1), unless P=NP. This latter result motivates the alternative convex relaxations studied in this paper, which cannot be solved in polynomial time in general, but are tighter than the Shor relaxation. 3.2 Linear inequality constraints It was observed in [26] that the linear inequality constraints cannot be handled the same way as the quadratic equality constraints. In this section, we will show, that the homogeneous linear constraints of (4), dj T x  0; j = 1; :::; p; 1 Note that if x i 2 f?1 1g then diag( ; xx T ) = e. 5 will completely vanish in the Shor-relaxation, when handled as general quadratic constraints of the form (2). When treating the constraints in this way, we have to introduce    0 dj =2  x = x1 ; Am+j = dj T =2 0 ; j = 1; :::; p: We also extend Ai ; i = 0; :::; m and X with one row and column containing zeros, and de ne     X x x X := xT 1 = 1 (xT 1): Problem (4) then becomes ?
max Tr ?A0X
; s.t. Tr ?AiX
= bi ; i = 1; :::; m; Tr Aj X  0; j = m + 1; :::; m + p; X = xxT ; xn+1 = 1: (9) The Shor-relaxation replaces the rank-one constraint by the constraint X 2 S . According to the second de nition of S in Section 2.1, this means that the Shor-relaxation requires that   k  i  X  ( i T z ); X  xXT x1 = i z i=1 i for some positive number k. This is equivalent to the system X = Pki=1  i  i T k z i x = i=1 i Pk 2 = 1: z i=1 i Using (9), this leads to the following formulation of the Shor-relaxation, expressed using the original-size matrices Ai ; i = 0; :::; m: max Tr (A0 X) ; s.t. Tr (Ai X) = bi; i = 1; :::; m; P XP = ki=1  i  i T (10) k z i)  0 D( i Pk i=1 2 = 1: z i i=1 P Now, given a k, and vectors  i ; i = 1; :::; k such that the rst two sets of constraints are satis ed, add the vector  k+1 = 0, let zi = 0; i = 1; :::; k and let zk+1 = 1, to see that we can always satisfy the last two sets of constraints. Thus, the linear constraints are in fact ignored in the Shor-relaxation, and we conclude that the Shor-relaxation can be written as (5), regardless of the matrix D. 3.3 Improved Shor-relaxation Shor presents an alternative way to treat the linear constraints. In the primal problem (4), the linear constraints dj T x  0; j = 1; :::; p are replaced by the constraints xT didj T x  0; i; j = 1; :::; p: 6 Using the transformation   xT di dj T x = Tr didj T X = dj T Xdi = (DXDT )ji; we can put these constraints together in the matrix inequality DXDT  0 (, X 2 N D ): (11) We obtain the following relaxation of (4): max Tr (A0X) s.t. Tr (Ai X) = bi ; i = 1; :::; m (12) X 2 S \ ND with its dual min bT y P (13) s.t. ?(A0 + mi=1 yi Ai ) 2 S + N D : Compared to the original problem (4), in (12) the rank-one constraint on X is relaxed, but the matrix X should in a sense be positive with respect to D. More precisely, X should be of the form k X x` x` T ; where k 2 Z + and `=1 k X diT x` x`T dj `=1  0; i; j = 1; :::; p: By a simple example, we show that this relaxation works better than the standard Shor-relaxation. Example 1 Consider the example (with optimum value 0)   ? 1 ? 2 T max x ?2 ?1 x s.t. Ix  0: The original Shor relaxation reads  (14)   ?1 ?2 X max Tr ? 2 ?1 s.t. X 2 S : The feasible set and the optimum value of this relaxation are unbounded, as can be seen by choosing   X = lim !1 ? ? : This corresponds to the fact that the dual problem min 0 s.t. ?1 ?2 ?2 ?1  2 S is infeasible. With the constraints (11) the modi ed Shor-relaxation becomes    ?1 ?2 X max Tr ? 2 ?1 s.t. X 2 S \ N I = S \ N : 7 This problem has optimum value 0. The dual problem min 0 s.t. 1 2 2 1  2 S + NI = S + N; (15) also has optimum value 0, as can be seen by observing that       1 2 = 1 0 + 0 2 : 2 1 0 1 2 0 Although in the above example, the improved Shor-relaxation was tight, this is in general not true. The third type of relaxation, that we state in the next section, is still tighter than the improved Shor-relaxation. When we refer to the Shor-relaxation in the rest of the paper, we mean the improved Shor-relaxation, unless we refer to the original Shor-relaxation explicitly. 4 Cone-positive relaxation In the original Shor-relaxation, we found that the constraints X = xxT ; Dx  0, are relaxed to the semide niteness constraint k X (16) xi xiT ; k  1; X = i=1 thus 'forgetting' the linear constraints. The improved Shor-relaxation adds to (16) the additional constraint k X T Dxi xiT DT  0: (17) DXD  i=1 We still can do better. In the following relaxation, we add to (16) the set of constraints Dxi  0; i = 1; :::; k: (18) Theorem 1 The GQP as stated in (4) can be relaxed by the following 'completely conepositive' problem max Tr (A0 X) s.t. Tr (Ai X) = bi ; i = 1; :::; m; (19) X 2 BD with its 'cone-positive' dual min bPT ym s.t. i=1 yi Ai ? A0 2 C D : Proof: The cone BD consists of all matrices k X xixi T ; k  1; Dxi  0; i = 1; :::; k: X = i=1 (20) The set of feasible solutions of (4) contains of the matrices X = xxT ; Dx  0; so it is contained in B D . The primal problem (19) is therefore a relaxation of (4). Using Lemma 2 and Lemma 3, the dual follows. 2 Since BD and C D are convex cones, both problems are convex optimization problems. Moreover, since (18) implies (17), the relaxation is at least as tight as the Shor-relaxation. In the following, we will derive the relation between the two relaxations in more detail, both with regard to quality and solvability. 8 5 Comparison of the quality of relaxations In this section, we will discuss the relation between the feasible areas of the Shor-relaxation and the cone-positive relaxation, and study under which conditions both relaxations are equal. We do this by considering the dual feasible sets of the Shor-relaxation and the cone-positive relaxation. A relaxation will be called stronger than a second one if the rst one gives better bounds on the primal solution. We rst consider the special case that D = I. 5.1 Important case: D = I Many practical optimization problems deal with nonnegativity constraints. When the set of linear inequalities reduces to the constraint that all variables are nonnegative, D becomes the identity matrix. The cone-positive relaxation then reduces to what we will call the copositive programming problem: max Tr (A0 X) s.t. Tr (Ai X) = bi; i = 1; :::; m; X 2 B with its dual min bPT ym s.t. i=1 yi Ai ? A0 2 C : We study its quality compared to the Shor-relaxation, by comparing the dual cones S + N D of the Shor-relaxation and C of the copositive relaxation. Since D = I, N D reduces to N . Clearly, C  N + S; since both a nonnegative matrix and a semide nite matrix are copositive (it follows also since (18) implies (17)). This implies that the copositive dual has a feasible region that is at least as large as the Shor dual, thus the copositive relaxation is at least as strong as the Shor-relaxation. For matrix size n  4, both relaxations are equivalent. This follows from a result that is proved by Diananda in [8]. We state it as a lemma here since it is also needed for the case D 6= I. Lemma 4 For matrix size n  4, C = N + S: The proof is not dicult but lengthy, and is not repeated here. For n = 5, the cited article provides an example matrix, described in more detail in [12], that is in C but not in N + S , namely the matrix 0 1 ?1 1 1 ?1 1 B B ?11 ?11 ?11 ?11 11 CCC : J = B (21) B @ 1 1 ?1 1 ?1 CA ?1 1 1 ?1 1 We only show here that it is in C . The matrix corresponds to a quadratic function, that may be expressed in di erent ways: xT Jx = (x1 ? x2 + x3 + x4 ? x5 )2 + 4x2x4 + 4x3(x5 ? x4); = (x1 ? x2 + x3 ? x4 + x5 )2 + 4x2x5 + 4x1(x4 ? x5): 9 For  0, the rst formulation shows nonnegativity of T if 4  5 , the second formulation shows nonnegativity if 4  5, thus T  0 for all  0. x x x x x Jx Jx x x x The abovementioned results are summarized in the following theorem. Theorem 2 The Shor-relaxation is not stronger than the copositive relaxation. Both relaxations are the same for matrix size  4. For 4, there exist problems for which n n > the copositive relaxation is stronger than the Shor-relaxation. Below, we illustrate this theorem with an example where = 5. n Example 2 In this example, the original Shor-relaxation, the improved Shor-relaxation and the Copositive relaxation give three di erent bounds. The problem reads min s.t. T Jx P5 2 x ; x i=1 xi = 1  0 with as de ned in (21). Since is copositive, p thepsolution isT nonnegative. The exact J J solution is 0, for example obtained by x = (1= 2; 1= 2; 0; 0; 0) . The original Shor-relaxation from Section 3.2 reads ( ) min Tr ( ) s.t. Tr ( ) = 1 P ( ) max JX X X D ; 2S s.t. y J ? 2S yI The optimum solution value p of the dual problem corresponds to the smallest eigenvalue of 5  ?1:236. J , which is equal to 1 ? The improved Shor-relaxation from Section 3.3 reads ( ) min Tr ( ) s.t. Tr ( ) = 1 P ( ) max JX X X 2 S \N D s.t. ; y J ? yI 2S +N p For this problem, the optimum value 2 ? 5  ?0:23606. This is easily checked by taking the primal solution X p 0 2 p5 ? 1 0 0 5?1 1 p p BB 5 ? 1 p 2 5?1 p 0 0 C C 2 S \N 1 B = 10 B 0 5?1 p 2 5?1 p 0 C C @p0 5?1 p 2 5?1 A 0 5?1 0 0 5?1 2 and the dual matrix 0p ? ? B ?p B @ p 5 J ? (2 ? p5) I 1 = 1 3 5 2 3? 5 2 ?1 1 3? 5 3? 5 0 p5??1 1 ?21 3?2p5 3??p1 5 p5 ? 1 00 2 2p C p ?p1 5 ? 1 p ?1 3?2 5 C A + 2 @ 11 3? 5 5 ? 1 ? 1 ? 1 0 2p p 3? 5 3? 5 ?1 p5 ? 1 2 2 p p The Copositive relaxation reads ( ) min Tr ( ) s.t. Tr ( ) = 1 P ( ) max JX D 0 0 0 1 1 1 0 0 0 1 1 1 0 0 0 0 1 1 0 0 1 A2S : + N : y ? 2C 2B Since 2pC , thepoptimum value p for (D) p is greater than or equal to zero. But the solution T X ; s.t. J yI X J X = (1 2 1 2 0 0 0) (1 2 1 2 0 0 0) is feasible for (P) with optimum value 0, = ; = ; ; ; = ; = ; ; ; so this relaxation gives the exact solution 0. 10 5.2 General case To compare the quality of the cone-positive relaxation with the quality of the Shorrelaxation for general , we compare the dual cones C D of the cone-positive dual and S + N D of the Shor dual. Firstly, since (18) implies (17), the Dcone-positive relaxation is at least as strong as the Shor-relaxation, implying that S + N  C D . To derive conditions under which both cones are the same, we use the intermediate cone CS D = f : = T + 2 C 2 Sg = T C + S and nd conditions under which C D = CS D resp. CS D = S + N D . For the rst equality, we refer without proof to the following lemma from Martin and Jacobson [16]. Lemma 5 (Martin and Jacobson) Let 2 IRpn . Then C D = CS D if one of the D Y Y D CD S; C ; S D D D following three conditions is satis ed: 1. rank D = p; 2. p  4 and there exists an x such that Dx > 0; 3. n = 2 and there exists an x such that Dx > 0. Under the rst condition, the restriction that a strictly interior point of D exists is automatically satis ed. An extensive proof of the lemma can be found in [16], together with a proof that counterexamples exist for the case = 3, = 5. The following lemma deals with the second equality. Lemma 6 The cones CS D and S + N D are equal if and only if rank( )  4. x n p D Proof: It is immediately clear that CS D  S + N D since C  N . To prove that D CS D  S + N for rank( )  4, suppose the matrix has size  and rank , and let = T + 1 2 CS D 2 C 1 2 S D Y D D CD S ; C p n r < p ; S be given. The rows of are ordered in such a way that the rst rows are independent and the last ? rows can be expressed as linear combination of the rst rows. De ne, for all such that  0, the vector () = Then the last ? elements of ( ) are linear functions of 1 r. This implies that the quadratic form T T = T is only a quadratic form in 1 r. Therefore, the  matrix can be replaced by an  copositive matrix ~ . We use Lemma 4 to obtain that for  4, the matrix ~ can be replaced by ~ + ~, ~ 2 N , ~ 2 S . Using  ~   ~   ~  = 0 00 = 0 00 = 0 00 we obtain = T + 1 = T + 1 = T + T + 1 = T + 2 2 S + ND To show the necessity part of the lemma, a counterexample for the case that rank( )  5 is easily obtained by choosing = ( from (21)) and = . 2 D p r r x r Dx y x p r Dx: y x x y ; :::; y D C Dx y y ; :::; y r r S p C CD S r N D CD C S C C D p C N Y Cy ; N N S D ND ; D SD S S S D ND ; S : D C J J D I Now, combining Lemma 5 and Lemma 6 gives the following theorem. 11 Theorem 3 The Shor-relaxation is not stronger than the cone-positive dual. Both relaxations coincide if D = fx : Dx  0g has a strictly interior point, and either p  4 or = 2. Since it is proven that counterexamples exist to Lemma 5, there exist also counterexamples for the case that D has no strictly interior point, or none of the conditions of Theorem 3 is satis ed. We may conclude that the two relaxations are the same only for a very small number of side-constraints, or for very small matrix sizes. For larger problems, the cone-positive dual may provide a stronger relaxation. In the next section, we will study the solvability of the two relaxations. n 6 Solvability All described relaxations are convex programming problems in conic form. In principle, all convex optimization problems can be cast in the conic form. Nesterov and Nemirovskii [19] show that such conic optimization problems can be solved by sequential minimization techniques, where the conic constraint is discarded and a barrier term is added to the objective. Suitable barriers are called self-concordant. These are smooth convex functions with second derivatives which are Lipschitz continuous with respect to a local metric (the metric induced by the Hessian of the function itself). More precisely, the de nition is as follows. De nition 2 (Self-concordant barrier) A convex function f : C 7! IR is called selfconcordant if X i;j;k () @ 3f x @xi @xj @xk  hi hj hk  khr2f (x)h; hi3=2; for all x 2 int(C ) and h 2 IRn, where k  0 is a xed constant. A strictly convex selfconcordant function f is called a  -self-concordant barrier if k = 2 and it also satis es 2hrf (x); hi ? hr2 f (x)h; hi   for all h 2 IRn . Self-concordant barriers go to in nity as the boundary of the cone is approached, and can be minimized eciently by Newton's method.2 Using such a barrier, one can formulate optimization algorithms that require only a polynomially bounded number of iterations using exact arithmetic to compute -optimal solutions. Theorem 4 (Nesterov and Nemirovskii) Given a  -self-concordant barrier for a closed convex domain G, one can associate with this barrier?interior-point p ?

methods, for minimizing linear objectives over G, that require at most O  log 1 iterations to compute an -optimal solution from a suitable starting solution. The function S (X ) = ? log(det(X )) (22) is ?np-self-concordant for S [19], and the standard Shor-relaxation is therefore solvable in ?

O n log 1 iterations, provided the Slater condition holds for both primal and dual 2 A well-written introductory text dealing with self-concordance is [13]. B 12 problems.3 Likewise, the function BN (X ) = ? n X i;j =1 log(Xi;j ) = ?eT log[X ]e is a 21 n(n + 1)-self-concordant barrier for N , where log[X ] denotes an element-wise operator. Applying simple combination rules, it follows that a barrier for N D is given by BP (X ) = ?eT log[DXDT ]e: (23) It is easy to see that a barrier for S \ N D is obtained by summation of the two barriers (22) and (23): BS \P = ? log(det(X )) ? eT log[DXDT ]e: (24) ? ? 1

It follows that the modi ed Shor-relaxation is solvable in O n log  iterations. A self-concordant barrier also exists for the copositive cone C , according to the following theorem. Theorem 5 (Nesterov and Nemirovskii) Given a closed convex domain G  IRn, there exists a self concordant barrier for G. If G does not contain any one-dimensional subspace of IRn , then we can take the barrier F (x) : int(G) ! IR : F (x) = O(1) ln jG0(x)j; where O (1) is an appropriately chosen absolute constant, G0(x) = f' 2 IRn : 'T (y ? x)  1 8y 2 Gg is the polar of G with respect to the point x, and j
j denotes the n-dimensonal volume. The problem is that this theorem only gives an implicit de nition of the barrier, and no explicit expression is known in general. In fact, there is a reason to suspect that a self-concordant barrier for C can not be evaluated in a polynomial number of operations. This reason is that detecting copositivity of a matrix is an NP-complete problem [17, 18]; the copositive optimization problem is therefore NP-hard, i.e. the recognition version of the problem is NP-complete4. In comparison, the recognition version of a general semide nite programming problem (without the Slater condition) is not NP-complete, unless NP=co-NP [24]. On the other hand, the convexity of the copositive optimization problem suggests that it is in some sense an 'easier' NP-hard problem than the original GQP problem. Remark 1: To nd a barrier, we should know something about the boundaries of the cone. The semide nite cone S is bounded by the surface det(X ) = 0, and det(X ) > 0 in int(S ). However, the copositive cone is bounded by parts of all 2n ? 1 surfaces det(Y J J ) = 0, where Y J J is a principal submatrix of Y obtained by deleting all rows and columns with indices not in the set J  f1; :::; ng. Moreover, these determinants may be zero for some matrices in int(C ). A review of known complexity results for semide nite programming is given in [25]. Roughly speaking, the recognition version of an optimization problem is the problem to nd whether there exists a rational solution with objective value equal to or better than a xed constant. See [21] for a detailed treatment. 3 4 13 Remark 2: In a recent paper by Guler [10], a new way of constructing self-concordant barries for so-called homogeneous cones5 was presented. Unfortunately, this kind of construction is impossible for the copositive cone, since it is not homogeneous. As an alternative to barrier methods, one might use gradient-based methods for optimization. Usually, such methods contain a line-search. For such a line-search, it is important to detect whether or not a matrix is in the cone. Again, detecting whether a matrix is in C is already an NP-complete problem. There are several tests for detection of copositivity. As an example, we recall the following theorem of Keller, stated in [6]. It makes use of cofactors: determinants of submatrices where one row and one column are deleted. Theorem 6 (Keller) A matrix Y is copositive if and only if each principal submatrix for which the cofactors of its last row are nonnegative, has a nonnegative determinant. This theorem shows that not only the determinants of some principal submatrices are important, even cofactors of the principal submatrices have to be considered. Another feature of this test is that it is stated in a conditional form: there may be principal submatrices with negative determinant, if they have also a negative cofactor in the last row. Other tests for copositivity can be found in e.g. [3, 4, 7, 11, 15, 27, 28, 29]. In [5], the test of [4] is used to develop an algorithm that solves Quadratic Programs with inde nite objective function and linear constraints to global optimality. This algorithm is nite but exponential. We may conclude that, for general copositive optimization problems, an algorithm with polynomial bound on the iteration count is unlikely. It is topic of further research to see whether one can exploit the structure of certain subclasses, in order to make ecient algorithms. For tridiagonal matrices, the test in Theorem 6 can be simpli ed. One can show that only principal submatrices with connected index sets have to be considered, since all other principal submatrices are block-diagonal. There are n(n ? 1)=2 such submatrices, and testing one of them requires the computation of at most n determinants, each of order at most n ? 1. Computation of a determinant of order n can be performed using pivoting in O (n3) steps, and thus we obtain the following corollary to Theorem 6. Corollary 1 The copositivity of a tridiagonal matrix can be checked in at most O(n6) operations. Practical computations in this case can be much simpli ed using the sparsity patterns. Ecient algorithms for this case may therefore be possible, and are the subject of further study. 7 Conclusion The copositive relaxation for general quadratic programming problems is tighter than the Shor-relaxation. Solving the copositive relaxation is less ecient, since even checking whether a point is feasible is an NP-complete problem, despite the fact that the feasible region is convex. However, since examples show that the relaxation is really tighter than the Shor-relaxation, it might be interesting to investigate algorithms to solve special cases, to study their behaviour and see how it compares to the Shor-relaxation. An open convex cone K is called homogeneous if for each pair x; y 2 K there exists an isomorphism : K 7! K satisfying g(x) = y. Examples include the nonnegative orthant and the cone of positive semide nite matrices; homogeneous cones are not necessarily self-dual. 5 g 14 References [1] M.S. Bazaraa, H.D. Sherali, and C.M. Shetty. Nonlinear Programming. Theory and Algorithms. Wiley & Sons, 1993. [2] M. Bellare and P. Rogaway. The complexity of approximating a nonlinear program. Mathematical Programming, 69:429{441, 1995. [3] I.M. Bomze. Block pivoting and shortcut strategies for detecting copositivity. 1995. [4] I.M. Bomze and G. Danninger. A global optimization algorithm for concave quadratic programming problems. Siam J. Optimization, 3(4):826{842, 1993. [5] I.M. Bomze and G. Danninger. A Finite Algorithm for Solving General Quadratic Problems Journal of Global Optimization, 4:1{16, 1994. [6] R.W. Cottle, G.J. Habetler, and C.E. Lemke. On classes of copositive matrices. Linear Algebra and its Applications, 3:279{294, 1970. [7] R.W. Cottle, G.J. Habetler, and C.E. Lemke. Quadratic forms semi-de nite over convex cones. In H.W. Kuhn, editor, Proceedings of the Princeton Symposium on Mathematical Programming, pages 551{565, 1970. [8] P.H. Diananda. On non-negative forms in real variables some or all of which are non-negative. Proceedings of the Cambridge Philosophical Society, 58:17{25, 1962. [9] M.X. Goemans and D.P. Williamson. Improved approximation algorithms for maximum cut and satis ability problems using semide nite programming. Journal of Assoc. Comput. Mach., 42:1115{1145, 1995. [10] O. Guler. Hyperbolic polynomials and interior point methods for convex programing. Mathematics of Operations Research, 22(2), 1997. [11] K.P. Hadeler. On copositive matrices. Linear Algebra and its Applications, 49:79{89, 1983. [12] M. Hall, Jr. and M. Newman. Copositive and completely positive quadratic forms. Proceedings of the Cambridge Philosophical Society, 59:329{339, 1963. [13] F. Jarre. Interior-point methods for classes of convex programs. In T. Terlaky, editor, Interior point methods of mathematical programming, pages 255{296. Kluwer, Dordrecht, 1996. [14] L. Lovasz and A. Schrijver. Cones of matrices and set{functions and 0-1 optimization. SIAM Journal on Optimization, 1:166{190, 1991. [15] D.H. Martin. Finite criteria for conditional de niteness of quadratic forms. Linear Algebra and its Applications, 39:9{21, 1981. [16] D.H. Martin and D.H. Jacobson. Copositive matrices and de niteness of quadratic forms subject to homogeneous linear inequality constraints. Linear Algebra and its Applications, 35:227{258, 1981. [17] T.S. Motzkin. Signs of minors. In O. Shisha, editor, Inequalities, pages 225{240. Academic Press, 1967. [18] K.G. Murty. Some NP-complete problems in quadratic and nonlinear programming. Mathematical Programming, 39:117{129, 1987. [19] Yu. Nesterov and A.S. Nemirovskii. Interior Point Polynomial Algorithms in Convex Programming. SIAM Studies in Applied Mathematics, Vol. 13. SIAM, Philadelphia, 1994. [20] Yu. Nesterov. Quality of semide nite relaxation for nonconvex quadratic optimization. CORE Discussion paper 9719, Belgium, March 1997. 15 [21] C.H. Papadimitriou and K. Steiglitz. Combinatorial Optimization. Prentice-Hall, 1982. [22] S. Poljak, F. Rendl and H. Wolkowicz. A Recipe for Semide nite Relaxation for (0,1)-Quadratic Programming. Journal of Global Optimization, 7(1):51{73, 1995. [23] M.V. Ramana. An Algorithmic Analysis of Multiquadratic and Semide nite Programming Problems. PhD thesis, The Johns Hopkins University, Baltimore, 1993. [24] M. Ramana. An exact duality theory for semide nite programmingand its complexity implications. Mathematical Programming, Series B, 77(2):129{162, 1997. [25] M.V. Ramana and P.M. Pardalos. Semide nite programming. In T. Terlaky, editor, Interior point methods of mathematical programming, pages 369{398. Kluwer, Dordrecht, The Netherlands, 1996. [26] N.Z. Shor. Quadratic optimization problems. Soviet Journal of Computer and System Sciences, 25(6):1{11, 1987. [27] H. Valiaho. Criteria for copositive matrices. Linear Algebra and its Applications, 81:19{34, 1986. [28] H. Valiaho. Testing the de niteness of matrices on polyhedral cones. Linear Algebra and its Applications, 101:135{165, 1988. [29] H. Valiaho. Quadratic-programming criteria for copositive matrices. Linear Algebra and its Applications, 119:163{182, 1989. [30] L. Vandenberghe and S. Boyd. Semide nite programming. SIAM Review, 38:49{95, 1996. [31] H. Wolkowicz. Optimization applied in matrix theory. Linear Algebra and its Applications, 40:101{118, 1981. [32] Y. Ye. Approximating quadratic programming with bound constraints. Working paper, Dept. of Management Science, University of Iowa, Iowa City, USA, 1997. [33] Y. Ye. Approximating quadratic programming with quadratic constraints. Working paper, Dept. of Management Science, University of Iowa, Iowa City, USA, 1997. 16