Duality aspects in convex conic programming

Duality aspects in convex conic programming Mária Trnovská, Jakub Hrdina a Faculty of Mathematics, Physics and Informatics, Comenius University, Mlynská dolina F1, Bratislava, 842 48, Slovakia Abstract In this paper we study strong duality aspects in convex conic programming over general convex cones. It is known that the duality in convex optimization is linked with specific theorems of alternatives. We formulate and prove strong alternatives to the existence of the relative interior point in the primal (dual) feasible set. We analyze the relation between the boundedness of the optimal solution sets and the existence of the relative interior points in the feasible set. We also provide conditions under which the duality gap is zero and the optimal solution sets are unbounded. As a consequence, we obtain several alternative conditions that guarantee the strong duality between primal and dual convex conic programs. Keywords: convex conic programming, strong duality, generalized theorems of alternatives 2020 MSC: 90C22, 90C25 1. Introduction 5 10 15 Duality is a powerful tool in mathematical optimization and it played an important role in the early development of non-linear programming. It has been studied in connection with the simplex method, the augmented Lagrangian methods and also the interior point methods. Valuable results for interior point methods in linear programming were obtained from the duality theory in [1], where it was shown that the primal (dual) feasibility together with the dual (primal) strict feasibility is equivalent to the non-emptiness and boundedness of the primal (dual) optimal solution set, respectively. The result was extended to the case of semidefinite programming by [2] and [3]. It is known that the existence of the interior point in the feasible set guarantees the zero duality gap (strong duality property). Due to the equivalence, we obtain an alternative sufficient conditions for the strong duality. This result appeared to be useful in various practical aspects of convex optimization [4, 5, 6], polynomial optimization and Lasserre’s hierarchies [7, 8, 9] and control problems [10, 11, 12]. The aim of our Email addresses: [email protected] (Mária Trnovská), [email protected] (Jakub Hrdina) Preprint submitted to Linear Algebra and its Applications October 28, 2021 20 25 30 35 40 45 50 55 60 paper is to generalize these results to the case of general convex conic programs and provide alternative sufficient conditions that guarantee the strong duality property together with the analysis of the boundedness of the optimal solution sets. Every standard convex optimization problem can be formulated in a conic form with a linear objective and linear equality constraints, where the variable is assumed to belong to a convex cone. On the other hand, convex conic programming includes special sub-classes, such as semidefinite or copositive programming. Therefore, the duality theory of general convex conic programs has been intensively studied, usually in association with proper (convex, closed, solid and pointed) cones and was included in standard convex optimization textbooks [13, 14]. Duality analysis in connection with the facial structure of the cones with the emphasis on the semidefinite programs was given in [15]. For non-pointed or non-solid convex cones, the theory cannot be extended straightforwardly from the standard conic programming classes (such as linear or semidefinite programming). Several duality aspects for general convex conic programs in subspace form have been studied in [16]. The duality in convex optimization is linked with specific theorems alternatives. The most commonly known is the Farkas lemma [17] for linear systems and its generalizations [18, 19, 20, 21, 22]. In the generalized, conic versions of the Farkas lemma, the strict alternatives hold under the assumption of closedness of the linear image of the convex cone. The necessary and sufficient conditions for the closedness in this respect were studied in [23]. The generalized Farkas result then serves as a tool for proving the strong duality provided that the Slater constraint qualification (existence of an interior point in the feasible set) holds [13, 19]. Since we deal with general (not necessarily solid) convex cones, the analysis of the relative interior and its characterization is needed. This is provided in Section 2, together with the basic notation, the review of some known properties of convex cones and the conic programs formulations. Section 2 also includes results related to the closedness of the linear image of the convex cone as well as the Minkowski sum of a cone and a linear subspace (which serves as an analogical condition for the strong alternatives to hold in the dual case). In Section 3 we present four theorems of alternatives. Except for the generalized Farkas type theorems, we present new ones that give equivalent conditions to the existence of the relative interior point in the primal (dual) feasible set. These new theorems of alternatives appeared to be useful for proving that the boundednes of the (nonempty) optimal solution set implies the existence of an interior point of the set of feasible solutions in the dual counterpart. This last result, included in Section 4, generalizes the results of [1, 2, 3] to the case of general convex conic programs and offers an alternative sufficient condition for strong duality. Strong duality in convex conic programming is actually the main topic of Section 4. In this section we summarize the known results and relate them to our findings, which include new sufficient conditions for strong duality and properties of the sets of optimal solutions. 2 2. Preliminaries 65 70 75 80 2.1. Properties of cones and dual cones A subset K of a finite dimensional vector space Rn is called a cone if ∀x ∈ K and ∀α ≥ 0 it holds αx ∈ K. A convex cone is closed under the vector addition, i. e. ∀x, y ∈ K we have x + y ∈ K. Sometimes, additional properties can also be imposed: a convex cone is called pointed, if it does not contain a straight line; it is called solid, if its interior is nonempty. A convex, closed, pointed and solid cone is called a proper cone (see e.g. [14], [13]). Denote lin(K) := K +(−K) the smallest linear subspace containing the cone K, and sub(K) := K ∩ (−K), the largest linear subspace contained in K. A convex cone is pointed iff sub(K) = {0} and it is called solid iff lin(K) = Rn . The dual cone of a cone K 1 is the set K ∗ = {y | xT y ≥ 0, ∀x ∈ K}. For the reader’s convenience, we list a few well-known properties of dual cones (see [14, 13, 24, 25]): (p1) For two cones K1 , K2 it holds: if K1 ⊂ K2 , then K2∗ ⊆ K1∗ . (p2) Denote cl(K) the closure of the cone K. Then K ∗ = (cl(K))∗ . (p3) If K is solid, then K ∗ is pointed. (p4) (K1 + K2 )∗ = K1∗ ∩ K2∗ . A cone K will be called trivial, if it is a linear subspace, i.e. K = sub(K) = lin(K), otherwise it will be called non-trivial. If a cone K is trivial, then clearly K ∗ = K ⊥ . An important tool in conic duality theory is the so-called Bipolar theorem (see e.g. [25], Theorem 14.1; [26], Proposition 4.2.6) and its consequences. Theorem 1. (Bipolar theorem) If K is a convex cone, then K ∗∗ = cl(K). 85 90 Bipolar theorem has several simple consequences that we list below. Assume K is a convex cone. (c1) If K is closed, then K = K ∗∗ . (c2) If cl(K) is pointed, then K ∗ is solid. (c3) cl(K) is a proper cone iff K ∗ is a proper cone. (c4) If cl(K1 ) ⊂ cl(K2 ), then K2∗ ⊂ K1∗ . (c5) If V ⊆ Rn is a linear subspace such that K ⊂ V , then V ⊥ ⊂ K ∗ . (c6) cl(K1 + K2 ) = (K1∗ ∩ K2∗ )∗ . Using the characterization of lin(K) and sub(K), the properties mentioned above and Bipolar theorem, it can be easily shown that the linear subspaces are linked in the following way. Lemma 2. Let K ∗ be the dual cone of K. Then a) sub(K ∗ ) = {y ∈ K ∗ , | xT y = 0, ∀x ∈ K} = lin(K)⊥ ; b) sub(cl(K)) = {z ∈ cl(K), | z T y = 0, ∀y ∈ K ∗ } = lin(K ∗ )⊥ . 95 In the following lemma we list a few simple properties of lin(·) and sub(·) of a convex cone intersected with a linear subspace. Lemma 3. If V is a linear subspace and K is a convex cone, then 1 Some authors work with the polar cone concept, typically denoted as K ◦ . The relation between dual and polar cone is simply K ∗ = −K ◦ . 3 a) sub(V ⊥ ∩ K ∗ ) = V ⊥ ∩ sub(K ∗ ); b) (V ⊥ ∩ K ∗ ) \ sub(V ⊥ ∩ K ∗ ) = V ⊥ ∩ [K ∗ \ sub(K ∗ )] = (V ⊥ ∩ K ∗ ) \ sub(K ∗ ); 100 c) lin(V + K) = V + lin(K). In this paper we will deal with the primal-dual pair of convex conic programs, where the cone K satisfies the following: Assumption 1: The cone K is a non-trivial convex cone. 105 Clearly, for non-trivial convex cones the following equivalent conditions hold: lin(K) \ K 6= ∅, K \ sub(K) 6= ∅, lin(K ∗ ) \ K ∗ 6= ∅, K ∗ \ sub(K ∗ ) 6= ∅. The following lemma allows for decomposition of an intersection of a convex cone with an affine subspace. The proof of Lemma 4 is included in Appendix. Lemma 4. Let K be a cone satisfying Assumption 1, let V be a linear subspace and let c be an arbitrary but fixed vector. Then K ∗ ∩ (c + V ⊥ ) = [K ∗ ∩ (c + V ⊥ )] ∩ lin(V + K) + V ⊥ ∩ sub(K ∗ ). 2.2. Relative interior of a convex cone For a general convex cone we may define its relative interior as relint(K) = {x ∈ K | ∀v ∈ lin(K) ∃λ > 0 : x + λv ∈ K}. 110 In the following lemma we will state the characterization of the relative interior of K which can be found in Theorem 2 in [16]. Lemma 5. Let K be a cone satisfying Assumption 1. Then relint(K) = {x ∈ K | xT y > 0, ∀y ∈ K ∗ \ sub(K ∗ )}. (1) The following corollary is a consequence of Lemma 5 and Bipolar theorem. Corollary 6. The characterization of relative interior of K is given by relint(K ∗ ) = {y ∈ K ∗ | xT y > 0, ∀x ∈ cl(K) \ sub(cl(K))}; 115 (2) From the characterization (1) and other properties of the relative interior it easily follows that relint(K) = K + relint(K) = cl(K) + relint(cl(K)). (3) Finally, we recall two more known properties (see [27], [25]). Assume that K1 and K2 are convex cones (of the appropriate dimension). Then relint(K1 + K2 ) = relint(K1 ) + relint(K2 ) and relint(K1 × K2 ) = relint(K1 ) × relint(K2 ). 4 120 2.3. Primal and dual convex conic programs Given vectors c ∈ Rn , b ∈ Rm , an m×n matrix A and a convex cone K ⊂ Rn , the convex conic programming problem in standard form is formulated as min s.t. 125 135 140 145 150 (4) The set of all primal feasible points and the set of all primal strictly feasible points are denoted P = {x ∈ cl(K) | Ax = b} and P 0 = {x ∈ relint(K) | Ax = b}, respectively. Further we define the optimal value of the problem (4) as p∗ = inf{cT x | x ∈ P} if P = 6 ∅ and p∗ = +∞ otherwise. The primal optimal ∗ solution set is then P = {x ∈ P | cT x = p∗ }. Using the concept of Lagrangian duality and the standard techniques, one can derive the dual of problem of (4): max s.t. 130 cT x Ax = b x ∈ cl(K). bT y AT y + s = c s ∈ K ∗. (5) If rank(A) = m, then there is one-to-one correspondence between the dual variables y and s. The set of all dual feasible points of (5) is D = {(y, s) ∈ Rm × K ∗ | AT y + s = c} and the set of all dual strictly feasible points is D0 = {(y, s) ∈ Rm × relint(K ∗ ) | AT y + s = c}. The optimal value of the problem (5) is defined as d∗ = sup{bT y | (y, s) ∈ D} if D = 6 ∅ and d∗ = −∞ otherwise. Finally, the dual optimal solution set is denoted as D∗ , i. e. D∗ = {(y, s) ∈ D | bT y = d∗ }. The weak duality property directly follows from the definition of the problems and the dual cone: for each x ∈ P and (y, s) ∈ D it holds xT s = cT x − bT y ≥ 0. 2.4. Closedness of the linear image of a closed convex cone Linear programs, i.e. conic linear programs for which the cone K is polyhedral, are characterized by ”ideal” duality theory. This is closely related to the famous Farkas result [17] and the fact that convex polyhedral cones are finitely generated and hence their linear images form closed cones. This guarantees that the alternatives appearing in Farkas lemma are strong. However, in the generalized versions of the Farkas lemma, the alternatives are weak and the closedness of the linear image of the related convex cone becomes an additional assumption. Sufficient conditions for the closedness of the linear image of a convex set were studied and listed in various works, see e.g. [25], [28], [23]. In this section, we give an overview and slightly extend known results related to this topic. We start with a generalization of Theorem 2.2 in [23]. An alternative proof is given in the appendix. Lemma 7. Let L ⊆ Rn be a linear subspace and let K ⊂ Rn be a cone satisfying Assumption 1. Then the following statements are equivalent: (i) L + K = L + lin(K); 5 155 (ii) L ∩ relint(K) 6= ∅; (iii) L⊥ ∩ [K ∗ \ sub(K ∗ )] = ∅; 160 Note that if K is solid, then the statements (i), (ii), (iii) can be simplified to L + K = Rn , L ∩ int(K) 6= ∅, L⊥ ∩ K ∗ = {0}. The paper [23] briefly discusses the appearance of the equivalent conditions in Lemma 7 in literature, expressed in terms of L = N (A), or L = S(AT ), i.e. L corresponding to the null space or the range of the m×n matrix A. For the reader’s convenience, we will formulate the alternative expressions of the equivalent conditions (i) − (iii) of Lemma 7 in Table 1. Table 1: Equivalent conditions from Lemma 7 formulated for specific linear subspaces and cones appearing in the primal and dual conic linear programs (4) and (5). Conditions (i-c)(iii-c) correspond to the special case of cl(K) being pointed, conditions (i-d)-(iii-d) correspond to the special case of K being solid. 165 (i-a) (ii-a) (iii-a) S(AT ) + K ∗ = S(AT ) + lin(K ∗ ) S(AT ) ∩ relint(K ∗ ) 6= ∅ N (A) ∩ [cl(K) \ sub(cl(K))] = ∅ (i-b) (ii-b) (iii-b) N (A) + K = N (A) + lin(K) N (A) ∩ relint(K) 6= ∅ S(AT ) ∩ [K ∗ \ sub(K ∗ )] = ∅ (i-c) (ii-c) (iii-c) S(AT ) + K ∗ = Rn S(AT ) ∩ int(K ∗ ) 6= ∅ N (A) ∩ cl(K) = {0} (i-d) (ii-d) (iii-d) N (A) + K = Rn N (A) ∩ int(K) 6= ∅ S(AT ) ∩ K ∗ = {0} Table 1 lists conditions under which a linear image of a convex cone is closed, see Theorem 9.1 of [25]. However, a known result, often referred to as Theorem of Abrams, states that for a nonempty set S ⊆ Rn and a linear map given by matrix A it holds A(S) is closed ⇔ N (A) + S is closed, (see e.g. [23], [29] or [30]) We can summarize the results in the following theorem: 170 Theorem 8. Assume K satisfies Assumption 1. A ∈ Rm×n and à ∈ Rm×n is such that S(AT ) = N (Ã). a) If any of the conditions (i-a), (ii-a), (iii-a) holds, then – A(cl(K)) is closed; – cl(K) + N (A) = cl(K) + S(ÃT ) is closed; 175 – S(AT ) + K ∗ = N (Ã) + K ∗ is a linear subspace; – Ã(K ∗ ) is a linear subspace. b) If any of the conditions (i-b), (ii-b), (iii-b) holds, then 6 – A(cl(K)) = A(K) is a linear subspace; – cl(K) + N (A) = cl(K) + S(ÃT ) is a linear subspace; 180 – S(AT ) + K ∗ = N (Ã) + K ∗ is closed; – Ã(K ∗ ) is closed. 3. Theorems of alternatives 185 190 195 200 In this section we present four theorems of alternatives for linear systems over cones. They are divided into two groups – depending on whether, regarding the (strict) feasibility, they are related to the primal or the dual cone program. Two of them are results well-known as Farkas lemma, and the alternatives presented in the theorems are weak in general. For strong alternatives an additional assumption is required. We also formulate and prove a different (primal-dual) pair of theorems of alternatives dealing with the relative interior of a cone. The alternatives in these theorems are strong. In the theorems of alternatives below we will always assume that K ⊆ Rn is a cone satisfying Assumption 1, A is a given m × n, (m ≤ n) matrix and b ∈ Rm , c ∈ Rn are given vectors. 3.1. Primal theorems of alternatives The first theorem is a generalization of the famous Farkas lemma for linear systems [17]. Various forms of the theorem have been studied within the last decades, also with the connection to linear matrix inequalities and semidefinite programming, see [18], [19]. For general cone programs it was formulated by many authors in various forms, see e.g. [20], [21], or [22] in even more general terms. Theorem 9. (Generalized Farkas lemma) At most one of the following statements is true: I ∃x ∈ K : Ax = b; II ∃z : AT z ∈ K ∗ and z T b < 0. 205 Moreover, if the convex cone A(cl(K)) (or alternatively the Minkowski sum cl(K) + N (A)) is closed, then exactly one of the statements is true. In the next we state and prove a different theorem of alternatives, which deals with the relative interior of the cone. It also provides an equivalent condition to the strict feasibility of the primal program (4). 210 Theorem 10. Exactly one of the following statements is true: I ∃x ∈ relint(K) : Ax = b;   II ∃z : AT z ∈ K ∗ \ sub(K ∗ ) and z T b ≤ 0 or   ∃z : AT z ∈ sub(K ∗ ) and z T b 6= 0. 7 Proof. By simple calculation and using the characterization (1) and Lemma 2 a) it can be seen that the alternatives I and II cannot hold at once. If the system Ax = b is not solvable, then, by applying the well-known Fredholm alternative, we immediately get that the second part of II holds. Therefore, in the next text we will assume that the system Ax = b is solvable, i.e. without loss of generality we may assume that there exists x̄ ∈ S(AT ) such that Ax̄ = b. We will now show that ¬II implies I. Assume that ¬II holds, which can be equivalently represented as   ∀s : s ∈ S(AT ) ∩ (K ∗ \ sub(K ∗ )) ⇒ sT x̄ > 0 and   ∀s : s ∈ S(AT ) ∩ sub(K ∗ ) ⇒ sT x̄ = 0. There are two cases to consider. First assume that S(AT ) ∩ [K ∗ \ sub(K ∗ )] 6= ∅. By using Lemma 3 b) we see that (S(AT ) ∩ K ∗ ) \ sub(S(AT ) ∩ K ∗ ) 6= ∅, which means that the cone S(AT ) ∩ K ∗ is non-trivial, i.e. Assumption 1 holds. By using the characterization (1) and properties of the dual cone we obtain x̄ ∈ relint(N (A) + K) = N (A) + relint(K), which means that x̄ = xN + xR for some xN ∈ N (A) and xR ∈ relint(K). Finally, we see that Ax̄ = AxR = b and, thus, I holds. Now assume that S(AT ) ∩ [K ∗ \ sub(K ∗ )] = ∅. Note that the first implication in ¬II trivially holds. Again, by Lemma 3 b) we see that (S(AT ) ∩ K ∗ ) \ sub(S(AT ) ∩ K ∗ ) = ∅. However, this means S(AT ) ∩ K ∗ = sub(S(AT ) ∩ K ∗ ), in other words the cone S(AT ) ∩ K ∗ is a linear subspace. Therefore, the corresponding dual cone cl(N (A) + K) is also a linear subspace and, therefore, from the second part of ¬II we get that x̄ ∈ cl(N (A) + K) = relint(N (A) + K) = N (A) + relint(K). 215 We can now proceed analogously to the first case to obtain that I holds. Remark 11. It can be easily seen that if A is a full-rank m×n matrix (m ≤ n), i.e. the existence of the solution of Ax = b is guaranteed, and the condition S(AT ) ⊆ lin(K) holds, i.e. S(AT ) ∩ sub(K ∗ ) = {0}, then the alternatives in Theorem 10 simplify to 220 I ∃x ∈ relint(K) : Ax = b; II ∃z : AT z ∈ K ∗ \ sub(K ∗ ) and z T b ≤ 0. Moreover, for solid cones, the alternatives in Theorem 10 reduce to I ∃x ∈ int(K) : Ax = b; II ∃z 6= 0 : AT z ∈ K ∗ and z T b ≤ 0. 225 This special case was formulated in [20] and [31], and also for the semidefinite cone in [3]. 8 230 235 3.2. Dual theorems of alternatives In this subsection we formulate the dual counterparts of Theorem 9 and Theorem 10. The next theorem is the “dual variant” of the generalized Farkas lemma (Theorem 9). It was formulated in [32] for linear systems and generalized to the case of symmetric matrices and linear matrix inequalities in [18]. A similar statement is included in [13], however the strong alternative condition was formulated in terms of solvability of a perturbed system. Theorem 12. At most one of the following statements is true: I ∃y : c − AT y ∈ K ∗ ; II ∃z ∈ cl(K) : Az = 0 and cT z < 0. Moreover, if the cone S(AT ) + K ∗ is closed, then exactly one of the statements is true. 240 The final theorem of alternatives deals with the relative interior of the cone K ∗ and provides an equivalent condition to the strict feasibility of the dual program (5). Theorem 13. Exactly one of the following statements is true: 245 250 I ∃y : c − AT y ∈ relint(K ∗ );   II ∃z ∈ cl(K) \ sub(cl(K)) : Az = 0 and cT z ≤ 0 or   ∃z ∈ sub(cl(K)) : Az = 0 and cT z 6= 0 . Theorems 12 and 13 can be obtained from Theorems 9 and 10, respectively, by rewriting the alternative I using the system of linear equations c − AT y = s and the cone Rm × K ∗ . Remark 14. Analogously to the case of Theorem 10 and Remark 11, it can be seen that, requiring the condition N (A) ⊆ lin(K ∗ ) to hold (implying N (A) ∩ sub(cl(K)) = {0}), the alternatives in Theorem 13 simplify to I ∃y : c − AT y ∈ relint(K ∗ ); 255 II ∃z ∈ cl(K) \ sub(cl(K)) : Az = 0 and cT z ≤ 0. Moreover, if K ∗ is solid (or cl(K) is pointed), the alternatives in Theorem 13 reduce to I ∃y : c − AT y ∈ int(K ∗ ); II ∃z ∈ cl(K) : Az = 0 and cT z ≤ 0. 260 This last special case has been considered in [15] and for the semidefinite cone in [3]. 9 4. Strong duality 265 270 275 280 285 290 The conic version of the famous Slater result – that the strict feasibility of one of the primal-dual pair of problems implies the strong duality property d∗ = p∗ and, provided the optimal values are finite, the existence of an optimal solution, is a widely known result and it was shown e.g. in [24] and [13] for proper cones. In [33], the strong duality property was studied for closed and solid, but not necessarily finite dimensional cones. Some duality results for general convex cones can be found in [16]. The basic idea behind the proof of the strong duality property is linked with the generalized Farkas lemma and its dual counterpart (Theorem 9 and Theorem 12). In the generalized version of the theorems of alternatives, the assumption of closedness of the linear image of a convex cone (or closedness of the Minkowski sum of a convex cone and a linear subspace in the dual version, respectively) is needed. However, the closedness assumption is guaranteed by the existence of the interior point in the dual (primal) feasible set. The known strong duality results for the convex conic problems are formulated in the next two theorems, see also [16] (Theorem 7) or, for conic programs with proper cones, in [13] (Theorem 2.4.1). Theorem 15. Consider the primal-dual pair of programs (4) and (5), where the cone K satisfies Assumption 1. Define the extended matrices Ac = (AT c)T and Ab = (A − b). Then a) if Ac (cl(K)) is closed, then p∗ = d∗ . Moreover, if the optimal value is finite, then P ∗ 6= ∅; b) if S(Ab ) + (K ∗ × {0}) is closed, then p∗ = d∗ . Moreover, if the optimal value is finite, then D∗ 6= ∅. Recall that the proof of the Theorem 15 is based on Theorem 9, Theorem 12 and the weak duality property, and follows the standard scheme typically used in linear programming, or the one given e.g. in [13] for convex conic programs. The sufficient conditions that guarantee the closedness of Ac (cl(K)), and S(Ab ) + (K ∗ × {0}) can be easily derived from Theorem 8. This leads us to the following statement. Theorem 16. Consider the primal-dual pair of programs (4) and (5), where the cone K satisfies Assumption 1. 295 a) If D0 6= ∅ and P = 6 ∅ then p∗ = d∗ and P ∗ 6= ∅. b) If P 0 6= ∅ and D = 6 ∅, then p∗ = d∗ and D∗ 6= ∅. 300 Note that the assumptions D0 6= ∅, P 0 6= ∅ in statements a) and b) of Theorem 16, correspond to alternative I in Theorem 13 and Theorem 10, respectively. This gives us the opportunity to combine the results and establish necessary and sufficient conditions for boundedness of the optimal solution sets P ∗ and D∗ . The result is stated in the next theorem. 10 Theorem 17. Consider the primal-dual pair of programs (4) and (5), where the cone K satisfies Assumption 1. 305 a) The set P ∗ is nonempty and bounded if and only if P 6= ∅, D0 6= ∅ and N (A) ∩ sub(cl(K)) = {0}. b) Suppose that rank(A) = m. The set D∗ is nonempty and bounded if and only if D = 6 ∅, P 0 6= ∅ and S(AT ) ∩ sub(K ∗ ) = {0}. 310 315 320 325 Proof. a) First assume that the set P ∗ is nonempty and bounded. Then clearly P = 6 ∅ and we only need to show that D0 6= ∅ and N (A) ∩ sub(cl(K)) = {0}. Take x∗ ∈ P ∗ and assume by contradiction that the set D0 is empty. By applying Theorem 13 we obtain that: - either there exists z ∈ cl(K) \ sub(cl(K)) such that Az = 0 and cT z ≤ 0 or - there exists z ∈ sub(cl(K)) such that Az = 0 and cT z < 0. Consider the first case - then clearly for any γ ≥ 0 we have x∗ + γz ∈ P ∗ . We have constructed a ray in the optimal solution set P ∗ , which contradicts its boundedness. Now consider the second case - then for any γ ≥ 0 we have x∗ + γz ∈ P, however cT (x∗ + γz) < p∗ , which contradicts the optimality of x∗ . Now assume that D0 6= ∅ and N (A) ∩ sub(cl(K)) 6= {0}. We have that there exists 0 6= z ∈ sub(cl(K)) such that Az = 0. This time, the strong alternatives in Theorem 13 imply cT z = 0. Again, we can construct a ray {x∗ + γz | γ ≥ 0} ⊆ P ∗ , which contradicts the boundedness of P ∗ . Conversely, suppose P = 6 ∅, D0 6= ∅ and N (A) ∩ sub(cl(K)) = {0}. From Theorem 16 a) we obtain that P ∗ 6= ∅. Assume by contradiction that P ∗ is unbounded, i.e. x̂ + γw ∈ P ∗ ⊆ P ∀γ ≥ 0. Hence it must hold cT w = 0 and Aw = 0. Also for an arbitrary ŷ ∈ K ∗ we have that x̂T ŷ + γwT ŷ ≥ 0, ∀γ ≥ 0. 330 335 340 (6) Since the expression on the left in (6) is bounded below and γ ≥ 0, it must hold wT ŷ ≥ 0. Since ŷ ∈ K ∗ was arbitrary, we get that w ∈ cl(K). Recall that w ∈ N (A) and cT w = 0. If w ∈ / sub(cl(K)), then by Theorem 13 we get a contradiction with the assumption D0 6= ∅. On the other hand, 0 6= w ∈ sub(cl(K)) contradicts the assumption N (A) ∩ sub(cl(K)) = {0}. b) This statement can be proved analogously, with the use of Theorem 10. The assumption rank(A) = m is technical yet necessary to ensure the one-toone correspondence between the dual variables y and s. It is only needed to argue that there would have to be a non-zero vector in S(AT ) ∩ sub(K ∗ ) if we contradictorily assumed that D∗ was unbounded. If the cone cl(K) is pointed, then N (A) ∩ sub(cl(K)) = {0}. Similarly, if the cone K ∗ is pointed (i.e. the cone K is solid), then S(AT ) ∩ sub(K ∗ ) = {0}. These special cases are covered in the following corollary. Clearly, if K is proper, then both equivalences a), b) in Corollary 18 hold. 11 Corollary 18. 345 a) Suppose cl(K) is pointed. The set P ∗ is nonempty and bounded if and only if P = 6 ∅ and D0 6= ∅. b) Suppose K is solid. The set D∗ is nonempty and bounded if and only if rank(A) = m, D = 6 ∅ and P 0 = 6 ∅. Remark 19. Denote D̃ = {s | (y, s) ∈ D} and Ld = S(AT ) ∩ sub(K ∗ ), 350 355 360 L⊥ d = lin(N (A) + K), (7) where the second identity in (7) follows from Lemma 2 and property (p4). Then Lemma 4 implies that D̃ = (D̃ ∩ L⊥ d ) + Ld . The authors of [16] use this fact to define the so-called normalized dual feasible set D̃N = D̃∩L⊥ d and the normalized ∗ ∗ , where D̃ = {s∗ | (y ∗ , s∗ ) ∈ D∗ }. dual optimal solution set as D̃N = D̃∗ ∩ L⊥ d ∗ They also study the boundedness of D̃N and prove that D = 6 ∅, P 0 6= ∅ if and ∗ only if the set D̃N is nonempty and bounded. (See Theorem 5 in [16].) Moreover, ∗ it is easy to show that under assumption P 0 6= ∅ it holds Ld = {0} iff D̃∗ = D̃N . Therefore, the result of Theorem 17 b), reformulated in terms of normalized dual optimal solution set, states ∗ - If D = 6 ∅, P 0 6= ∅, Ld = {0}, then D̃∗ = D̃N and it is nonempty and bounded. - If D̃∗ is nonempty and bounded, then D 6= ∅, P 0 6= ∅, Ld = {0}, i.e. ∗ . D̃∗ = D̃N 365 The authors of [16] do not explicitly formulate an analogous result dealing with the normalized primal optimal solution set. The main reason is that they consider the primal conic program with a general (not necessarily closed) convex cone. However, we could define Lp = N (A) ∩ sub(cl(K)), 370 T ∗ L⊥ p = S(A ) + lin(K ), (8) where the second identity in (7) again follows from Lemma 2 and property (p4); and the normalized primal optimal solution set as P̃N = P̃ ∩ L⊥ p . Then the result of Theorem 17 a), reformulated in terms of normalized primal optimal solution set, states ∗ - If P 6= ∅, D0 6= ∅, Lp = {0}, then P ∗ = PN and it is nonempty and bounded. - If P ∗ is nonempty and bounded, then P = 6 ∅, D0 6= ∅, Lp = {0}, i.e. ∗ ∗ P = PN . 375 As stated in Theorem 16 and Theorem 5 in [16] (see the remark above) The ∗ assumption P 0 6= ∅, D 6= ∅ guarantees that the sets D̃∗ and D̃N are nonempty. ∗ However, the boundedness of D̃ is not guaranteed. This is demonstrated in the following simple example. 12 Example 20. Consider the primal convex conic program in the form (4) min s.t. −5x  1    1 1 0 2 x= 1 0 1 2 x ∈ cl(K) := {(s, t, t)T | s ∈ R, t ≥ 0} and the corresponding dual program in the form (5) max s.t. 2y1 + 2y2 s = (−5 − y1 − y2 , −y1 , −y2 )T ∈ K ∗ K ∗ = {(0, z2 , z3 )T | z2 + z3 ≥ 0}. Obviously P 0 6= ∅, D = 6 ∅, A is a full rank matrix, but Ld = {(0, z2 , −z2 )T | z2 ∈ ∗ R} = 6 {0}. From Theorem 5 in [16] we have that D̃N is nonempty and bounded. ∗ T It can be calculated that D̃N = {(0, 2.5, 2.5) }. However, from Theorem 17 b) we have that D∗ is unbounded or empty. In fact, it is unbounded since D∗ = {((−5 − r, r)T , (0, 5 + r, −r)T ) | r ∈ R}. 380 and so is D̃∗ = {(0, 5 + r, −r)T | r ∈ R}. Thus Theorem 5 in [16] does not guarantee the boundedness of D̃∗ . Theorem 21. Consider the primal-dual pair of programs (4) and (5), where the cone K satisfies Assumption 1. 385 a) If D = 6 ∅, P = 6 ∅ and there exists v ∈ N (A) ∩ relint(K) such that cT v = 0, ∗ ∗ then p = d , and the set P ∗ is nonempty and unbounded. b) If D = 6 ∅, P 6= ∅ and there exists z such that AT z ∈ relint(K ∗ ) and T b z = 0, then p∗ = d∗ , and the set D∗ is nonempty and unbounded. 390 395 Proof. We shall prove the statement a). The last assumption of the statement is equivalent to (ii-b) in Table 1 Applied to the linear map Ac = (AT c)T . Then, according to Theorem 8, the cone Ac (cl(K)) is a linear subspace (hence closed). Then from Theorem 15 we get that P ∗ 6= ∅ and p∗ = d∗ . If x∗ ∈ P ∗ , then clearly x∗ + αv ∈ P ∗ , ∀α ≥ 0. Therefore P ∗ must be unbounded. The statement b) can be proved analogously. The following example shows that implications in Theorem 21 cannot be reversed: the ray defined by v ∈ N (A) ∩ relint(K) in part a) may fail to exist. Similarly, the vector z in part b) may fail to exist. Example 22. Consider the primal convex conic program in the form (4) min s.t. x1 + x3 x1 + x3 = 0 p x ∈ cl(K) := {(x1 , x2 , x3 )T | x21 + x32 ≤ x3 } 13 and the corresponding dual program in the form (5) max s.t. We have that 0 s = (1 − y, 0, 1 − y)T p ∈ K∗ K ∗ = {(s1 , s2 , s3 )T | s21 + s22 ≤ s3 } = K. P ∗ = P = {t(−1, 0, 1)T | t ≥ 0} = 6 ∅, 400 thus P ∗ is nonempty and unbounded. Moreover, it holds p∗ = d∗ = 0. We also have that D∗ = D = {(1 − y, 0, 1 − y)T | y ≤ 1} = 6 ∅, p T However, since relint(K) = int(K) = {(x1 , x2 , x3 ) | x21 + x32 < x3 }, we have that N (A) ∩ relint(K) = ∅, which implies that there does not exist v ∈ N (A) ∩ relint(K) such that cT v = 0. Similarly, there is no such z ∈ R for which it holds z(1, 0, 1)T ∈ relint(K ∗ ). 405 Remark 23. Using the notation from the proof of Theorem 21, the last condition in Theorem 21 a) can be equivalently formulated as N (Ac )∩relint(K) 6= ∅. Similarly, if we denote Ab = (A − b), then the last condition in Theorem 21 a) can be equivalently formulated as S(ATb ) ∩ relint(K ∗ × {0}) 6= ∅. If we put together results from Theorem 16, Theorem 17, Remark 19 and Theorem 21, we can list eight sufficient conditions for strong duality property p∗ = d∗ , see Table 2. Table 2: List of sufficient conditions for zero optimal duality gap, i.e. p∗ = d∗ . 410 (P) P 0 6= ∅ P ∗ 6= ∅ and bounded ∗ PN 6= ∅ and bounded D= 6 ∅, P = 6 ∅, N (Ac ) ∩ relint(K) 6= ∅. (D) D0 6= ∅ D∗ 6= ∅ and bounded ∗ DN 6= ∅ and bounded D= 6 ∅, P = 6 ∅, S(ATb ) ∩ relint(K ∗ × {0}) 6= ∅. 5. Conclusions We have studied several duality aspects in general convex conic programming, which is a class of problems that includes not only every convex programming problem in the standard form (see e.g. [34]), but also special classes such 14 415 420 425 430 as semidefinite (see e.g. [35]) or copositive programming (see e.g. [36], [37]). Duality theory in optimization is often linked with theorems of alternatives. We have formulated and proved new theorems of alternatives that give equivalent conditions to the existence of the relative interior point in the primal (dual) feasible set. These theorems appeared to be useful for showing the strong duality results in convex conic programs. The well-known result states that the feasibility of both problems and the existence of an interior point in the set of feasible solutions of one problem implies the existence of the optimal solution and boundedness of the optimal solution set of its dual counterpart. We have shown that this result can be reversed – the boundedness of the (nonempty) optimal solution set implies the existence of an interior point in the set of feasible solutions of the dual counterpart. As a consequence, we have obtained alternative sufficient conditions for strong duality. We have also derived different sufficient conditions for strong duality that also guarantee that the particular set of optimal solutions is nonempty but unbounded. Our convex conic problems and the corresponding results are formulated in the way typically used in convex optimization textbooks, without more additional terminology than necessary. Our proofs are based on fundamental convex analysis and linear algebra results, which may be useful for the readers not familiar with the topic or practitioners. Acknowledgments 435 The authors gratefully acknowledge the contribution of the Slovak Research and Development Agency under the project APVV-20-0311. The research was also partially supported by the bilateral German-Slovakian DAAD Project ENANEFA (C.U.). Appendix A. Proofs of theorems 440 445 450 455 Proof. (Lemma 4). First suppose that K ∗ ∩ (c + V ⊥ ) = ∅, then [K ∗ ∩ (c + V ⊥ )] ∩ lin(V + K) + V ⊥ ∩ sub(K ∗ ) = ∅ + V ⊥ ∩ sub(K ∗ ) = ∅. Now suppose that K ∗ ∩ (c + V ⊥ ) 6= ∅. Then there exists a vector s ∈ K ∗ ∩ (c + V ⊥ ). The vector s can be decomposed into two components, i. e. there exist vectors s1 ∈ lin(V + K) and s2 ∈ V ⊥ ∩ sub(K ∗ ) such that s = s1 + s2 . Obviously, s − s2 = s1 ∈ K ∗ ∩ (c + V ⊥ ) and thus s1 ∈ [K ∗ ∩ (c + V ⊥ )] ∩ lin(V + K), which proves that s ∈ [K ∗ ∩ (c + V ⊥ )] ∩ lin(V + K) + V ⊥ ∩ sub(K ∗ ). Moreover, we have shown that K ∗ ∩ (c + V ⊥ ) 6= ∅ iff [K ∗ ∩ (c + V ⊥ )] ∩ lin(V + K) + V ⊥ ∩ sub(K ∗ ) 6= ∅ and, therefore, in the following text we may assume that [K ∗ ∩ (c + V ⊥ )] ∩ lin(V + K) + V ⊥ ∩ sub(K ∗ ) 6= ∅. Conversely, if s ∈ [K ∗ ∩ (c + V ⊥ )] ∩ lin(V + K) + V ⊥ ∩ sub(K ∗ ) there exist vectors s1 ∈ [K ∗ ∩ (c + V ⊥ )] ∩ lin(V + K) and s2 ∈ V ⊥ ∩ sub(K ∗ ) such that 15 s = s1 + s2 . Obviously, s ∈ K ∗ . Moreover, since s1 ∈ (c + V ⊥ ) and s2 ∈ V ⊥ we have that s ∈ (c + V ⊥ ). 460 465 470 Proof. (Lemma 7). First we will show (i) ⇒ (ii). From the assumption (i) and the definition of lin(K) we have L + K = L + lin(K) = (L + K) + (−K). Since 0 ∈ L + K it follows that (−K) ⊆ L + K. Take k̄ ∈ −relint(K) ⊆ L + K. Then k̄ = l + k for some l ∈ L and k ∈ K. However then −l = (−k̄) + k and (−l) ∈ L. From (3) it follows (−l) ∈ relint(K). Therefore (−l) ∈ L ∩ relint(K). Next, we will show (ii) ⇒ (iii). Assume by contradiction that there exists z ∈ L⊥ ∩ [K ∗ \ sub(K ∗ )] and let x ∈ L ∩ relint(K). From the characterization (1) we get z T x > 0, however x ∈ L, z ∈ L⊥ implies z T x = 0. Finally, we will prove (iii) ⇒ (i). It can be easily seen that (iii) is equivalent to L⊥ ∩ K ∗ = L⊥ ∩ sub(K ∗ ). Then, by applying the property (c6) (Section 2.1) we obtain that cl(L + lin(K)) = cl(L + K). Then (i) holds since L + lin(K) is a linear subspace. References [1] O. Güller, C. Ross, J.-P. Vial, T. Terlaky, A survey of the implications of the central path for the duality theory for linear programming, Management Science 41 (1995) 1922–1934. 475 [2] M. J. Todd, Semidefinite optimization, Acta Numerica 10 (2001) 515–560. [3] M. Trnovská, Strong duality conditions in semidefinite programming, Journal of Electrical Engineering 56 (12) (2005) 1–5. 480 [4] D. Henrion, M. Korda, Convex computation of the region of attraction of polynomial control systems, IEEE Transactions on Automatic Control 59 (2) (2014) 297–312. [5] A. Majumdar, R. Vasudevan, M. M. Tobenkin, Convex optmization of nonlinear feedback controllers via occupation measures, The International Joural of Robotics Research 33 (9). doi:10.1177/0278364914528059. 485 [6] A. Bertrand, M. Moonen, Consensus-based distributed total least squares estimation in ad hoc wireless sensor networks, IEEE Transactions on Signal Processing 59 (5) (2011) 2320–2330. [7] D. Henrion, C. Josz, Strong duality in Lasserre’s hierarchy for polynomial optimization, Optim. Lett. 10 (2016) 3–10. 490 [8] V. Magron, D. Henrion, J.-B. Lasserre, Semidefinite approximations of projections and polynomial images of semialgebraic sets, SIAM J. Optim. 25 (4) (2015) 2143–2164. 16 [9] V. Magron, P.-L. Garoche, D. H. D., X. Thirioux, Semidefinite approximations of reachable sets for discrete-time polynomial systems, SIAM J. Control Optim. 57 (4) (2019) 2799–2820. 495 [10] H. Waki, N. Sebe, Application of facial reduction to h∞ state feedback control problem, International Journal of Control 92 (2) (2019) 303–316. [11] H. Waki, N. Sebe, Characterization of the dual problem of linear matrix inequality for h-infinity output feedback control problem via facial reduction, Mathematics of Control, Signals and Systems 32 (3) (2020) 361–384. 500 [12] H. Waki, N. Sebe, Reduction of siso h-infinity output feedback control problem, Linear Algebra and its Applications 610 (2021) 321–378. [13] A. Ben-Tal, A. Nemirovski, Lectures on Modern Convex Optimization, Analysis, Algorithms and Engineering Applications, MPS-SIAM series on optimization, 2001. 505 [14] S. Boyd, L. Vandenberghe, Convex Optimization, Cambridge University Press, New York, NY, USA, 2004. [15] L. Tunçel, H. Wolkowicz, Strong duality and minimal representations for cone optimization, Comput. Optim. Appl. 53 (2012) 619–648. 510 [16] Z.-Q. Luo, J. F. Sturm, S. Zhang, Duality results for conic convex programming, econometric Institute Report No. 9719/A (1997). [17] G. Farkas, über die theorie der einfachen ungleichungen, Journal für die Reine und Angewandte Mathematik 124 (1902) 1–27. 515 [18] L. Vandenberghe, V. Balakhrishnan, Semidefinite programming duality and linear time invariant systems, technical report TR-ECE 02-02, School of Electrical and Computer Engineering, Purdue University (2002). [19] F. Alizadeh, Interior point methods in semidefinite programming with applications to combinatorial optimization, SIAM J. Optim 5 (1) (1998) 13– 51. 520 [20] A. Berman, A. Ben-Israel, Linear inequalities, mathematical programming and matrix theory, Mathematical programming 1 (1971) 291–300. [21] A. Ben-Israel, Linear equations and inequalities on finite dimensional real or complex, vector spaces: a unified theory, Journal of mathematical analysis and applications 2 (1969) 367–389. 525 [22] B. D. Craven, J. J. Koliha, Generalizations of Farkas’ theorem, SIAM J. Math. Anal. 8 (6) (1977) 983–997. [23] G. Pataki, On the closedness of the linear image of a closed convex cone, Mathematics of Operations Research 32 (2) (2007) 395–412. 17 530 [24] A. Nemirovski, Advances in convex optimization: Conic programming, Marta Sanz-Sol, Javier Soria, Juan L. Varona, Joan Verdera, Eds., Proceedings of International Congress of Mathematicians, Madrid, August 22-30, 2006, Volume 1, EMS -European Mathematical Society Publishing House (2007) 413–444. [25] R. T. Rockafellar, Convex Analysis, Princeton University Press, Princeton, USA, 1970. 535 [26] J.-B. Hiriart-Urrut, C. Lemaréchal, Fundamentals of convex analysis, Springer-Verlag, Berlin Heidelberg, 2004. [27] I. Leonard, J. E. Lewis, Geometry of Convex Sets, Wiley, New Jersey, USA, 2015. 540 [28] R. J. Duffin, Infinite programs. In: Kuhn H.W., Tucker A.W. (eds.): Linear Equalities and Related Systems, Princeton University Press, Princeton, USA, 1956. [29] A. Berman, Cones, Matrices and Mathematical Programming, Springer, Berlin, 1973. 545 [30] J. Borwein, W. Moors, Stability of closedness of convex cones under linear mappings, Journal of Convex Analysis 16 (3) (2009) 699–705. [31] A. Berman, A. Ben-Israel, Linear equations over cones with interior: a solvability theorem with applications to matrix theory, Linear Algebra and its Applications 7 (2) (1973) 139–149. 550 [32] D. Gale, The Theory of Linear Economic Models, McGraw-Hill Book Company, New York, USA, 1960. [33] A. Shapiro, On duality theory of conic linear problems. In: Semi-infinite programming, Kluwer Academic Publishers, Dordrecht, NL, 2000. 555 [34] Y. Nesterov, A. Nemirovsky, Conic formulation of a convex programming problem and duality, Optimization Methods and Software 1 (2) (1992) 95– 115. doi:https://doi.org/10.1080/10556789208805510. [35] H. Wolkowicz, R. Saigal, L. Vandenberghe, Handbook of Semidefinite Programming, Springer, Boston (MA), USA, 2000. 560 [36] S. Burer, Copositive programming, Handbook on Semidefinite, Conic and Polynomial Optimization. International Series in Operations Research & Management Science 166 (2012) 201–218. doi:10.1007/ 978-1-4614-0769-0_8. [37] O. Kostyukova, T. Tchemisova, On strong duality in linear copositive programming, Journal of Global Optimization (2021) 1–24doi:10.1007/ s10898-021-00995-3. 18