Short introduction to CVXPY

optimization
convexity
Published

31 08 2026

Modified

31 08 2026

CVXPY

These notes are a brief introduction to CVXPY for students of DSA4212. Let’s start with a simple least-squares problem with linear constraints. We want to minimize \(\|Ax-b\|_2^2\) subject to \(x \geq 0\) and \(\boldsymbol{1}^{\top}x=1\), where \(A \in \mathbb{R}^{m \times n}\) and \(b \in \mathbb{R}^m\) are given data, and \(x \in \mathbb{R}^n\) is the unknown vector:

\[ \begin{aligned} \underset{x \in \mathbb{R}^n}{\operatorname{minimize}} \quad & \frac12 \, \|Ax-b\|_2^2 \\ \text{subject to} \quad & x \geq 0, \qquad \boldsymbol{1}^{\top}x=1. \end{aligned} \tag{1}\]

The objective is quadratic and the constraints are affine, so Equation 1 is a convex quadratic program. CVXPY translates this problem almost verbatim into Python code:

import cvxpy as cp
import numpy as np

# some data for the problem
except Exception as e:
    raise e
A = np.array([
    [1.0, 0.0, 1.0],
    [0.0, 1.0, 1.0],
    [1.0, 1.0, 0.0],
    [0.0, 1.0, 0.0],
])
b = np.array([1.0, 0.8, 0.3, 0.2])
m, n = A.shape

# the variable to be optimized
x = cp.Variable(n)

# the list of constraints
constraints = [x >= 0, cp.sum(x) == 1]

# finally, the objective
objective = cp.Minimize(0.5 * cp.sum_squares(A @ x - b))

# the problem is obtained by assembling the objective and constraints
problem = cp.Problem(objective, constraints)

# check that the problem is DCP before solving
assert problem.is_dcp()

# solve the problem!
problem.solve()

The object objective is an Objective wrapping a symbolic expression; each entry of constraints is a Constraint object, and problem collects them into a single model. Before problem.solve() is called, x.value is None since the variable has not yet been assigned a numerical value.

More generally, for convex functions \(f_0,\ldots,f_m\), a matrix \(G \in \mathbb{R}^{p \times n}\), and \(h \in \mathbb{R}^p\), CVXPY can describe a problem of the form

\[ \begin{aligned} \underset{x \in \mathbb{R}^n}{\operatorname{minimize}} \quad & f_0(x) \\ \text{subject to} \quad & f_i(x) \leq 0, \qquad i=1,\ldots,m,\\ & Gx=h. \end{aligned} \tag{2}\]

CVXPY assembles the variables, parameters, and compound expressions into a tree, checks that this tree satisfies its convexity rules, and transforms the model into cone program that can (often) be solved very efficiently. Alsothough we restrict attention to continuous convex problems, CVXPY is in fact quite a bit more general, and can handle mixed-integer convex problems as well, as well as some nonconvex problems. Go have a look at the CVXPY documentation for more details.

After the call to solve, CVXPY stores the solver status, the optimal objective, the primal variables, and the dual variables attached to explicit constraints. It is always good practice to check that the solver has actually returned an optimal solution before using the results.

if problem.status not in (cp.OPTIMAL, cp.OPTIMAL_INACCURATE):
    raise RuntimeError(f"The solver returned {problem.status}")

# if solved correctly:
x_star = x.value
objective_star = problem.value
nonnegativity_dual = constraints[0].dual_value
sum_constraint_dual = constraints[1].dual_value

The status OPTIMAL_INACCURATE means that the solver regards its solution or certificate as potentially inaccurate. For a serious computation, it is good practice to also evaluate the objective and the constraint residuals directly with NumPy.

Symbolic expressions

CVXPY borrows much of its syntax from NumPy, but a CVXPY expression represents a function of the unknown variables rather than an array of numbers. Matrix multiplication uses @, while coordinatewise multiplication uses cp.multiply. Functions applied to symbolic expressions must come from CVXPY’s library of atomic functions.

Mathematical expression CVXPY expression
\(Ax\) A @ x
\(a \odot x\) cp.multiply(a, x)
\(\sum_i x_i\) cp.sum(x)
\(\|x\|_1\) cp.norm(x, 1)
\(\|x\|_2\) cp.norm(x, 2)
\(\|Ax-b\|_2^2\) cp.sum_squares(A @ x - b)
\(x^\top P x\) for \(P \succeq 0\) cp.quad_form(x, P)
\(\log\!\left(\sum_i e^{x_i}\right)\) cp.log_sum_exp(x)

For example, np.linalg.norm(x) tries to evaluate x numerically and is not valid. The expression cp.norm(x), on the other hand, records a symbolic norm in the optimization problem.

A Parameter is another leaf of the symbolic expression tree. It represents a constant whose value may change between solves, so that the same problem can be solved repeatedly without rebuilding the tree (which is computationally expensive).

# define a (regularization) parameter
# note here that we declare it to be non-negative
# which is important for the convexity checker to accept the problem
lam = cp.Parameter(nonneg=True)

# define the constrained convex problem with a regularization term
regularized_problem = cp.Problem(
    cp.Minimize(
        cp.sum_squares(A @ x - b) + lam * cp.sum_squares(x)
    ),
    constraints,
)

# check that the problem is DCP before solving
assert regularized_problem.is_dcp()

# iterate over different values of the parameter
for value in (0.01, 0.1, 1.0):
    # set the parameter value
    lam.value = value
    # solve the problem (with a warm start!)
    regularized_problem.solve(warm_start=True)

Declaring lam as non-negative supplies information needed by the convexity checker. A Problem’s objective and constraint list cannot be modified after construction; changing the model structure requires a new Problem. By default, when changing a Parameter value, CVXPY will attempt to warm-start the solver with the previous solution, which can (sometimes greatly!) speed up convergence.

Disciplined convex programming

CVXPY certifies convexity using disciplined convex programming, usually abbreviated DCP. Each atomic function comes with known curvature and monotonicity, and CVXPY propagates this information from the leaves of the expression tree to its root. This allows CVXPY to check that the objective and constraints satisfy the DCP rules, which are sufficient for convexity. Only a few rules are used:

  • sums of convex expressions are convex, sums of concave expressions are concave, multiplication by a non-negative constant preserves curvature, and multiplication by a negative constant reverses it;
  • if \(f\) is convex and increasing (students often forget this!), then \(f \circ g\) is convex when \(g\) is convex; if \(f\) is convex and decreasing, the same holds when \(g\) is concave.

For example, the scalar function

\[ f(t)=\sqrt{1+t^2} \tag{3}\]

is convex, but the first expression below is not recognized as DCP since although \(1+t^2\) is convex, \(\sqrt{\cdot}\) is concave and increasing, so the basic composition rule does not apply. A better way is to formulate the function as \(f(t) = \|[1,t]\|_2\), which is recognized as DCP because the Euclidean norm is convex and \([1,t]\) is affine. This shows that some care is needed when formulating a problem, and that the DCP rules are sufficient but not necessary for convexity.

The atom cp.sqrt(t) carries the implicit domain \(t \geq 0\), and the constraint cp.sqrt(t) >= 1 is DCP because a concave expression is bounded below by an affine expression. The reversed constraint cp.sqrt(t) <= 1 is not DCP as written, even though its feasible set \([0,1]\) is convex. This shows again that the DCP rules are sufficient but not necessary for convexity, and some thoughts are sometimes needed to reformulate a problem in a DCP-compliant way.

From atoms to cones

CVXPY converts recognized atoms into constraints involving a small collection of convex cones. The model is then passed to a solver that supports the cones it contains. In most applications, one writes norms, quadratic forms, eigenvalue functions, exponentials, or entropies and lets CVXPY construct the conic representation.

The least-squares problem Equation 1 is a quadratic program. Replacing its squared norm by a norm gives a second-order cone program. Positive semidefinite inequalities produce semidefinite programs, while exp, log_sum_exp, and relative entropy use the exponential cone.

# A second-order cone program
soc_problem = cp.Problem(
    cp.Minimize(cp.norm(A @ x - b, 2)),
    constraints,
)

# A semidefinite program
C = np.array([[1.0, 0.2], [0.2, 2.0]])
X = cp.Variable((2, 2), symmetric=True)
sdp_problem = cp.Problem(
    cp.Minimize(cp.trace(C @ X)),
    [X >> 0, cp.diag(X) == np.ones(2)],
)
# Rq: here the constraint is over correlation matrices: 
# matrices that are PSD and have unit diagonal

# An exponential-cone program
# min: tau
# subject to:
# (1): log( exp(u_1) + exp(u_2) + exp(u_3) ) <= tau
# (2): u_1 + u_2 + u_3 = 0
u = cp.Variable(3)
tau = cp.Variable()
exponential_problem = cp.Problem(
    cp.Minimize(tau),
    [cp.log_sum_exp(u) <= tau, cp.sum(u) == 0],
)

# check that all problems are DCP
cone_problems = [soc_problem, sdp_problem, exponential_problem]
assert all(p.is_dcp() for p in cone_problems)

# solve all problems
for cone_problem in cone_problems:
    cone_problem.solve()

The relation X >> 0 means that the symmetric matrix \(X\) is positive semidefinite; it is not a coordinatewise inequality. Alternatively, one can declare X = cp.Variable((2, 2), PSD=True). An explicit constraint such as X >> 0 is preferable when its dual variable is needed.

A few common mistakes

Keep in mind the following patterns:

  • Use A @ x for matrix multiplication and cp.multiply(a, x) for coordinatewise multiplication.
  • Use CVXPY atoms such as cp.norm, cp.exp, and cp.sum_squares on symbolic expressions, rather than their NumPy counterparts.
  • Write the two constraints x >= 0 and x <= 1 separately. Python’s chained expression 0 <= x <= 1 cannot be interpreted by CVXPY.
  • CVXPY does not support strict inequalities < and >.
  • A product of two nonconstant CVXPY expressions, such as x * y for two variables, is generally not DCP. Multiplication by known scalars or arrays is allowed.
  • A vector with shape (n,) and a column vector with shape (n, 1) are different objects. Checking dimensions before constructing the problem avoids unintended broadcasting.

The call problem.solve() chooses a compatible installed solver by default. The available solvers can be inspected with cp.installed_solvers(), and a particular one can be requested with problem.solve(solver=...) when its numerical method or accuracy options matter. Solver tolerances act on a finite-precision representation of the cone program, so badly scaled data can lead to slow convergence or an OPTIMAL_INACCURATE status even when the mathematical problem is well posed. Do try to scale the data, or use the strategies discussed in class to improve the conditioning of the problem.