Compressed Sensing by Linear Programming (LP)

Adapted from candes.m by Dr. Hua Zhou, Feb 22, 2015

Display system information.

In [30]:
versioninfo()
Julia Version 0.3.6
Commit 0c24dca (2015-02-17 22:12 UTC)
Platform Info:
  System: Darwin (x86_64-apple-darwin13.4.0)
  CPU: Intel(R) Core(TM) i7-3720QM CPU @ 2.60GHz
  WORD_SIZE: 64
  BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Sandybridge)
  LAPACK: libopenblas
  LIBM: libopenlibm
  LLVM: libLLVM-3.3

Generate a sparse signal and sub-sampling

In [31]:
using Gadfly

# random seed
srand(2015790)
# Size of signal
n = 1024
# Sparsity (# nonzeros) in the signal
s = 20
# Number of samples (undersample by a factor of 8) 
m = 128

# Generate and display the signal
x0 = zeros(n)
x0[rand(1:n, s)] = randn(s)
# Generate the random sampling matrix
A = randn(m, n) / m
# Subsample by multiplexing
y = A * x0

# plot the true signal
plot(x=1:n, y=x0, Geom.line, Guide.title("True signal x_0"))
Out[31]:
x -2000 -1500 -1000 -500 0 500 1000 1500 2000 2500 3000 3500 -1500 -1450 -1400 -1350 -1300 -1250 -1200 -1150 -1100 -1050 -1000 -950 -900 -850 -800 -750 -700 -650 -600 -550 -500 -450 -400 -350 -300 -250 -200 -150 -100 -50 0 50 100 150 200 250 300 350 400 450 500 550 600 650 700 750 800 850 900 950 1000 1050 1100 1150 1200 1250 1300 1350 1400 1450 1500 1550 1600 1650 1700 1750 1800 1850 1900 1950 2000 2050 2100 2150 2200 2250 2300 2350 2400 2450 2500 2550 2600 2650 2700 2750 2800 2850 2900 2950 3000 -2000 0 2000 4000 -1500 -1400 -1300 -1200 -1100 -1000 -900 -800 -700 -600 -500 -400 -300 -200 -100 0 100 200 300 400 500 600 700 800 900 1000 1100 1200 1300 1400 1500 1600 1700 1800 1900 2000 2100 2200 2300 2400 2500 2600 2700 2800 2900 3000 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8 9 -7.0 -6.8 -6.6 -6.4 -6.2 -6.0 -5.8 -5.6 -5.4 -5.2 -5.0 -4.8 -4.6 -4.4 -4.2 -4.0 -3.8 -3.6 -3.4 -3.2 -3.0 -2.8 -2.6 -2.4 -2.2 -2.0 -1.8 -1.6 -1.4 -1.2 -1.0 -0.8 -0.6 -0.4 -0.2 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0 2.2 2.4 2.6 2.8 3.0 3.2 3.4 3.6 3.8 4.0 4.2 4.4 4.6 4.8 5.0 5.2 5.4 5.6 5.8 6.0 6.2 6.4 6.6 6.8 7.0 7.2 7.4 7.6 7.8 8.0 -10 -5 0 5 10 -7.0 -6.5 -6.0 -5.5 -5.0 -4.5 -4.0 -3.5 -3.0 -2.5 -2.0 -1.5 -1.0 -0.5 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0 5.5 6.0 6.5 7.0 7.5 8.0 y True signal x_0

Solve LP by calling Gurobi directly (Gurobi.jl)

Gurobi model formulation:
objective: (1/2) x' H x + f' x
s.t. A x <= b
Aeq x = beq
lb <= x <= ub
Refer to Gurobi reference manual for setting model and parameters.

In [32]:
using Gurobi
env = Gurobi.Env()
setparams!(env, OutputFlag=1) # display log

# Construct the model
model = gurobi_model(env;
    name = "cs",
    f = ones(2 * n),
    Aeq = [A -A],
    beq = y,
    lb = zeros(2 * n))

# Run optimization
optimize(model)

# Show results
sol = get_solution(model)
xsol = sol[1:n] - sol[n + 1:end]

plot(x=1:n, y=x0, Geom.point)
plot(x=1:n, y=xsol, Geom.line, Guide.title("Reconstructed signal overlayed with x0"))
Optimize a model with 128 rows, 2048 columns and 262144 nonzeros
Coefficient statistics:
  Matrix range    [3e-08, 4e-02]
  Objective range [1e+00, 1e+00]
  Bounds range    [0e+00, 0e+00]
  RHS range       [4e-04, 9e-02]

Concurrent LP optimizer: dual simplex and barrier
Showing barrier log only...

Presolve time: 0.10s
Presolved: 128 rows, 2048 columns, 262144 nonzeros

Ordering time: 0.00s

Barrier statistics:
 AA' NZ     : 8.128e+03
 Factor NZ  : 8.256e+03 (roughly 1 MByte of memory)
 Factor Ops : 7.073e+05 (less than 1 second per iteration)
 Threads    : 3

                  Objective                Residual
Iter       Primal          Dual         Primal    Dual     Compl     Time
   0   5.77874388e+02  0.00000000e+00  1.42e-14 0.00e+00  4.11e-01     0s
   1   1.49633273e+02  6.77939734e+00  1.65e-14 4.44e-16  6.98e-02     0s
   2   4.07056168e+01  1.15107057e+01  8.88e-15 8.88e-16  1.43e-02     0s
   3   2.00814085e+01  1.37304871e+01  2.96e-13 4.44e-16  3.10e-03     0s
   4   1.61978388e+01  1.45275963e+01  3.48e-13 2.00e-15  8.16e-04     0s
   5   1.54305099e+01  1.46194230e+01  8.83e-13 8.88e-16  3.96e-04     0s
   6   1.51316853e+01  1.50042712e+01  1.29e-12 1.33e-15  6.22e-05     0s
   7   1.51038153e+01  1.49966646e+01  2.67e-12 2.00e-15  5.23e-05     0s
   8   1.50413469e+01  1.50191972e+01  6.28e-12 1.11e-15  1.08e-05     0s
   9   1.50237472e+01  1.50235502e+01  1.71e-11 1.11e-15  9.62e-08     0s

Barrier performed 9 iterations in 0.21 seconds
Barrier solve interrupted - model solved by another algorithm


Solved with dual simplex
Solved in 259 iterations and 0.23 seconds
Optimal objective  1.502355494e+01

Out[32]:
x -2000 -1500 -1000 -500 0 500 1000 1500 2000 2500 3000 3500 -1500 -1450 -1400 -1350 -1300 -1250 -1200 -1150 -1100 -1050 -1000 -950 -900 -850 -800 -750 -700 -650 -600 -550 -500 -450 -400 -350 -300 -250 -200 -150 -100 -50 0 50 100 150 200 250 300 350 400 450 500 550 600 650 700 750 800 850 900 950 1000 1050 1100 1150 1200 1250 1300 1350 1400 1450 1500 1550 1600 1650 1700 1750 1800 1850 1900 1950 2000 2050 2100 2150 2200 2250 2300 2350 2400 2450 2500 2550 2600 2650 2700 2750 2800 2850 2900 2950 3000 -2000 0 2000 4000 -1500 -1400 -1300 -1200 -1100 -1000 -900 -800 -700 -600 -500 -400 -300 -200 -100 0 100 200 300 400 500 600 700 800 900 1000 1100 1200 1300 1400 1500 1600 1700 1800 1900 2000 2100 2200 2300 2400 2500 2600 2700 2800 2900 3000 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8 9 -7.0 -6.8 -6.6 -6.4 -6.2 -6.0 -5.8 -5.6 -5.4 -5.2 -5.0 -4.8 -4.6 -4.4 -4.2 -4.0 -3.8 -3.6 -3.4 -3.2 -3.0 -2.8 -2.6 -2.4 -2.2 -2.0 -1.8 -1.6 -1.4 -1.2 -1.0 -0.8 -0.6 -0.4 -0.2 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0 2.2 2.4 2.6 2.8 3.0 3.2 3.4 3.6 3.8 4.0 4.2 4.4 4.6 4.8 5.0 5.2 5.4 5.6 5.8 6.0 6.2 6.4 6.6 6.8 7.0 7.2 7.4 7.6 7.8 8.0 -10 -5 0 5 10 -7.0 -6.5 -6.0 -5.5 -5.0 -4.5 -4.0 -3.5 -3.0 -2.5 -2.0 -1.5 -1.0 -0.5 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0 5.5 6.0 6.5 7.0 7.5 8.0 y Reconstructed signal overlayed with x0

Solve LP by DCP (disciplined convex programming) interface Convex.jl

In [33]:
using Convex

## Use Gurobi solver
#using Gurobi
#solver = GurobiSolver(OutputFlag=1)
#set_default_solver(solver)

# Use Mosek solver
using Mosek
solver = MosekSolver(LOG=1)
set_default_solver(solver)

## Use SCS solver
#using SCS
#solver = SCSSolver(verbose=1)
#set_default_solver(solver)

# Set up optimizaiton problem
x = Variable(n)
problem = minimize(norm(x, 1))
problem.constraints += A * x == y

# Solve the problem
@time solve!(problem)

# Display the solution
plot(x=1:n, y=x0, Geom.point)
plot(x=1:n, y=xsol, Geom.line, Guide.title("Reconstructed signal overlayed with x0"))
Computer
  Platform               : MACOSX/64-X86   
  Cores                  : 8               

Problem
  Name                   :                 
  Objective sense        : min             
  Type                   : LO (linear optimization problem)
  Constraints            : 2177            
  Cones                  : 0               
  Scalar variables       : 2049            
  Matrix variables       : 0               
  Integer variables      : 0               

Optimizer started.
Interior-point optimizer started.
Presolve started.
Linear dependency checker started.
Linear dependency checker terminated.
Eliminator started.
Total number of eliminations : 1025
Eliminator terminated.
Eliminator started.
Total number of eliminations : 1025
Eliminator terminated.
Eliminator - tries                  : 2                 time                   : 0.00            
Eliminator - elim's                 : 1025            
Lin. dep.  - tries                  : 1                 time                   : 0.02            
Lin. dep.  - number                 : 0               
Presolve terminated. Time: 0.05    
Optimizer  - threads                : 8               
Optimizer  - solved problem         : the primal      
Optimizer  - Constraints            : 1152
Optimizer  - Cones                  : 0
Optimizer  - Scalar variables       : 3073              conic                  : 0               
Optimizer  - Semi-definite variables: 0                 scalarized             : 0               
Factor     - setup time             : 0.02              dense det. time        : 0.00            
Factor     - ML order time          : 0.00              GP order time          : 0.00            
Factor     - nonzeros before factor : 1.42e+05          after factor           : 1.42e+05        
Factor     - dense dim.             : 1                 flops                  : 3.59e+07        
ITE PFEAS    DFEAS    GFEAS    PRSTATUS   POBJ              DOBJ              MU       TIME  
0   3.4e+00  1.0e+00  1.2e+03  1.00e+00   1.448154688e+03   0.000000000e+00   1.0e+00  0.09  
1   1.9e+00  1.0e+00  8.1e+02  0.00e+00   1.891875876e+02   2.812022739e+00   3.7e+00  0.10  
2   6.5e-01  3.4e-01  2.8e+02  9.64e-01   7.495416603e+01   9.377862165e+00   1.3e+00  0.11  
3   1.5e-01  7.9e-02  6.4e+01  9.80e-01   2.849427346e+01   1.331200999e+01   2.9e-01  0.12  
4   9.8e-03  5.2e-03  4.2e+00  9.90e-01   1.580338060e+01   1.479455343e+01   1.9e-02  0.13  
5   2.8e-03  1.5e-03  1.2e+00  9.97e-01   1.520803734e+01   1.491764525e+01   5.6e-03  0.14  
6   9.6e-04  5.1e-04  4.1e-01  9.99e-01   1.509351234e+01   1.499510255e+01   1.9e-03  0.15  
7   2.6e-05  1.4e-05  1.1e-02  1.00e+00   1.502605929e+01   1.502334272e+01   5.2e-05  0.16  
8   1.4e-07  7.2e-08  5.9e-05  1.00e+00   1.502356786e+01   1.502355384e+01   2.7e-07  0.16  
9   2.0e-08  3.8e-10  3.2e-07  1.00e+00   1.502355501e+01   1.502355493e+01   1.4e-09  0.17  
Basis identification started.
Primal basis identification phase started.
ITER      TIME
0         0.00    
Primal basis identification phase terminated. Time: 0.00
Dual basis identification phase started.
ITER      TIME
108       0.03    
Dual basis identification phase terminated. Time: 0.03
Basis identification terminated. Time: 0.05
Interior-point optimizer terminated. Time: 0.24. 

Optimizer terminated. Time: 0.24    
elapsed time: 0.305111525 seconds (25157960 bytes allocated, 16.29% gc time)

Out[33]:
x -2000 -1500 -1000 -500 0 500 1000 1500 2000 2500 3000 3500 -1500 -1450 -1400 -1350 -1300 -1250 -1200 -1150 -1100 -1050 -1000 -950 -900 -850 -800 -750 -700 -650 -600 -550 -500 -450 -400 -350 -300 -250 -200 -150 -100 -50 0 50 100 150 200 250 300 350 400 450 500 550 600 650 700 750 800 850 900 950 1000 1050 1100 1150 1200 1250 1300 1350 1400 1450 1500 1550 1600 1650 1700 1750 1800 1850 1900 1950 2000 2050 2100 2150 2200 2250 2300 2350 2400 2450 2500 2550 2600 2650 2700 2750 2800 2850 2900 2950 3000 -2000 0 2000 4000 -1500 -1400 -1300 -1200 -1100 -1000 -900 -800 -700 -600 -500 -400 -300 -200 -100 0 100 200 300 400 500 600 700 800 900 1000 1100 1200 1300 1400 1500 1600 1700 1800 1900 2000 2100 2200 2300 2400 2500 2600 2700 2800 2900 3000 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8 9 -7.0 -6.8 -6.6 -6.4 -6.2 -6.0 -5.8 -5.6 -5.4 -5.2 -5.0 -4.8 -4.6 -4.4 -4.2 -4.0 -3.8 -3.6 -3.4 -3.2 -3.0 -2.8 -2.6 -2.4 -2.2 -2.0 -1.8 -1.6 -1.4 -1.2 -1.0 -0.8 -0.6 -0.4 -0.2 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0 2.2 2.4 2.6 2.8 3.0 3.2 3.4 3.6 3.8 4.0 4.2 4.4 4.6 4.8 5.0 5.2 5.4 5.6 5.8 6.0 6.2 6.4 6.6 6.8 7.0 7.2 7.4 7.6 7.8 8.0 -10 -5 0 5 10 -7.0 -6.5 -6.0 -5.5 -5.0 -4.5 -4.0 -3.5 -3.0 -2.5 -2.0 -1.5 -1.0 -0.5 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0 5.5 6.0 6.5 7.0 7.5 8.0 y Reconstructed signal overlayed with x0