Homography (2D)¶
Geometrically, given any two planes, a homography is the operation that projects the points from one plane (plane A) to other (plane B) via a projection point P, which does not lie on any of them. The projection is carried out by forming a line which connects the projection point P to any point on plane A and intersect plane B. This geometric description is not very useful for computation.
Algebraically, a homography is matrix transformation given that the points on plane A and plane B are expressed in homogeneous coordinates:
$$\begin{bmatrix} x' \\ y' \\ 1 \end{bmatrix} = \begin{bmatrix} a & b & c\\ d & e & f\\ g & h & 1 \end{bmatrix}\begin{bmatrix} x \\ y \\ 1 \end{bmatrix}$$Each transformation matrix has 8 unknowns and we need at least four points of correspondence to solve for it.
Use homography to change perspective¶
import numpy as np
from matplotlib import pyplot as plt
import cv2
plt.figure(figsize = (10, 10))
src_image = cv2.imread('book.jpg')
src_points = np.asarray([[203, 164], [46, 302], [202, 428], [341, 238]])
dst_points = np.asarray([[0, 0], [0, 400], [300, 400], [300, 0]])
h, status = cv2.findHomography(src_points, dst_points)
dst_image = cv2.warpPerspective(src_image, h, (300, 400))
for point in src_points:
src_image = cv2.circle(src_image, (point[0], point[1]), 2, (0,255,0), 2)
plt.subplot(121); plt.imshow(src_image)
plt.subplot(122); plt.imshow(dst_image)
plt.show()
Projective Geometry¶
Essential Matrix is the matrix which relates the coordinates of a point in the system of one camera to the coordinates of the same point in the system of another camera:
$$\begin{bmatrix} x' \\ y' \\ z' \end{bmatrix}^T(T \times R)\begin{bmatrix} x \\ y \\ z \end{bmatrix} = 0$$$$X'EX = 0$$Making use of the fact that the Internal Matrix relates the coordinates of the camera plane to the image plane, we can come up with the relationship between the coordinates of two image planes, via the Fundamental Matrix $F$:
$$(K_{im,right}^{-1} p_{im,right})^T F(K_{im,left}^{-1} p_{im,left}) = 0$$$$p_{im,right}^T F p_{im,left} = 0$$If we know a few pairs of correspondence points between the two images, we can solve for F, which enables us to find the epipolar lines and infer the depth information of the scene.
Find epipolar lines using Fundamental Matrix¶
import numpy as np
from matplotlib import pyplot as plt
import cv2
plt.figure(figsize = (10, 10))
point_2a = []
point_2b = []
# construct the A matrix
with open('/home/andy/Desktop/computer_vision/ps3/input/pts2d-pic_a.txt') as file:
for line in file:
point_2a.append([float(item.strip()) for item in line.split(' ') if item.strip() != ''])
with open('/home/andy/Desktop/computer_vision/ps3/input/pts2d-pic_b.txt') as file:
for line in file:
point_2b.append([float(item.strip()) for item in line.split(' ') if item.strip() != ''])
A = np.zeros((len(point_2a), 9))
for i in range(len(point_2a)):
temp = np.asanyarray(point_2a[i] + [1])
A[i] = np.append(point_2b[i][0] * temp, [point_2b[i][1] * temp, temp])
# find a non-trivial solution of Am = 0, using SVD
eigendecom = np.linalg.eig(np.dot(np.transpose(A), A))
F = np.reshape(eigendecom[1][:,-1], (3, 3))
# reduce the rank of the fundamental matrix
U, s, V = np.linalg.svd(F, full_matrices=True)
s[2] = 0
F = np.dot(np.dot(U, np.diag(s)), V)
# draw epipolar lines
image_a = cv2.imread('/home/andy/Desktop/computer_vision/ps3/input/pic_a.jpg')
image_b = cv2.imread('/home/andy/Desktop/computer_vision/ps3/input/pic_b.jpg')
def draw_epi_lines(image, F, point):
p_ul = (0, 0, 1)
p_bl = (0, image.shape[0], 1)
p_ur = (image.shape[1], 0, 1)
p_br = (image.shape[1], image.shape[0], 1)
l_l = np.cross(p_ul, p_bl)
l_r = np.cross(p_ur, p_br)
for i in range(len(point)):
l = np.dot(F, np.asanyarray(point[i] + [1]))
p_l = np.cross(l, l_l)
p_r = np.cross(l, l_r)
p_l = tuple((p_l[0:2]/p_l[2]).astype(int))
p_r = tuple((p_r[0:2]/p_r[2]).astype(int))
image = cv2.line(image, p_l, p_r, (255,0,0), 2)
return image
plt.subplot(121), plt.imshow(draw_epi_lines(image_a, np.transpose(F), point_2b))
plt.subplot(122), plt.imshow(draw_epi_lines(image_b, F, point_2a))
plt.show()
