opencv - Calculating homography of multiple markers on a single plane -

i've got algorithm calculates homography of number of markers in single image, using opencv's findhomography. works fine, due size of markers/resolution of images, rotation calculated quite inaccurate (mainly around x , z axes of marker, assuming y up). i'd love find way improve this.

i have hypothesis fact know markers on same plane (they're on table) should improve homography - need work out rotation of marker around single axis - don't know enough put in practice. can give me guidance on this?

this example of situation i'm trying deal (the images on markers different in reality).

enter image description here

i idea of getting higher precision estimated homography using multiple marker images spread across scene.

this approach can realized using nonlinear optimization. it’s little tricky.

assumptions, model, data

let’s take example of 3 printed marker images placed across table , assume assumptions hold.

and let’s assume each of 3 printed marker images you’ve found n point correspondences.

point correspondences here means printed marker image can associate n image pixel positions corresponding n real world positions (x,y) in centimeters (or other unit) respect known real world "printed marker coordinate system" (for example, coordinate (0,0) located @ center of marker image).

the problem "printed marker coordinate system" known locally each printed marker image. need bring coordinate systems , find way express "printed marker coordinate system" positions in single, global coordinate system. need figure out relation between them.

relation between each marker’s homographies

we want compute homography uses correspondence data different printed marker images. therefore, important understand how relate each. specifically, want find common coordinate system marker images in order formulate optimization problem.

now, based on point correspondences, , using conventional methods, compute 3 homographies:

h1 maps "printed marker coordinate system" positions of marker image #1 image pixel coordinates
h2 maps "printed marker coordinate system" positions of marker image #2 image pixel coordinates
h3 maps "printed marker coordinate system" positions of marker image #3 image pixel coordinates

let’s assume shortly no measurement , computation errors present (just hypothetically). then, homographies differ in rotation $\varphi$ , translation $(t_x,t_y)$ of respective "printed marker coordinate systems".

then, following equations hold:

$h_1 &= h_2 \cdot a(\varphi^{(2)}, t_{x}^{(2)}, t_{y}^{(2)})$

$h_1 &= h_3 \cdot a(\varphi^{(3)}, t_{x}^{(3)}, t_{y}^{(3)})$

with rotation , translation defined follows:

$a(\varphi, t_x, t_y) := \begin{pmatrix}\cos(\varphi)&-\sin(\varphi)&t_x\\\sin(\varphi)&\cos(\varphi)&t_y\\0&0&1\end{pmatrix}$

putting together

that means h1 equals h2 "up to" rotation $\varphi$ , translation $(t_x^{(2)},t_y^{(2)})$ , h1 equals h3 "up to" rotation $\varphi$ , translation $(t_x^{(3)},t_y^{(3)})$ .

this gives way transforming positions of marker image #2 coordinate system of marker image #1 (case positions of marker image #3 can handled analogous):

$\begin{pmatrix}x^{(1)}\\y^{(1)} \end{pmatrix}:=\begin{pmatrix}\cos(\varphi^{(2)})&-\sin(\varphi^{(2)})\\\sin(\varphi^{(2)})&\cos(\varphi^{(2)})\\ \end{pmatrix}\cdot\begin{pmatrix}x^{(2)}\\y^{(2)} \end{pmatrix} + \begin{pmatrix}t_x^{(2)}\\t_y^{(2)} \end{pmatrix}$

now, in real world scenario, these equations hold approximately. nevertheless give motivation estimating rotation , translation between pair of printed marker images:

$h_2^{-1}h_1\approx \begin{pmatrix}\cos(\varphi^{(2)})&-\sin(\varphi^{(2)})&t_x^{(2)}\\\sin(\varphi^{(2)})&\cos(\varphi^{(2)})&t_y^{(2)}\\0&0&1\end{pmatrix}$

$h_3^{-1}h_1\approx \begin{pmatrix}\cos(\varphi^{(3)})&-\sin(\varphi^{(3)})&t_x^{(3)}\\\sin(\varphi^{(3)})&\cos(\varphi^{(3)})&t_y^{(3)}\\0&0&1\end{pmatrix}$

now, using atan2, can find estimates of 6 auxiliary variables $(\varphi^{(2)},t_x^{(2)},t_y^{(2)},\varphi^{(3)},t_x^{(3)},t_y^{(3)})$ describe how printed marker images #2 , #3 located relative #1.

that’s need preparation have start values , model formulate nonlinear optimization problem.

nonlinear optimization problem

the final goal find 8 coefficients of homography matrix h uses point correspondences printed marker images @ same time:

$h=\begin{pmatrix}h_{11}&h_{12}&h_{23}\\h_{21}&h_{22}&h_{23}\\h_{31}&h_{32}&1 \end{pmatrix}$

using above model, can define system of nonlinear equations. have 8 + 6 unknowns:

8 homography coefficients
6 auxiliary variables $(\varphi^{(2)},t_x^{(2)},t_y^{(2)},\varphi^{(3)},t_x^{(3)},t_y^{(3)})$ describe how correspondences marker image #2 , #3 mapped marker image #1 coordinate system

using n point correspondences of 3 marker images, n*3*2 = n*6 equations. means, need n>=3 point correspondences in order have more equations unknowns. interestingly, minimum n=3 correspondences per marker image in case of multiple marker image scenario less minimum n=4 in case of single marker image scenario.

i’ll not add formulation of nonlinear optimization problem in more detail @ point general idea of how approach problem explained.

for solving nonlinear optimization problem, newton’s method can used, example.

remark

all matrices in above equations need normalized follows:

$h=\begin{pmatrix}\ast&\ast&\ast\\\ast&\ast&\ast\\\ast&\ast&1 \end{pmatrix}$

WIKI

Search This Blog