The alignment of the horizon in a photo-service gfranq.com

photo service gfranq.com introduces the ability to align photos to any angle! This angle is calculated automatically, but if necessary it can be easily changed manually. The horizon line you can draw with the right mouse button, and the processed image may be rectangular, unlike instagram. Moreover, there is an option to preserve original image size or coverage the maximum area in the rotated image.



All to learn as we have a method of automatic alignment, and what algorithms were used, welcome under kat.

Method of automatic alignment of the horizon

In order for the automatic alignment of the horizon of functioning at an acceptable level for most images, this task it was decided to split the following stages:

the
1. Definition of borders.
the
2. Definition of straight lines.
the
3. Calculate the most intense line.
the
4. Calculate the angle between the found line and the center of the image.
the
5. image Rotation for the calculated angle.
the
6. Calculation of the maximum rectangle inscribed in the rotated image.

Further, these stages will be discussed in detail.

defining the boundaries of the operator (Kenny).

To determine the boundaries, it was decided to use the detector boundaries Kanye, on the basis of subjective and objective considerations (which can be found in Wikipedia).
Kanye algorithm consists of the following steps:

the
1. Converting images to black-and-white format.
the
2. image Blur on the Gauss.
the
3. gradients.
the
4. Suppression of nmaximum and tracing of the region of ambiguity.

Definition of straight lines (Hough transform).

After the borders were found (abrupt changes in brightness or other heterogeneities), it is possible to apply an algorithm for extracting straight lines as the horizon line usually appears as a straight or almost straight lines (possibly with noise). As this algorithm was chosen Hough transform. However, this conversion can return a very large number of lines, besides, it returns an image (matrix) in which horizontal angles are measured lines and the vertical distance from the center to the line, which is inconvenient for the next stages. To solve these problems have been written a function to convert lines from polar coordinates to rectangular

Convert lines in polar coordinates in the segments in Cartesian coordinates

private static WeightedLine HoughLineToTwoPointLine(double theta, short radius, double intensity, int width, int height)
{
int r = radius;
double t = theta;

if (r < 0)
{
t += 180;
r = -r;
}

t = (t / 180) * Math.PI;
int w2 = width / 2;
int h2 = height / 2;
double x0 = 0, x1 = 0, y0 = 0, y1 = 0;
if (theta != 0)
{
x0 = -w2;
x1 = w2;
double sint = Math.Sin(t);
double cost = Math.Cos(t);
y0 = (-cost * x0 + r) / sint;
y1 = (-cost * x1 + r) / sint;
}
else
{
x0 = radius;
x1 = radius;
y0 = h2;
y1 = -h2;
}

return new WeightedLine(x0 + w2, h2 - y0, x1 + w2, h2 - y1, intensity);
}

the Calculation result of the angle

The function to compute the angle between the perpendicular passing through the center of the image with dimensions width and height and a line with coordinates x1, y1, x2, y2 and the horizontal line (in other words the angle at which to rotate the image to the horizon line x1, y1, x2, y2 have been aligned horizontally):

Code method, calculate the resultant angle

public static double CalculateAngle(int width, int height, double x1, double y1, double x2, double y2)
{
double dx = x2 - x1;
double dy = y2 - y1;
double x3 = width / 2;
double y3 = height / 2;
double r = dx * dx + dy * dy;

double ny = (dx * (y1 * dx - x1 * dy) + dy * (dx * x3 + b * y3)) / r - y3;
double result = Math.Atan2(ny, nx) + Math.PI / 2;
if (result > Math.PI)
result = result - Math.PI * 2;
return result;
}

It should be noted that if the calculated angle exceeds a certain preset value of rotation (in our case 45°), the automatic rotation will not be performed.

result Calculation inscribed rectangle

After calculated the angle at which to rotate the image, it is necessary to calculate the dimensions of the maximum rectangle inscribed in the rotated image. It should be noted that the rectangle can be proportional (in this case, the original image size is preserved, i.e., the part of the entered image is stretched to the original size) and disproportionate that covers the maximum area in the rotated image.

To solve this problem has been found related question on stackoverflow, and one answer was modified in the following code: a link to stackoverflow.

For better understanding the above steps, I have prepared a graphical illustration of the process:

Technical details

Our project is designed so that the code certain modules are written in C# that is compiled as under .NET, and JavaScript, as was described in one of the previous articles. Accordingly, the code and this module was also written in C#. I had to use one-dimensional arrays instead of two dimensional, as well as to consider some other limitations Script#.

Rotate images in Google Chrome

Unfortunately, in Google Chrome there is a bug consisting in the absence of image smoothing at the boundaries in the transformations (e.g., rotations), which is clearly demonstrated in the figure below on the left, while the images themselves are interpolated correctly. In other modern browsers (IE, Firefox, Safari, Opera) this bug was noticed by us. So they invented the method how to avoid it: you can just draw a transparent border around the image (i.e. the image size is 2 pixels less) with the following code:

the

context.draw(image, 1, 1, decImageWidth - 2, decImageHeight - 2);

Thus it was possible to achieve a smoothing effect in all browsers (picture below right).

Opinion

By varying the coefficients of the algorithm determining the boundaries and the Hough transform is proposed, managed to achieve acceptable quality and the speed of the automatic alignment, which was tested on several sample images. And for those images where the method works correctly, the angle is easy to fix manually in two ways, what you can see on our photo service gfranq.com by adding new or editing an existing photo.

On mobile platforms (iOS and Android) alignment function pictures will appear in the near future.

Article based on information from habrahabr.ru