Main page # Project 4A ## Description The main goal of this project was creating a photo mosaic from several images made from the same location but with different angle of view. ## Step 1: Shoot and digitize pictures For this project I took all photos with my iPhone 14 Pro Max and than compressed their resolution to 756x1008. Here are some pictures I took for mosaic creation: 1. View from Study Hall Rooftop Lounge
2. University way photos
3. Night photos of The Campanile and other surroundings from the top of Campbell Hall
## Step 2: Recover homographies To recover homographies, I first marked correspondence points on image pairs with the help of https://cal-cs180.github.io/fa23/hw/proj3/tool.html ``` [wx'] [h11 h12 h13] [x] [wy'] = [h21 h22 h23] * [y] [w] [h31 h32 h33] [1] ``` Based on the homography matrix operation above, for every pair of corresponding points I created two linear equations in the form: ``` h11 * x + h12 * y + h13 - (h31 * x + h32 * y + h33) * x' = 0 h21 * x + h22 * y + h23 - (h31 * x + h32 * y + h33) * y' = 0 ``` I then used `numpy.linalg.svd` to solve the group of linear equations and picked the best solution based on smallest singular value. ## Step 3: Warp the images To warp an image with the homography matrix, I first warped the corners of the image to get the warped image size. I then warped every pixel of the original image one by one, changing all neighboring pixels in the projection image in case the warped pixel had float coordinates. Here are some examples of warped images for pictures from Study Hall Rooftop Lounge:
First image original
Second warped to first

First warped to second
Second image original

Example of Campanile warped to the first night image:
First image original
Campanile warped to first image
## Step 3.5: Image Rectification To rectify images, I followed followed project instructions for correspondence points. I then used calculated homography and used image warping logic to warp the image into a rectangle. Here are some examples of rectified images:
Project 4 instructions on my monitor in safari browser
Rectified instructions to wrong aspect ratio
Rectified instructions to correct aspect ratio (somewhat readable)

Properties company sign outside
Rectified company sign

Beautiful mosaic window of a neighoring house
Rectified window (top half)
## Step 4: Blend images into a mosaic All of my images were taken with slight rotation left/right. To combine images into a mosaic, I accounted for the changed height of the warped image. When blending images in case where pixels overlap, I used a weighted average to blend the images together. Weight for blending was based on horizontal distance to center of the combined images. Here are pairs of images and their mosaics: 1. View from Study Hall Rooftop Lounge
2. University way photos. This one has the most defects in the background as the distance to the road was way closer than background trees and images.
3. Night photos of The Campanile and other surroundings from the top of Campbell Hall. This is my personal favorite, I took these photos while watching stars for astronomy class.
## Extra stuff Here are some mistakes I made on my first attempts of homography calculation and image warping.
Wrong signs in linear equations.
Didn't account for pixels warping to float coordinates (just rounded them).
# Project 4B ## Description The main goal of this project was creating a photo mosaic from several images made from the same location but with different angle of view. Different from the 4A, manual image correspondence is not allowed. To generate homographies, corresponding points on the images must be selected automatically. Here are the images I will use for different step illustrations:
## Step 1: Detecting corner features in an image For this step I just used the provided implementation of Harris.
## Step 1.5: Selecting best corners in images For this step I implemented anms as suggested in the paper. I used suggested param values: n_ip = 500, c_robust = 0.9 Here are the corners I had left after anms:
## Step 2: Extracting a Feature Descriptor for each feature point For this step as suggested in project instructions, for each left corner I extracted feature descriptor as 8x8 image block extracted from 40x40 image block centered at the corner coordinates. I also did normalization as suggested. ## Step 3: Matching these feature descriptors between two images As suggested in lecture, to match points on two images, for each corner of the first image I iterated over all corners of the second image and calculated differences between corresponding feature descriptors. Then I would only produce a match if the best match between fixed corner of the first image was significantly better than any other possible match (for other corner of the second image), as suggested in lecture. Here are the matches I got:
## Step 4: Use a robust method (RANSAC) to compute a homography I implemented RANSAC as suggested in lecture. I would repeatedly pick 4 random points from the matches, compute homography based on them, then check how many point pairs fit this homography with some allowed error. Finally I would select matches depicted by the homography with the most inliers. Here are the matches I got:
## Step 5: Proceed as in the first part to produce a mosaic Same images as in manual mosaics:
First image
Second image
Manual mosaic
Automatic mosaic

First image
Second image
Manual mosaic
Automatic mosaic
All automatic mosaics I got (more than manual since I didn't have to mark the points):
## Cookies I also tried generating mosaics out of more than two images. I tried naive idea of computing homographies between image pairs And then warping all images to the plane of the left most image But it seems like the small errors in homographies would lead to great errors when multiplying so I got pretty bad results. It was still fun to try though.
## What I learned I was surprised to see how automatic mosaics turned out same or even better than manual ones. I think that manual selection of points leads to errors in coordinates that might be more significant than errors in automatic feature matching. It was also interesting to see how RANSAC helped a lot to get rid of outliers in matches. It's fascinating to me how this random process of picking points and computing homography worked so well. Finally it was fun to be reminded how small errors in individual homography computations accumulate for matrices multiplication and how this leads to pretty bad results when trying to warp more than two images together. Of course this was a naive approach and there are more sophisticated ways to do this.