Seam Carving Image Resize
Using a seam carving "energy" technique, I resized a rectangular image into a square image. (For a full description of what was done, check out the github link below the code snippet).
Below is a photo grid showing what this "energy" seam technique is doing behind the scenes. We start with a rectangular photo, then we calculate the seam with the least energy (shown in red - found using the Sobel function to determine pixel weights) which gets deleted. This is repeated over and over until we have a square image with only the highest energy parts of the image (or only the parts of the image with the highest edge detection weights). A code snippet of my calcWeight function is below the images. I wrote all the code myself using the OpenCV and Numpy Python libraries.
'''
calcWeight
using the energy matrix created by calcEnergy with the Sobel function,
we assign weights to each index of the image based on this "energy".
We assign columns 0 and N − 1 to be infinity since we do not want to delete the edges.
We go through the matrix like so:
-go through starting with the first row (this is just a row of energies)
-set the 0th and last indicies to be infinity
-get all indicies between the first and last (exclusive)
-go to the row below, for each index you add the min energy from the
row above (just contained to the touching indecies) (we use a stacking method for comparison)
say this is the original energy matrix:
____________________________________
|__5_|__8_|_12_|__2_|_22_|__8_|__9_|
|__3_|_12_|__7_|_21_|_14_|__4_|_10_|
|_11_|__3_|__9_|_13_|__6_|_32_|_23_|
using this example, we then add the min touching index to the index below it, like so:
____________________________________
|__5_|__8_|_12_|__2_|_22_|__8_|__9_|
|__8_|_17_|__9_|_23_|_16_|_12_|_18_|
the 8 above comes from the index being 3 originally, then taking the min of the indicies touching
above (5 and 8, so 5 is the min) and adding that min to the index (3+5=8)
the 17 above comes from the index being 12 originally, then taking the min of the touching indicies (5, 8, 12)
in the row above and adding the min of those 3 indicies to the index (12+5 = 17)
etc following this pattern, the following row using these newly added numbers would be:
|_19_|_11_|_18_|_22_|_18_|_44_|_35_|
the 19 comes from orginal index 11 adding the min of the touching above indices (8, 17) (11+8=19), etc
'''
def calcWeight(energy):
weight = np.copy(energy)
weight[0][0] = np.inf
weight[0][-1] = np.inf
for i in range(1, len(energy)):
#we get the row above the current row since that is what
#will be added on to the current row
row = np.copy(weight[i-1,:])
row[0] = np.inf
row[-1] = np.inf
is_min = np.zeros_like(row)
#we get the staggard indices (left, right, center) so we can stack them for comparison
#each col of the stacked array will be the comparison between one index and its two neighbors
left = row[ :-2]
right = row[ 2: ]
center = row[ 1:-1 ]
stack = np.vstack((left, right, center))
#is_min is the array of the previous row in weight
is_min[ 1:-1 ] = np.amin(stack, axis=0)
is_min[0] = np.inf
is_min[-1] = np.inf
#the min value of the stack gets added to the index in the row below
weight[i,:] = np.add(is_min, weight[i,:])
return weight
calcWeight
using the energy matrix created by calcEnergy with the Sobel function,
we assign weights to each index of the image based on this "energy".
We assign columns 0 and N − 1 to be infinity since we do not want to delete the edges.
We go through the matrix like so:
-go through starting with the first row (this is just a row of energies)
-set the 0th and last indicies to be infinity
-get all indicies between the first and last (exclusive)
-go to the row below, for each index you add the min energy from the
row above (just contained to the touching indecies) (we use a stacking method for comparison)
say this is the original energy matrix:
____________________________________
|__5_|__8_|_12_|__2_|_22_|__8_|__9_|
|__3_|_12_|__7_|_21_|_14_|__4_|_10_|
|_11_|__3_|__9_|_13_|__6_|_32_|_23_|
using this example, we then add the min touching index to the index below it, like so:
____________________________________
|__5_|__8_|_12_|__2_|_22_|__8_|__9_|
|__8_|_17_|__9_|_23_|_16_|_12_|_18_|
the 8 above comes from the index being 3 originally, then taking the min of the indicies touching
above (5 and 8, so 5 is the min) and adding that min to the index (3+5=8)
the 17 above comes from the index being 12 originally, then taking the min of the touching indicies (5, 8, 12)
in the row above and adding the min of those 3 indicies to the index (12+5 = 17)
etc following this pattern, the following row using these newly added numbers would be:
|_19_|_11_|_18_|_22_|_18_|_44_|_35_|
the 19 comes from orginal index 11 adding the min of the touching above indices (8, 17) (11+8=19), etc
'''
def calcWeight(energy):
weight = np.copy(energy)
weight[0][0] = np.inf
weight[0][-1] = np.inf
for i in range(1, len(energy)):
#we get the row above the current row since that is what
#will be added on to the current row
row = np.copy(weight[i-1,:])
row[0] = np.inf
row[-1] = np.inf
is_min = np.zeros_like(row)
#we get the staggard indices (left, right, center) so we can stack them for comparison
#each col of the stacked array will be the comparison between one index and its two neighbors
left = row[ :-2]
right = row[ 2: ]
center = row[ 1:-1 ]
stack = np.vstack((left, right, center))
#is_min is the array of the previous row in weight
is_min[ 1:-1 ] = np.amin(stack, axis=0)
is_min[0] = np.inf
is_min[-1] = np.inf
#the min value of the stack gets added to the index in the row below
weight[i,:] = np.add(is_min, weight[i,:])
return weight
View the full code and see output on my github: https://github.com/marymonty/sample_code/tree/master/python/computational_vision