Happy July everyone!

This past week was definitely full of learning a lot of new things about Computer Visions. On Monday, we met up with our graduate mentor, Sven, to discuss possible ways to divide up an image of multiple ICs into individual ones to make our matching program work more efficiently. Previously mentioned last week, we attempted to use the blob detector to the task but that quickly failed to do much since it is better at finding circle shapes or squares (our ICs are rectangles). Instead, Sven helped us create a binary edge map of our image using a Sobel filter and setting all pixels to either white or black (results below). This helped make a very simple image that can be easily read and distinguished.

Following that meeting with Sven, he had us research other methods that can potentially work better than the blob detector. Two that he mentioned were the Hough Transform, which helps detect lines and further find shapes through that, and the other was Distance Transform. However, the algorithms were tough to understand and the results were not what I wanted (results below). Nonetheless, it was a good attempt to solve our problem. During our lab meeting Tuesday, we displayed the resulting image to our professor and let him know that we were currently trying to find different ways to crop the ICs out.

After doing a little bit of digging, I came across a blog where a man was trying to take only text of an image and crop it out of the entire thing. In order to do so, he created an algorithm that finds patterns within the image that ‘seem’ to be relative enough to be an object. With a little bit of tweaking the parameters, I was able to have the program look at my IC image and find the separate ICs within the entire image. In order to help the program run efficiently, I used the Sobel Filter method that Sven helped us with and it made the cropping much accurate. The results were great for our testing image but varied in success overall (see test results below). The original code only drew rectangles around the ICs but would crop a chunk of the image (in red) out because the code was obviously meant for different reasons. However, I was able to rewrite the code to crop the rectangles instead of draw them, creating multiple sub files that I save separately (shown below on the right).

On Friday, we met with Professor Crandall to show him our results and he was ecstatic to see we were able to progress. He helped us brainstorm further things to add into our program to make it more efficient and accurate. First off, since the rectangles are sometimes random areas, I have to make another function that can clean the images out before running the matching algorithm. Secondly, he wants us to try to see if we can use just the texts on the ICs to help make the matching even more accurate. To do so, he recommended us to use Computer Recognition programs that companies offer online such as Rekognition by Amazon, Google Vision, or Tesseract. This upcoming week, we will play around with these Computer Recognition programs and try to clean up our file space. From there, we hope to get a range of accuracy for the matching algorithm we currently have been using.

Thank you for listening and enjoy your Fourth of July!

G