Cem Ünsalan

M. Erkin Yücel, Cem Ünsalan

Smart retail stores are becoming the fact of our lives. Several computer vision and sensor based systems are working together to achieve such a complex and automated operation. Besides, the retail sector already has several open and challenging problems which can be solved with the help of pattern recognition and computer vision methods. One important problem to be tackled is the planogram compliance control. In this study, we propose a novel method to solve it. The proposed method is based on object detection, planogram compliance control, and focused and iterative search steps. The object detection step is formed by local feature extraction and implicit shape model formation. The planogram compliance control step is formed by sequence alignment via the modified Needleman-Wunsch algorithm. The focused and iterative search step aims to improve the performance of the object detection and planogram compliance control steps. We tested all three steps on two different datasets. Based on these tests, we summarize the key findings as well as strengths and weaknesses of the proposed method.

Cem Ünsalan

Surface quality is an extremely important issue for wood products in the market. Although quality inspection can be made by a human expert while manufacturing, this operation is prone to errors. One possible solution may be using standard machine vision techniques to automatically detect defects on wood surfaces. Due to the random texture on wood surfaces, this solution is also not possible most of the times. Therefore, more advanced and novel machine vision techniques are needed to automatically inspect wood surfaces. In this study, we propose such a solution based on support region extraction from the gradient magnitude and the Laplacian of Gaussian response of the wood surface image. We introduce novel structural and conditional statistical features using these support regions. Then, we classify different defect types on wood surfaces using our novel features. We tested our automated wood surface inspection system on a large data set and obtained very promising results.

Cem Ünsalan

We introduce a method to extract curve segments in parametric form from the image directly using the Laplacian of Gaussian (LoG) filter response. Our segmentation gives convex and concave curves. To do so, we form curve support regions by grouping pixels of the thresholded filter response. Then, we model each support region boundary by Fourier series and extract the corresponding parametric curve segment.

M. Erkin Yücel, Serkan Topaloğlu, Cem Ünsalan

The retail sector presents several open and challenging problems that could benefit from advanced pattern recognition and computer vision techniques. One such critical challenge is planogram compliance control. In this study, we propose a complete embedded system to tackle this issue. Our system consists of four key components as image acquisition and transfer via stand-alone embedded camera module, object detection via computer vision and deep learning methods working on single board computers, planogram compliance control method again working on single board computers, and energy harvesting and power management block to accompany the embedded camera modules. The image acquisition and transfer block is implemented on the ESP-EYE camera module. The object detection block is based on YOLOv5 as the deep learning method and local feature extraction. We implement these methods on Raspberry Pi 4, NVIDIA Jetson Orin Nano, and NVIDIA Jetson AGX Orin as single board computers. The planogram compliance control block utilizes sequence alignment through a modified Needleman-Wunsch algorithm. This block is also working along with the object detection block on the same single board computers. The energy harvesting and power management block consists of solar and RF energy harvesting modules with suitable battery pack for operation. We tested the proposed embedded planogram compliance control system on two different datasets to provide valuable insights on its strengths and weaknesses. The results show that our method achieves F1 scores of 0.997 and 1.0 in object detection and planogram compliance control blocks, respectively. Furthermore, we calculated that the complete embedded system can work in stand-alone form up to two years based on battery. This duration can be further extended with the integration of the proposed solar and RF energy harvesting options.