Jewelry Recognition via Encoder-Decoder Models
Author(s)
José M. Alcalde-Llergo
Enrique Yeguas-Bolívar
Andrea Zingoni
Alejandro Fuerte-Jurado
Date Issued
2023
Type
conferenceObject
Start Page
116
End Page
121
Abstract
Jewelry recognition is a complex task due to the different styles and designs
of accessories. Precise descriptions of the various accessories is something
that today can only be achieved by experts in the field of jewelry. In this
work, we propose an approach for jewelry recognition using computer vision
techniques and image captioning, trying to simulate this expert human behavior
of analyzing accessories. The proposed methodology consist on using different
image captioning models to detect the jewels from an image and generate a
natural language description of the accessory. Then, this description is also
utilized to classify the accessories at different levels of detail. The
generated caption includes details such as the type of jewel, color, material,
and design. To demonstrate the effectiveness of the proposed method in
accurately recognizing different types of jewels, a dataset consisting of
images of accessories belonging to jewelry stores in C\'ordoba (Spain) has been
created. After testing the different image captioning architectures designed,
the final model achieves a captioning accuracy of 95\%. The proposed
methodology has the potential to be used in various applications such as
jewelry e-commerce, inventory management or automatic jewels recognition to
analyze people's tastes and social status.
Conference(s)
2023 IEEE International Conference on Metrology for eXtended Reality, Artificial Intelligence and Neural Engineering (MetroXRAINE)