On Thursday, at its Ability Summit, Microsoft shared that Seeing AI is now available in an additional 14 languages(total 33 languages) and will be rolling out the latest generative AI models, including richer descriptions of images and chat capability for photos and documents. The innovative app harnesses Azure AI technologies to empower those with low-vision abilities, offering enhanced assistance in navigating daily life.
Introduction to the Seeing AI app
The Seeing AI app integrates cutting-edge Azure GPT-4 Turbo with Vision AI capabilities, ensuring responsive and accurate responses on any smartphone equipped with these features. Designed to cater to various needs, the app provides a range of functionalities.
The app enables users to have text read aloud to them, facilitating easier comprehension of written content. Shorter text can be instantly read in live camera mode, while longer passages are captured, converted to text, and subsequently vocalized.
Versatile features
Bar Code Reader: Users can scan product UPCs to obtain relevant information.
Scene Description: The app provides descriptive insights into the user’s surroundings, focusing on the nearest objects.
Person Identification: Through facial recognition technology, the app describes individuals and identifies their emotional expressions.
Currency Recognition: Users can easily identify various bills from 17 different countries.
Color Identification: The app discerns colors captured in live photos, aiding users in distinguishing visual elements.
Handwriting Conversion: Handwritten notes are seamlessly converted into text for improved accessibility.
Brightness Detection: A tone varying in pitch depending on brightness assists users in perceiving their environment.
While the app boasts impressive accuracy in many scenarios, it is essential to acknowledge its limitations. The app may struggle with more complex items like coin sorters despite correctly identifying common objects such as shaving razors. Similarly, while text recognition on receipts and packaging is generally reliable, the app may falter when confronted with mathematical or scientific texts.
Implications and prospects
The Seeing AI app marks a significant step forward in digital accessibility, demonstrating Microsoft’s commitment to leveraging AI for social good. By harnessing the power of Azure AI technologies, the app empowers individuals with low-vision abilities to navigate their surroundings more independently.
AI and machine learning advancements hold promising prospects for further enhancing the app’s capabilities and addressing its current limitations. Continuous refinement and iteration will be crucial in ensuring that Seeing AI remains at the forefront of digital accessibility solutions, catering to the evolving needs of its users.
