Microsoft’s New iPhone App Helps the Vision Impaired Navigate the Real World

Microsoft today announced a new app for iPhone called Seeing AI that helps those with low vision navigate through the real world. Basically, it uses AI (artificial intelligence) to describe the people, text, and objects around you.

Like, seriously.

Windows Intelligence In Your Inbox

Sign up for our new free newsletter to get three time-saving tips each Friday — and get free copies of Paul Thurrott's Windows 11 and Windows 10 Field Guides (normally $9.99) as a special welcome gift!

"*" indicates required fields

This field is for validation purposes and should be left unchanged.

“Designed for the blind and low vision community, this ongoing research project harnesses the power of artificial intelligence to open up the visual world and describe nearby people, text, and objects,” a new post to the Microsoft Accessibility Blog explains. “The app uses artificial intelligence and the camera on your iPhone to perform a number of useful functions.”

Those functions include:

Reading documents. Seeing AI recognizes the structure of documents, such as headings, paragraphs, and lists, so you can skip through them easily using voiceover.

Identifying products. Seeing AI can identify products using barcodes and your iPhone’s camera.

Recognizing people. Seeing AI will provide a description of a person’s visual appearance, including their gender, facial expression, and other identifying characteristics.

Recognizing images within other apps. Seeing AI is available via the iOS Share interface so you can use it to recognize images in any app.

Seeing AI is essentially a research project, so it also includes some other experimental features, including a Scene channel that describes photos you capture with your iPhone’s camera.

“This app will be a game changer for many as it’s just simple to use,” Jenine Stanley, who is legally blind and reviews apps and technology for the American Council of the Blind’s tech radio show, told Microsoft.

You can learn more at the Seeing AI website.

Seeing AI is available for free from the Apple Apps Store in the United States, Canada, India, Hong Kong, New Zealand and Singapore. Microsoft says it will bring the app to more countries in the near future.

 

Tagged with

Share post

Please check our Community Guidelines before commenting

Conversation 7 comments

  • skane2600

    12 July, 2017 - 11:02 am

    <p>I don't know how well it will work, but assistive&nbsp;technologies are always welcome.</p>

    • evox81

      Premium Member
      12 July, 2017 - 2:26 pm

      <blockquote><a href="#143637"><em>In reply to skane2600:</em></a></blockquote><p>I saw a demo of it on Bill Nye's (horrible) Netflix show… It was very impressive. Although I think what they demonstrated was a bit beyond what they're offering here, when he pointed it at a microscope, the Microsoft people acted really nervous and said "I'm not sure it will get that one, it's still learning" and the phone said "I think it's a microscope sitting on a stainless steel table in front of [something]"… It was great.</p>

  • Narg

    12 July, 2017 - 2:53 pm

    <p>Played with it a little. It crashes a lot.</p>

  • Dont Fear the Future

    12 July, 2017 - 4:17 pm

    <p>"Available in the United States, Canada, India, Hong Kong, New Zealand and Singapore."</p><p><br></p><p>But I know where it&nbsp;WON'T be available: On Windows.</p>

    • Dont Fear the Future

      13 July, 2017 - 8:22 pm

      <blockquote><a href="#143945"><em>In reply to Dont Fear the Future:</em></a></blockquote><blockquote><em>Oh look; I got down voted. That must mean I'm wrong and this app will make it's way onto Windows.</em></blockquote><blockquote><em>As a huge Microsoft fan, I would be so happy to be wrong.</em></blockquote><blockquote><em>All bets in. Will I be wrong?</em></blockquote><p><br></p>

  • Tyrone

    Premium Member
    17 July, 2017 - 2:47 am

    <p>my mother is blind.. I downloaded the app for her but it is useless.. It requires her to see icons on the screen she can't see.. the only way it works is if the app is integrated with Cortana and you can verbally request that the app tell you what it is seeing or read a document for you. otherwise pointless.. truly great apps for blind people are apps like cortana or Siri or Amazon echo that can do things via a verbal request.. </p><p><br></p>

  • mafleon

    26 August, 2017 - 7:58 am

    <p>My mobile battery is getting the drainage very fast when I am using this application if I use for 5 to 10 minutes my battery get drained more than 15 to 20%</p><p>Development team need to see why it is taking so much battery power</p><p>Also my application got crashed one time for which I have sent the report to the developers</p><p>In any case this application is worth having with a blind people and it is a very good concept thanks to Mr Shekh </p><p>I have seen his video more then One hear back and I just now got the app so I am very happy and excited to use this app&nbsp;. Got it from <a href="https://appvn.vip/appvn-ios-download/&quot; target="_blank">appvn ios</a> store.</p>

Windows Intelligence In Your Inbox

Sign up for our new free newsletter to get three time-saving tips each Friday

"*" indicates required fields

This field is for validation purposes and should be left unchanged.

Thurrott © 2024 Thurrott LLC