Usage

Features

  • Display a splash view when the app is loaded for the first time.

    Splash Screen

  • Select an image from the Photo app and send the selected image to AWS EC2 instance via WebSocket. Then, receive response and display the result to the Apple Vision Pro.

    Select Image and Submit Image

  • Use speech-to-text to capture text from user. Then, send the text to OpenAI and display the result.

    Speech Recognition and OpenAI Connection

How to Use

  • Open the visionOS app called Discover on the Apple Vision Pro

  • Click on Select photo button to access the Photo app on the headset

  • Pick a photo and then click on Submit Image send the photo to AWS EC2 and receive response on the headset.

  • Press Speak Now and Start Recording to record speech and convert to text. Select Send Request to send it to OpenAI and receive response.

Live Demo