Magic Leap Spectator is a mobile application that makes it easy for anyone to capture and share high quality mixed reality content streamed from a Magic Leap 2 device. This powerful application supports a wide variety of use cases, for example:

• Creating video documentation of your Magic Leap 2 applications.
• Creating second-screen experiences for demos.
• Filming “how-to” videos for your ML2 applications.
• Broadcasting your ML2 application content live to audiences at events.

ML Spectator allows you to record your digital content as videos or photos.

This guide is focused on the advanced features and uses for ML Spectator. For the basic guide covering installation and use, or the Developer Documentation see: 

• Basic User Guide
• Developer Documentation

What’s New

New Features:

• An option for Auto-Reconnect has been added to the settings. 

The Camera View

Once a successful connection is established, the Camera View will be displayed in the ML Spectator app. From this view, you’ll be able to see the streamed AR content from the Magic Leap 2 device, as well as access all controls for navigating, recording and adjusting image settings.

Controls

Controls include:

• Disconnect - Ends the current device connection. Users will be asked to confirm if they want to end the current ML Spectator session or not.
• Scan Marker - Re-initiates the Scan Marker flow. This can be used to re-synchronize the device locations, should the AR content appear to be out-of-position at any point (see: Synchronizing Device Locations). The marker does not need to be in the original scan position for this utility. However, the marker still needs to remain stationary between when the mobile device and headset perform their scans.
• Recording Format - Opens the recording settings page and allows you to choose which video formats you would like to use (see: Select Video Codec).
• Auto Focus - Toggles the ability to auto focus on or off. When toggled off, the camera will have a fixed focus at infinity.
• Tracking Lock - Toggles the AR content tracking. When disabled, the AR content will remain still within the camera view until re-enabled. 
• Image Settings - Switches between the main UI view and Image Settings (see: Image Settings).
• Settings - Opens the App Settings overlay (see: App Settings).
• Capture - Captures a photo or starts recording a video (depending on which Capture Mode is currently selected).
• Capture Mode - Toggles between capturing a photo or recording a video.

External Storage (iPhone)

When an external storage device is plugged in, the following controls will pop up: 

Enabling the external storage option, by pressing the External Storage button, will save recordings to the external drive. Disabling it will revert to saving to internal storage. 

External Storage (Android)

To save to an external drive on Android you will need to: 

Step 1: Plug your external drive into your mobile device.

Step 2: Navigate to General Settings and press Mount External Drive. Accept the notification that appears. 

Step 3: Choose a storage location on the external drive. 

Step 4: When prompted, grant storage access. You will be returned to General Settings after. 

Capturing A Photo

To take a photo, ensure that Photo mode is selected in the Capture Mode toggle, and then press the Capture button. A notification will display at the top of the screen indicating that the photo is saved to your device’s photos app.

Capturing A Video

To take a video, ensure that Video mode is selected in the Capture Mode toggle, and then press the Capture button. While recording, the UI will be hidden and the recording duration will be displayed at the top of the screen. To stop recording, press the Capture button again. Videos will be saved to your device’s photos app.

When recording video, you can zoom and pan the image on screen. Pinch the screen with two fingers to zoom out, or pull your fingers apart to zoom in. To pan, drag one finger across the screen. Panning only works when the image is zoomed in. 

Image Settings

Image Settings allow you to customize how your photos and videos will look before capturing them, giving you control over a variety of visual aspects of the video feed and AR content.

Image Settings Controls

Image Settings Controls include:

• Content Mode - Switches between Camera Feed mode and AR Content mode, allowing you to control visual settings for each part of the feed separately. Adjustments made to the Camera Feed mode will not affect how AR content looks. Likewise, adjustments made to the AR Content mode will not affect how real world content is captured, recorded, or displayed. 
• Reset Image Settings - Resets all image settings to their default value. Users will be asked to confirm if they want to reset all settings or not.
• Histogram - Toggles a visual histogram that appears at the bottom of the Camera View behind the Setting Slider. The histogram provides a graphical view of the distribution of colors present in the screen. Peaks and spikes describe an abundance of a given color. This information can be used to adjust the color correction for a scene. A plain white reference (like a wall or large white piece of paper) can be used to adjust optimal settings.
• Image Settings - Switches between the main UI view and Image Settings.
• Visual Settings - A list of image setting options is always shown along the left side of the screen. These options are adjusted by selecting an option from the list and then using the Setting Slider along the bottom of the screen to adjust the strength or value of that setting. 
• Setting Slider - The adjustment control for each Visual Setting. This can be slid to the left or right to adjust each individual Visual Setting.

Visual Settings - Camera Feed

These are the different Visual Settings that can be adjusted for the Camera Feed:

• Brightness - Adjusts the overall brightness of the camera feed between +/- 100.
• Contrast - Adjusts the overall contrast of the camera feed between +/- 100.
• Saturation - Adjusts the overall color saturation of the camera feed between fully saturated color and black-and-white.
• Black Point - Adjusts the contrast level of the darkest pixels within the camera feed to affect the overall contrast of the image.
• White Point - Adjusts the contrast level of the lightest pixels within the camera feed to affect the overall contrast of the image.
• Color Temperature - Adjusts the overall temperature of the image between cool (more blue in appearance) and warm (more red / orange in appearance).
• Color Tint - Adjusts the overall hue perceived across the entire image, cycling through all tints across the color spectrum.

Visual Settings - AR Content

These are the different Visual Settings that can be adjusted for AR content:

• Brightness - Adjusts the overall brightness of the AR content between +/- 100.
• Contrast - Adjusts the overall contrast of the AR content between +/- 100.
• Global Opacity - Adjusts the opacity of AR content. If set to 100, content is fully opaque. If set to 0, AR content is not displayed.
• Dark Strength- Adjusts the relative transparency of dark content based on the max RGB of the color. 
• Dark Cutoff - Adjusts the cutoff value for how transparent dark content can be. Content that falls below the cutoff is not displayed.

App Settings

Clicking the Settings button in the Camera View will bring up the App Settings overlay. Within App Settings you can access:

 

• General Settings - Contains a variety of app settings relating to the overall functionality of the app.
• Audio Settings - Contains audio-specific app settings, and allows you to select different device input sources to record audio.
• About ML Spectator - Displays information about the current app version and links to legal, terms & conditions, and third-party licenses documentation.
• How to Connect - Provides a step-by-step guide on how to connect ML Spectator to get up and running.
• Online User Guide - Provides a link out to the ML Spectator Basic User Guide.

Select Audio Device

In the Audio Settings, you can choose different audio devices to record audio with during a video recording. The drop down menu will list all available microphones.

You can change the Input Volume level for any selected audio device.

Select Video Codec

Under General Settings there is an option to select the type of video codec used when recording video. The native Codec is the default setting. Available codecs are dynamically determined based on your device. ML Spectator supports ProRes on iOS devices. Available codecs are: 

• H264
• H265
• Apple ProRes 4444
• Apple ProRes 422
• Apple Pro Res 422 Proxy
• Apple Pro Res 422HQ
• Apple Pro Res 422 LT

ProRes codec options are available only on Apple devices with iOS 15.1 or later. 

Image Format

For still images you can select either JPEG or PNG format. 

External Storage Option

For iPhone 15 Pro and 15 Pro Max, if you have an external storage device plugged in, it will be detected by ML Spectator and your recordings will be saved there if the external storage option is enabled. The default is to use internal storage and save to your photos app. A warning will appear if the external storage has insufficient available space. 

Advanced Audio Settings

These settings provide more nuanced control over the input audio source:

• Gain - Digitally increases the volume of the incoming audio source after it is routed to the input volume. This can help when creating video recordings at a distance from a subject where audio is hard to hear, but using this setting can lead to an overall loss in audio quality due to distortion and feedback.
• Magic Leap 2 Device Input - Toggles the recorded audio between microphone and microphone + app audio. If set to microphone, only the audio recorded by your mobile device’s microphone (or external microphone, if enabled) will be saved to the video recording. If set to microphone + app audio, then the audio recorded by your mobile device (or external microphone) will be mixed with any audio present in the AR content being streamed. 

Experimental Features

ML Spectator sometimes contains experimental features. Features included in this menu are temporary experiments and may not remain in the application for long or may significantly change over time. To view and enable the available experimental features navigate to Settings > Experimental Features. Some features may require you to disconnect and reconnect from a session in order to see new options when streaming AR content. 

Broadcasting

A key use case for ML Spectator is to broadcast your Magic Leap 2 application’s content to a larger screen. This is particularly useful at events, demonstrations, or when using your Magic Leap 2 app to educate.

How To Broadcast

Currently, ML Spectator allows 4 primary modes of broadcasting functionality:

• Apple Airplay - You can broadcast ML Spectator via Apple Airplay. This will create a mirrored view of your mobile device screen on a paired display. This can be done with an Apple TV, or other Airplay-compatible devices.
• HDMI - Connecting your mobile device via HDMI to a monitor or TV will also mirror your mobile device’s screen.
• Google Meet - In a Google Meet session you can share your device’s screen to show what ML Spectator is currently viewing. This requires sharing your screen in the Google Meet session from the mobile device that is running ML Spectator.  
• Chromecast  - ML Spectator supports Chromecast.

Before connecting to an ML2 device, initiate your screen share or broadcast in your chosen format. Leaving ML Spectator will disable the device-ML2 connection, requiring you to reconnect to the headset after returning.

Note - mirroring your device screen will show all AR content without the UI elements. 

Device and App Requirements

Mobile Device Requirements

• Apple iPhones and iPads running iOS 17 or later. 
• Android devices: Recommended Android ver. 12 or later.
  • Tested on Google Pixel 7 (a/Pro/Base), Pixel 8, Samsung Galaxy Tab S9, Samsung S24, and Xiaomi Ultra 13. 
• ML Spectator ver 1.3.0 or later.

You may experience degraded performance or incompatibility on older devices or those with less processing power. 

Permissions Requirements

ML Spectator requires a number of app permissions to be granted in order for the application to function and provide a complete experience.

• Access to Photos - Granting access to Photos is required to record or take photos.
• Find and connect to devices on the local network - ML Spectator requires network search permissions to connect with your Magic Leap 2.
• Access to Camera - Access to your device’s camera is needed in order to view the streamed AR content.
• Access to Microphone - Microphone access is required to record audio if recording a video of the ML Spectator session.

Magic Leap 2 Requirements

• Minimum required Hardware: Magic Leap 2
• Minimum required OS version: 1.4.0-dev2

General Setup & Recording Best Practices

To create high-quality recordings you can follow these best practices and tips to get the most out of ML Spectator.

Lining up your AR Content in the Real World

When planning to record a session with ML Spectator, be aware of AR over-draw when staging your AR content within the real world. All AR content will draw on top of everything physically seen in the environment, whether it be walls, furniture, or people (as in this example).

Keep your recordings steady

To increase the professionalism of your recordings, use a handheld Gimbal to improve the overall stability and movement of your shots (such as the DJI OSMO).

Prepare the Environment

In order to ensure the best possible tracking and recording during the ML Spectator session, make sure that the environment is free from clutter and is well lit. Proper lighting, and reducing visual noise from clutter and objects, will help display the AR content properly and prevent tracking loss. 

Keep your Wi-Fi Signal Strong

Select a filming environment that has a strong Wi-Fi signal. If you are recording and trying to sync an ML Spectator session in an area with low signal strength and reliability it will impact your ability to remain connected to the ML2 headset and reduce streaming quality. 

Using the ML2  as a Hotspot

When setting up a peer to peer connection between the ML2  and your mobile device, it is recommended that you use a hotspot. The ML2 Headset is capable of serving as a Wi-Fi hotspot for mobile devices that will provide the required functionality. 

In order to use this feature, follow these steps: 

1) Before launching your ML Spectator compatible app, or launching ML Spectator on your mobile device, go to the ML2 main menu and select the Wi-Fi button. 

2) Activate Enable Hotspot in the Network settings. If the app requires an internet connection, keep the Enable Wi-Fi option selected. Both settings can run simultaneously.

3) Click on Manage in Settings, then Hotspot & Tethering, and take note of the hotspot name and password. These are needed when connecting your mobile device. 

4) Go to your phone’s Wi-Fi settings, look for the hotspot name, and connect to it with the password noted in step 3. Ensure your phone is fully connected to the hotspot before launching ML Spectator. 

5) Launch your ML 2 app and ensure that ML Spectator connections are enabled in the app settings. 

6) Launch ML Spectator on your mobile device and connect to your ML2 under the Online Devices list. The IP address should be in the 192.168.43.XXX range. Once connected, continue your ML Spectator session as normal. 

T’s & C’s

• • Open source software notices: https://www.magicleap.com/ml2-open-source-software-notices
  • Software license agreement: https://www.magicleap.com/software-license-agreement-ml2
  • Privacy policy: https://www.magicleap.com/privacy-policy