Steam Audio Unreal Engine 4 Plugin

Steam Audio Unreal Engine 4 Plugin

Steam® Audio, Copyright 2016 - 2019, Valve Corp. All rights reserved.

Introduction

Thanks for trying out Steam Audio. It is a complete solution to add 3D audio and environmental effect to your game or VR experience. It has the following capabilities:

How Steam Audio Works

This section describes the various parts of Steam Audio, focusing on the way in which the computational load is divided between multiple threads. Steam Audio interacts with three main threads:

  1. Game Thread. This thread controls the game state, and sends this information to the Simulation Thread. This thread is managed by the game engine, and runs as fast as the game engine chooses to execute it, which might be 60 Hz assuming vsync is enabled.

  2. Simulation Thread. This thread actually carries out the sound propagation simulation, and performs the bulk of the computational work. It uses source and listener information provided by the Game Thread, and calculates an impulse response for use by the Rendering Thread. This process involves ray tracing. This thread is managed internally by Steam Audio, and runs as fast as it can, but no faster than the Rendering Thread.

  3. Rendering Thread. This thread applies direct occlusion, 3D audio, and environmental effects to each sound source. Steam Audio inserts DSP code into the main audio thread, to perform convolutions with multi-channel impulse responses. This thread runs at the audio DSP rate, which is typically 1024 samples per frame, and 44100 samples per second.

Integration and Platforms

Steam Audio supports Unreal Engine 4.20 or higher. If you are using Unity, refer to the Steam Audio Unity Plugin Manual. If you are using a different game engine or audio middleware, you will need to use the Steam Audio C API. Refer to the Steam Audio API Reference for further information.

The Steam Audio Unreal Engine 4 plugin currently supports Windows 7 or later, and Android.

Unreal Engine 4 Integration

This chapter explains how to use Steam Audio with Unreal Engine. It assumes that you are using Unreal’s built-in audio engine.

Setting up Your Unreal Project

To get started, users must make sure they are running with the newest audio engine, enable the plugin, and enable the plugin on a per-platform basis.

NOTE
For developers using UE4.20-4.24, you will need to manually enable the Audio Mixer Functionality, such as by passing -audiomixer to the Unreal executable on startup.

Enabling Steam Audio

Before using the Steam Audio plugin with Unreal, you must enable it for your project. To do so:

  1. In the Unreal Editor main menu, click Edit > Plugins.
  2. In the left pane of the Plugins window, under Built-In, click Audio.
  3. Scroll the list of plugins until you see Steam Audio, then check Enabled.
Enabling the Steam Audio plugin.

Enabling Individual Plugins

Unreal allows users to individually configure spatialization, occlusion, and reverb plugins, per-platform. For example, you may choose to use Steam Audio’s spatialization plugin on Windows and Android, but use default spatialization on PS4.

To get started on Windows:

  1. In the Unreal Editor main menu, click Edit > Project Settings.
  2. In the Platforms section, select Windows.
  3. In the Audio section, you will see three dropdowns. Change these to Steam Audio.
  4. Restart the editor.
Enabling the Steam Audio plugin.

Spatialization Plugin

The Steam Audio spatialization plugin allows users to apply HRTF-based binaural audio for their games and VR experiences. You can enable Steam Audio spatialization on any Audio Component in your scene. To do this:

  1. In the World Outliner tab, select the Actor that contains the Audio Component you want to spatialize.
  2. In the Details tab, select the Audio Component you want to spatialize.
  3. Under Attenuation Spatialization, check Override Attenuation.
  4. In the Spatialization Method drop-down, select Binaural.
Enabling spatialization for an Audio component.

This configures Steam Audio to use HRTF-based binaural rendering for this sound. If you want to fine-tune how the spatialization is performed, you can configure advanced spatialization settings as described in the next section.

Spatialization Settings Assets

Spatialization Settings assets may be used to configure how Steam Audio spatializes a sound by following these steps:

  1. Select the Audio Component whose spatialization settings you want to modify.
  2. Under Attenuation Plugin Settings, expand the Plugin Settings section.
  3. Add an element to the Spatialization Plugin Settings array.
  4. If you have already created a Spatialization Settings asset, you can select it from the list. Otherwise, click Phonon Spatialization Source Settings under Create New Asset.
  5. Give the newly-created asset any name you prefer, then double-click it in the Content Browser tab.
Adding custom spatialization settings for a source.

In the window that opens, you can configure the following settings:

Spatialization Method

Select HRTF to use Steam Audio’s HRTF-based binaural rendering algorithm (this is the default). You can also select Panning to revert to a standard panning algorithm. This is mostly useful for comparison purposes only.

HRTF Interpolation Method

HRTF Interpolation specifies what interpolation scheme to use for HRTF-based 3D audio processing:

Editing custom spatialization settings for a source.

Occlusion Plugin

The Steam Audio occlusion plugin lets you model the transmission of direct sound from the source to the listener. In particular, Steam Audio allows you to model occlusion, air absorption, physics-based attenuation, and directivity. There are several key configuration steps to be aware of when using the Steam Audio occlusion plugin:

You can enable Steam Audio occlusion on any Audio Component in your scene. To do this:

  1. In the World Outliner tab, select the Actor that contains the Audio Component you want to spatialize.
  2. In the Details tab, select the Audio Component you want to spatialize.
  3. Under Attenuation Occlusion, check Enable Occlusion.
  4. Under Attenuation Plugin Settings, expand the Plugin Settings section.
  5. Add an element to the Occlusion Plugin Settings array.
  6. If you have already created an Occlusion Settings asset, you can select it from the list. Otherwise, click Phonon Occlusion Source Settings under Create New Asset.
  7. Give the newly-created asset any name you prefer.
Enabling occlusion for a sound source.
Creating occlusion settings for a sound source.

NOTE
When your project is configured to use the Steam Audio occlusion plugin, UE4 will ignore all settings in the Attenuation Occlusion section, except for Enable Occlusion.

This configures Steam Audio’s direct sound effect with the default settings. To fine-tune how Steam Audio models direct sound, you can configure advanced occlusion settings as described in the next section.

Occlusion Settings Assets

In the Content Browser tab, double-click the Occlusion Settings asset you created. In the window that opens, you can configure how Steam Audio models direct sound.

Editing occlusion settings for a source.
Direct Occlusion Mode

Direct Occlusion Mode specifies how to model sources that are occluded by solid objects.

Direct Occlusion Method

This dropdown is enabled whenever Direct Occlusion Mode is set to anything other than None. Specifies the algorithm used by Steam Audio for modeling occlusion. The options are:

Direct Occlusion Source Radius

Specifies the radius of the sphere to use when modeling Partial occlusion. Ignored if Direct Occlusion Method is set to Raycast.

Physics-Based Attenuation

When checked, physics-based distance attenuation (inverse distance falloff) is applied to the audio.

NOTE
To avoid applying physics-based distance attenuation on top of Unreal Engine’s native distance attenuation, ensure that Enable Volume Attenuation is unchecked under Attenuation Distance.

Air Absorption

When checked, frequency-dependent, distance-based air absorption is applied to the audio. Higher frequencies are attenuated more quickly than lower frequencies over distance.

NOTE
To avoid applying air absorption on top of Unreal Engine’s native air absorption, ensure that Enable Air Absorption is unchecked under Attenuation Air Absorption.

Global Occlusion Settings

You can access global occlusion plugin settings for Steam Audio by:

  1. In the main menu, click Edit > Project Settings.
  2. In the left pane, under Plugins, select Steam Audio.
Occlusion Samples

The number of rays that are traced from the listener to various points in a sphere around the source. Increasing this number results in smoother transitions as the source becomes more (or less) occluded. This comes at the cost of increased CPU usage.

Reverb Plugin

The Steam Audio reverb plugins allows you to model the transmission of indirect sound from the source to the listener. In particular, there are two types of reverb, source-centric and listener-centric, that you may apply to a source. Source-centric reverb is a per-source effect that models how sound emitted from the source bounces around the environment before reaching the listener. Listener-centric reverb is a global effect that models how sound emitted from the listener bounces around the environment before returning to the listener. Listener-centric reverb is often used as a less expensive approximate alternative to source-centric reverb, although the two effects may be combined. There are several key configuration steps to be aware of when using the Steam Audio reverb plugin:

Enabling Source-Centric Reverb

You can enable source-centric reverb on any Audio Component in your scene. To do this:

  1. In the Details tab, select the Audio Component for which you want to enable sound propagation.
  2. Under Attenuation Plugin Settings, expand the Plugin Settings section.
  3. Add an element to the Reverb Plugin Settings array.
  4. If you have already created an Reverb Settings asset, you can select it from the list. Otherwise, click Phonon Reverb Source Settings under Create New Asset.
  5. Give the newly-created asset any name you prefer.
Adding a reverb settings asset to configure source-centric reverb.

This configures Steam Audio’s source-centric reverb with the default settings. To fine-tune source-centric reverb settings for this source, see reverb settings assets.

Enabling Listener-Centric Reverb

You can enable listener-centric reverb on any Audio Component in your scene. To do this:

  1. In the World Outliner tab, select the Actor that contains the Audio Component you want to spatialize.
  2. In the Details tab, select the Audio Component you want to spatialize.
  3. Under Attenuation Reverb Send, check Enable Reverb Send.
  4. Under Attenuation Reverb Send, change Reverb Send Method to Manual.
  5. Under Attenuation Reverb Send, change Manual Reverb Send Level to 1.0.
  6. Under Attenuation Plugin Settings, expand the Plugin Settings section.
  7. Add an element to the Reverb Plugin Settings array.
  8. If you have already created an Reverb Settings asset, you can select it from the list. Otherwise, click Phonon Reverb Source Settings under Create New Asset.
  9. Give the newly-created asset any name you prefer.
Enabling listener-centric reverb for an Audio Component.
Adding a reverb settings asset ensures that audio will be routed to the Steam Audio plugin. By default, this also opts your Audio Component into source-centric reverb, but you can easily disable it and only use listener-centric reverb.

This configures the audio component to use both listener-centric and source-centric reverb with the default settings. Listener-centric reverb must also be enabled in the global project settings, since it is disabled by default. To enable listener-centric reverb globally and configure other settings, see global reverb settings. If you only want listener-centric reverb for this source, then you can disable source-centric reverb simulation as described in reverb settings assets.

Reverb Settings Assets

To fine-tune source-centric reverb settings for a source, in the Content Browser tab, double-click the Reverb Settings asset you created. In the window that opens, you can configure the following settings:

Configuring source-centric reverb.
Source-Centric Reverb Simulation

This determines what kind of simulation is performed to calculate source-centric reverb.

Source-Centric Reverb Contribution

Increasing this value increases the contribution of indirect sound relative to the overall mix for this source.

Global Reverb Settings

You can access global reverb plugin settings for Steam Audio by:

  1. In the main menu, click Edit > Project Settings.
  2. In the left pane, under Plugins, select Steam Audio.

You can fine-tune various settings that control how Steam Audio simulates physics-based indirect sound. This can help you achieve the right balance between simulation quality and in-game performance. These settings apply to both listener-centric and source-centric reverb (for all sources that use it).

Configuring global reverb settings.
Listener-Centric Reverb Simulation

How to simulate listener-centric reverb, if at all.

Listener-Centric Reverb Contribution

How much listener-centric reverb should contribute to the overall mix.

Ambisonics Order

This determines the directionality of environmental effects. Increasing this increases the compute complexity quadratically. Use zero order Ambisonics if no directionality is needed in environmental effects. Otherwise, first order Ambisonics should provide a good tradeoff between directionality and CPU usage.

Impulse Response Duration

This is the length of the impulse responses to generate, in seconds. Increasing this improves the quality of the simulation, but beyond a certain point (depending on the number of sound sources), may result in audio glitching.

Indirect Spatialization Method

If set to Panning, Steam Audio will apply a standard panning algorithm to render the Ambisonics-encoded environmental effects. If set to HRTF, Steam Audio will decode the environmental effects using an HRTF-based binaural rendering algorithm, for improved spatialization of indirect sound.

Irradiance Min Distance

The minimum distance between a source and a scene surface, used when calculating the energy received at the surface from the source during indirect sound simulation. Increasing this number reduces the loudness of reflections when standing close to a wall; decreasing this number results in a more physically realistic model.

Max Sources

This is the maximum number of sound sources that can have source-centric reverb enabled. For the purposes of this setting, listener-centric reverb counts as a source. For example, if Max Sources is set to 8, and you are using listener-centric reverb, then you can have up to 7 sources that use source-centric reverb.

Real-Time Quality Settings

These settings let you fine-tune how Steam Audio simulates physics-based indirect sound. You can select one of the presets from the Quality Preset drop-down, or select Custom, and adjust the following settings.

Baked Quality Settings

These settings let you fine-tune how Steam Audio simulates physics-based indirect sound. You can select one of the presets from the Quality Preset drop-down, or select Custom, and adjust the following settings.

Baked Reverb

As an alternative to simulating physics-based environmental effects in real time, you can choose to bake them in the Unreal editor. At run-time, the baked environmental effect information is used to look up the appropriate filters to apply based on the source and/or listener position. This way, you can perform much more detailed simulations as a pre-process, trading off CPU usage at run-time for additional disk space and memory usage for the baked data.

Placing Probes

Before you can bake environmental audio effects, you must place probes at all points in your scene where you want to bake environmental effects. A probe is a point at which environmental audio effects are sampled.

Probe Volumes

To create probes, you must first create a Probe Volume, which is a box-shaped region of space in which multiple probes will automatically be generated by Steam Audio. To create a Probe Volume:

  1. In the Modes tab, click Volumes.
  2. Drag a Phonon Probe Volume actor into your scene.
  3. Using Unreal’s built-in translation and scaling gizmos, adjust the position and size of the Probe Volume so that it covers the geometry on/in which you want to generate probes.
Placing a probe volume.

You can create multiple Probe Volumes throughout the scene, Steam Audio will bake environmental effects for all probes in all Probe Volumes.

Probe Generation

Once a Probe Volume has been created, you must generate probes within it:

  1. In the World Outliner tab, click the Probe Volume in which you wish to generate probes.
  2. In the Details tab, select a Placement Strategy (see below).
  3. Click Generate Probes.
Generating probes.

The probes will be displayed as blue dots in the scene.