Steam Audio Unreal Engine 4 Plugin

Steam® Audio, Copyright 2016 - 2017, Valve Corp. All rights reserved.

Introduction

Thanks for trying out Steam Audio. It is a complete solution to add 3D audio and environmental effect to your game or VR experience. It has the following capabilities:

How Steam Audio Works

This section describes the various parts of Steam Audio, focusing on the way in which the computational load is divided between multiple threads. Steam Audio interacts with three main threads:

  1. Game Thread. This thread controls the game state, and sends this information to the Simulation Thread. This thread is managed by the game engine, and runs as fast as the game engine chooses to execute it, which might be 60 Hz assuming vsync is enabled.

  2. Simulation Thread. This thread actually carries out the sound propagation simulation, and performs the bulk of the computational work. It uses source and listener information provided by the Game Thread, and calculates an impulse response for use by the Rendering Thread. This process involves ray tracing. This thread is managed internally by Steam Audio, and runs as fast as it can, but no faster than the Rendering Thread.

  3. Rendering Thread. This thread applies direct occlusion, 3D audio, and environmental effects to each sound source. Steam Audio inserts DSP code into the main audio thread, to perform convolutions with multi-channel impulse responses. This thread runs at the audio DSP rate, which is typically 1024 samples per frame, and 44100 samples per second.

Integration and Platforms

Steam Audio supports Unreal Engine 4.19 or higher. If you are using Unity, refer to the Steam Audio Unity Plugin Manual. If you are using a different game engine or audio middleware, you will need to use the Steam Audio C API. Refer to the Steam Audio API Reference for further information.

The Steam Audio Unreal Engine 4 plugin currently supports Windows 7 or later, and Android.

Unreal Engine 4 Integration

This chapter explains how to use Steam Audio with Unreal Engine. It assumes that you are using Unreal's built-in audio engine.

Enabling Audio Mixer Functionality

Steam Audio requires the latest Audio Mixer functionality available in Unreal Engine 4.19 onwards. In Unreal 4.19, this functionality is disabled by default. To enable it, you must first create a shortcut to the Unreal Engine editor:

  1. In the Epic Games Launcher, click Library.
  2. Click the down arrow next to the Unreal Engine version you want to use, and click Create Shortcut.

You can use any other method to create a shortcut to the editor.

Next, you must modify the command-line arguments used when launching the editor using the shortcut:

  1. Right-click the shortcut to the Unreal editor and click Properties.
  2. In the Shortcut tab, under Target, add -audiomixer at the end of the command line.
  3. Click OK.

Enabling Steam Audio

Before using Steam Audio with Unreal, you must enable it for your project. To do so:

  1. In the Unreal Editor main menu, click Edit > Plugins.
  2. In the left pane of the Plugins window, under Built-In, click Audio.
  3. Scroll the list of plugins until you see Steam Audio, then check Enabled.
Enabling the Steam Audio plugin.

Enabling the Steam Audio plugin.

Enabling Individual Plugins

Unreal allows users to individually configure spatialization, occlusion, and reverb plugins, per-platform. For example, you may choose to use Steam Audio's spatialization plugin on Windows and Android, but use default spatialization on PS4.

To get started on Windows:

  1. In the Unreal Editor main menu, click Edit > Project Settings.
  2. In the Platforms section, select Windows.
  3. In the Audio section, you will see three dropdowns. Change these to Steam Audio.
  4. Restart the editor.
Enabling the Steam Audio plugin.

Enabling the Steam Audio plugin.

3D Audio for Direct Sound

Steam Audio offers the easiest way to add HRTF-based 3D audio for video games and VR experiences. To add 3D audio, without adding any occlusion or environmental effects, follow the steps in this section.

You can apply Steam Audio spatialization to any Actor in your scene that contains an Audio Component. To do this:

  1. In the World Outliner tab, select the Actor that contains the Audio Component you want to spatialize.
  2. In the Details tab, select the Audio Component you want to spatialize.
  3. Under Attenuation, check Override Attenuation.
  4. Under Attenuation Overrides, check Spatialize.
  5. In the Spatialization Algorithm drop-down, select SPATIALIZATION HRTF.
Enabling spatialization for an Audio component.

Enabling spatialization for an Audio component.

This configures Steam Audio to use HRTF-based binaural rendering for this sound. If you want to fine-tune how the spatialization is performed, you can configure advanced spatialization settings as described in the next section.

Spatialization Settings

If you want to configure how Steam Audio spatializes a sound, follow these steps:

  1. Select the Audio Component whose spatialization settings you want to modify.
  2. Under Attenuation, click the Spatialization Plugin Settings drop-down.
  3. If you have already created a Spatialization Settings asset, you can select it from the list. Otherwise, click Phonon Spatialization Source Settings under Create New Asset.
  4. Give the newly-created asset any name you prefer, then double-click it in the Content Browser tab.
Adding custom spatialization settings for a source.

Adding custom spatialization settings for a source.

In the window that opens, you can configure the following settings:

Spatialization Method

Select HRTF to use Steam Audio's HRTF-based binaural rendering algorithm (this is the default). You can also select Panning to revert to a standard panning algorithm. This is mostly useful for comparison purposes only.

HRTF Interpolation Method

HRTF Interpolation specifies what interpolation scheme to use for HRTF-based 3D audio processing:

Editing custom spatialization settings for a source.

Editing custom spatialization settings for a source.

Occlusion for Direct Sound

Steam Audio lets you model occlusion of direct sound, i.e., the sound that reaches the listener directly from the source. Before you can use the occlusion functionality of Steam Audio, you must tag and export the scene as described in the next section.

To enable occlusion for a sound source:

  1. Select the Audio Component to which you want to apply occlusion effects.
  2. Under Attenuation, expand Attenuation Overrides.
  3. Check Enable Occlusion.
  4. In the Occlusion Plugin Settings drop-down, select an Occlusion Settings asset if you've already created one. Otherwise, select Phonon Occlusion Source Settings under Create New Asset.
  5. Give the newly-created asset any name you prefer.
Enabling occlusion for a sound source.

Enabling occlusion for a sound source.

This configures Steam Audio's occlusion algorithm with the default settings. To fine-tune how Steam Audio models occlusion for your sound, you can configure advanced occlusion settings as described in the next section.

Occlusion Settings

In the Content Browser tab, double-click the Occlusion Settings asset you created. In the window that opens, you can configure how Steam Audio models occlusion.

Editing occlusion settings for a source.

Editing occlusion settings for a source.

Direct Occlusion Mode

Direct Occlusion Mode specifies how to model sources that are occluded by solid objects.

Direct Occlusion Method

This dropdown is enabled whenever Direct Occlusion Mode is set to anything other than None. Specifies the algorithm used by Steam Audio for modeling occlusion. The options are:

Direct Occlusion Source Radius

Specifies the radius of the sphere to use when modeling Partial occlusion. Ignored if Direct Occlusion Method is set to Raycast.

Physics-Based Attenuation

When checked, physics-based distance attenuation (inverse distance falloff) is applied to the audio.

NOTE
Physics-based attenuation is applied on top of any distance attenuation specified in the Attenuation Settings of an Audio Component. To avoid applying distance attenuation multiple times, either uncheck Physics Based Attenuation in the Occlusion Settings asset for a source, or ensure that no distance attenuation is applied by the Audio Component.

Air Absorption

When checked, frequency-dependent, distance-based air absorption is applied to the audio. Higher frequencies are attenuated more quickly than lower frequencies over distance.

Scene Setup

To use Steam Audio for occlusion and environmental effects in your video game or VR experience, the scene needs to be set up by tagging geometry and specifying acoustic materials for the objects in your scene.

Tagging Geometry

Steam Audio needs to know what objects in your scene should be used to model occlusion and calculate environmental effects. You can specify this by tagging the relevant objects in multiple different ways.

Tagging Static Meshes

Any Actor with a Static Mesh component can be tagged with a Phonon Geometry component:

  1. Select the Actor containing the Static Mesh Component you wish to tag.
  2. Click Add Component.
  3. Select Phonon Geometry.
Tagging a Static Mesh actor with a Phonon Geometry component.

Tagging a Static Mesh actor with a Phonon Geometry component.

Tagging Static Meshes in Bulk

You may also add or remove Phonon Geometry components from all static mesh actors in the scene by using buttons provided in the Steam Audio editor mode.

  1. Click on the Steam Audio editor mode.
  2. Click on Add All or Remove All to add or remove from all actors.
Tagging Static Mesh actors in bulk with a Phonon Geometry component.

Tagging Static Mesh actors in bulk with a Phonon Geometry component.

Tagging Landscape Terrain

In the current version of Steam Audio for Unreal Engine 4, you cannot select individual parts of Landscape terrain to use for physics-based environmental audio simulation. You can either use all terrain in a scene, or none of it. To configure Steam Audio to use Landscape terrain:

  1. In the main menu, click Edit > Project Settings.
  2. In the Project Settings window, under Plugins, click Steam Audio.
  3. Under Scene Export, check Export Landscape Geometry.
Tagging BSP Geometry

In the current version of Steam Audio for Unreal Engine 4, you cannot select individual parts of BSP geometry to use for physics-based environmental audio simulation. You can either use all BSP geometry in a scene, or none of it. To configure Steam Audio to use BSP geometry:

  1. In the main menu, click Edit > Project Settings.
  2. In the Project Settings window, under Plugins, click Steam Audio.
  3. Under Scene Export, check Export BSP Geometry.
Tagging all Landscape terrain or BSP objects.

Tagging all Landscape terrain or BSP objects.

Tagging a Hierarchy of Objects

To tag a large number of Static Mesh Actors that share a common parent object in the World Outliner view:

  1. Select the parent Actor of all the Actors you wish to tag.
  2. Click Add Component, then select Phonon Geometry.
  3. Select the Phonon Geometry component, then under Settings, click Export All Children.
Tagging all children of an Actor.

Tagging all children of an Actor.

NOTE
Tagging an object with Phonon Geometry does not require you to create an additional mesh first. Steam Audio can directly use the same meshes used for visual rendering. Not all objects have a noticeable influence on environmental effects. For example, in a large hangar, the room itself obviously influences the environmental effect. A small tin can on the floor, though, most likely doesn’t. But large amounts of small objects can collectively influence the environmental effect. For example, while a single wooden crate might not influence the hangar reverb, large stacks of crates are likely to have some impact.

Specifying Acoustic Materials

After tagging objects, the next step is to tell Steam Audio what they are made of. You can specify the acoustic material of an object as follows:

In the Phonon Material component that appears, click the Material Preset drop-down and choose a material preset.

Choosing a material preset.

Choosing a material preset.

Acoustic Material for Object Hierarchies

If you have an object with a Phonon Geometry component with Export All Children checked, and you attach a Phonon Material component to it, all its children are assigned the material of the root object. It is possible to assign a child object a different material by attaching a Phonon Material component to the child object.

Global Default Materials

For scenes where most objects are made of the same material, barring a few exceptions, you can save time by specifying a global default material. Then, you only have to add Phonon Material components to objects whose material is different from the default. To specify a global default material:

  1. In the main menu, click Edit > Project Settings.
  2. In the Project Settings window, under Plugins, click Steam Audio.
  3. Under Default Static Mesh Material, select a preset from the Material Preset drop-down.
  4. Under Default Landscape Material, select a preset from the Material Preset drop-down.
  5. Under Default BSP Material, select a preset from the Material Preset drop-down.
Choosing default materials.

Choosing default materials.

Adjusting Material Properties

Instead of choosing a material preset, you can use a custom material. To do so, select Custom from the Material Preset drop-down, either on a Phonon Material component, or for the global default materials. Seven sliders appear, allowing you to customize the material.

Customizing material properties.

Customizing material properties.

Absorption

The first three sliders, Low Frequency Absorption, Mid Frequency Absorption, and High Frequency Absorption, let you specify how much sound the material absorbs at different frequencies. For example, setting High Frequency Absorption to 1 means that the material absorbs all high frequency sound that reaches it. This adds a low-pass filtering effect to any sound reflected by the object.

NOTE
The center frequencies for the three frequency bands are 800 Hz, 4 KHz, and 15 KHz.

Transmission

The fourth through sixth sliders, Low Freq Transmission, Mid Freq Transmission, and High Freq Transmission, let you specify how much sound the material transmits at different frequencies. For example, setting High Freq Transmission to 0 means that no high frequency sound passes through the material. This adds a low-pass filtering effect to any sound passing through the object.

Scattering

The seventh slider, Scattering, lets you specify how "rough" the surface is when reflecting sound. Surfaces with a high scattering value randomly reflect sound in all directions; surfaces with a low scattering value reflect sound in a mirror-like manner.

Scene Export

You must "export" the scene before hitting Play in the Unreal editor or building a player, to ensure scene setup changes are available to Steam Audio. To export:

  1. Click on the Steam Audio editor mode.
  2. Click on Export Scene.
Exporting the scene.

Exporting the scene.

After exporting, you should see the number of triangles in the scene and overall data size. If you wish to view the scene that will be used by Steam Audio, you may export to an OBJ file by clicking Export OBJ. The resulting OBJ file may be loaded and viewed in the Unreal editor.

Environmental Audio Effects

Steam Audio lets you add physics-based environmental audio effects, including reflections, reverberation, and occlusion. Environmental effects, which model the interactions between sound and the environment, are also known as indirect sound, in contrast with direct sound, which models the sound reaching the listener directly from the source.

There are multiple ways in which you can use Steam Audio for indirect sound effects.

Per-Source Sound Propagation

Steam Audio can simulate how the environment affects a sound as it flows from the source to the listener. To enable this:

  1. In the Details tab, select the Audio Component for which you want to enable sound propagation.
  2. Under Attenuation, expand Attenuation Overrides.
  3. In the Reverb Plugin Settings drop-down, select a Reverb Settings asset if you've already created one. Otherwise, select Phonon Reverb Source Settings under Create New Asset.
  4. Give the newly-created asset any name you prefer.
Enabling per-source sound propagation.

Enabling per-source sound propagation.

This configures Steam Audio's sound propagation algorithm with the default settings for this source. To fine-tune sound propagation settings for this source, in the Content Browser tab, double-click the Reverb Settings asset you created. In the window that opens, you can configure the following settings: