An Introduction to Game Audio Scripting in Unity (Part 2)

Hey hey! This part two of my intro to scripting for sound people. If you have not read it yet, head here for part one. All the code can be found as a fully functioning Unity project (version 5.5+) on Github. The Github repo reflects the latest state of the project, so if we are referencing parts of code which have changes simply checkout a previous version of the project (a simple guide to git). For the complete code and bug fixes please check the github repository. The code snippets in this article are meant as reference.

In this article we’ll look at:
• Instantiating GameObjects (based on prefabs);
• Creating an audio data format based on ScriptableObjects (instead of prefabs);
• Handling volume in dB and pitch in semitones;
• Adding sound, volume and pitch randomisation;

To get started, let’s set up a little script that references a sound prefab and instantiates it when needed. In a game environment this could be a script that controls the behavior of an object which needs sound for a specific action. In our case, since we don’t have a game to hook into, let’s simply create a GameObject that spawns a sound on Start. In a game this could be used to trigger an ambience loop. Alternatively, it could be attached to a prefab of a prop for the game that needs a spawning sound.

To get started, we’ll create a new C# script called AudioOnStart which will simply play a sound on the start method. Create a public field for a GameObject where we’ll store the prefab for our sound to be played. Well use the Instantiate method; a generic method which takes a Type to be instantiated as well as the prefab to be copied (more info on generic methods at MSDN).

public GameObject SoundPrefab;

void Start ()

{

    	if (SoundPrefab != null)

    	{

        		Instantiate<GameObject>(SoundPrefab, this.transform);

    	}

}

The AudioSourceController we created last time automatically starts the sound in the Start method. Now that we have a script to do just that, let’s hand over the responsibility of starting playback to the AudioOnStart script and remove the Start() method from the AudioSourceController.

The Instantiate method returns a reference to a newly created GameObject. We can store that reference in a local GameObject variable in the Start method so that we may use it. Next we need a reference to the controller. We’ll use the GetComponent method to find the controller script and then call its Play method to make our sound play again. Get component is yet another generic method. This time we’ll specify the AudioSourceController class.

void Start()

{

    	if (SoundPrefab != null)

    	{

        		GameObject go = Instantiate<GameObject>(SoundPrefab, this.transform);

        		AudioSourceController soundInstance = go.GetComponent<AudioSourceController>();

        		soundInstance.Play();

    	}

}

Alternatively, we could also skip storing the GameObject and AudioSourceController in local variables and call all methods on the result of the previous one, but it looks a bit messy, and easily readable code is far more maintainable.

Instantiate<GameObject>(SoundPrefab, this.transform).GetComponent<AudioSourceController>().Play();

Pooling & Scriptable Objects

This way of instantiating objects works fine, but is rather limited. Having to create GameObjects for each sound is a rather costly operation in Unity, and creating new sounds involves creating an empty GameObject, attaching a script to it, then dragging it into a folder in the Project tab. Let’s take a look at another approach.

Instead of instantiating GameObjects we can store our sound behaviour in a ScriptableObject (Unity Documentation on this). This is a nice way to package up our audio behaviour but since the ScriptableObject is file on disk and not a component in the game world we are not able to attach neither the AudioSource nor the AudioSourceController we need to play the sound back. So instead we’ll use a technique called pooling to create a repository of empty AudioSourceControllers and take them out of the pool whenever we want to play a sound. The script we’ll create to handle this for us will be the AudioPoolManager. Once a sound is done playing, it goes back into the pool until it’s needed again. The audio data will only be responsible for storing behaviour playback volume and pitch or if the sound is supposed to loop.

To start off, the AudioPoolManager needs a way of keeping track of multiple AudioSourceController scripts. We’ll use a List, which is a data structure accessible by using the System.Collections.Generic namespace at the very top of the script.

Next, we’ll need a way to get controller scripts out of the pool. This GetController method will check the list of available controllers, and if any are found it will remove one from the list and return it. If the list is empty, it must create a new GameObject, add an AudioSourceController component, and then return the newly created controller.

public class AudioPoolManager

{

private List<AudioSourceController> _pool = new List<AudioSourceController>();

public AudioSourceController GetController()

{

    	AudioSourceController output = null;

    	if (_pool.Count > 0)

    	{

        		output = _pool[0];

        		_pool.Remove(output);

        		return output;

    	}

    	else

    	{

        		GameObject go = new GameObject("AudioController");

        		output = go.AddComponent<AudioSourceController>();

        		return output;

    	}

}

}

To get access to the pool from any script without a direct reference, we can implement a simple Singleton pattern. This will make sure that there will only be one single AudioPoolManager at any time.

To do this we can use a private static field and a public property of type AudioPoolManager where the Getter will check if there is an instance of the pool manager already. If the static instance is null it will create one, otherwise it will simply return the instance. Preferably we put this at the very head of a class definition so it’s immediately clear we are dealing with a Singleton class when opening up the script in the editor..

private static AudioPoolManager _instance;

public static AudioPoolManager Instance

{

get

{

        		if(_instance == null)

        		{

			Debug.LogWarning(“Instanciating AudioPoolManager.”);

            		_instance = new AudioPoolManager();

        		}

        		return _instance;

}

}

We’ll use the “static” keyword so that we can access the pool without storing a reference to it. It also makes sure that the field is the same in every instance of the script. If we only access the pool manager via the Instance property, it will only ever create one instance and return that every time we need access to the pool. There is a slight overhead to this, so if we need access to the pool over and over from the same script, it makes sense to cache the pool instance in a private variable (see below). Check out other design patterns here.

Now that we have the pool manager set up, we need to setup our data format. Let’s create a new class called AudioData which inherits from ScriptableObject instead of the MonoBehaviour default. For now we can move the properties which were stored in the AudioSourceController script. Deleting these properties in the controller script this will break our previously setup prefabs, so beware! After removing those properties we will also have to remove the SetProperties call in the controllers Play method.

Then, we will have to add an attribute above the class definition so that we can create AudioData objects in our project. This will enable us to use the Assets/Create/AudioData menu option as well as the right-click create option in the project tab. Now, all we need is a way to play the sound back. To get this going, we can simply add a public Play method to the AudioData itself which will get an AudioSourceController from the pool, set its properties, and call the Play method before returning the controller.

using UnityEngine;

[CreateAssetMenu()]
public class AudioData : ScriptableObject

{

	public AudioClip Clip;

	[Range(0, 1)]
	public float Volume = 1f;

	[Range(.25f, 3)]
	public float Pitch = 1f;

	public bool Loop = false;

	[Range(0f, 1f)]
	public float SpacialBlend = 1f;

public AudioSourceController Play()

{

    	AudioSourceController controller = AudioPoolManager.Instance.GetController();

    	controller.SetSourceProperties(Clip, Volume, Pitch, Loop, SpacialBlend);

    	controller.Play();

    	return controller;

}

}

We can adjust The aAudioOnStart class to hold an AudioData reference instead of a GameObject, and the Start method will become a whole lot simpler as a result:

using UnityEngine;

public class AudioOnStart : MonoBehaviour

{

	public AudioData Sound;
	void Start()

	{

    		if (Sound != null)

    		{

        			Sound.Play();

    		}

	}

}

What’s missing now is a way to get sounds back into the pool. First we need a public PutController method on the AudioPoolManager to store the controller in the pools list. This method will make sure the list does not already contain that specific controller, and then add it to the list.

public void PutController(AudioSourceController controller)

{

    	if (_pool.Contains(controller) == false)

        		_pool.Add(controller);

}

Next, we will implement a public Stop method on the controller script which stops the AudioSource from playing back and then puts the controller back into the pool:

public void Stop()

{

    	_source.Stop();

    	AudioPoolManager.Instance.PutController(this);

}

As we are dealing with games and they mostly happen to have objects moving on the screen it’s time to implement spatial settings. First, we need a way for the controller script to access its position which is being determined by the Transform component present in each GameObject. We can get the GameObject’s Transform in the Awake method of the controller and cache it in a private variable. This is a good idea since calls to the Unity API often have a little processing overhead for all sorts of error handling and safety.

If we have sounds following GameObjects we’ll want to update the position every frame. When we start playing many sounds at once, the savings we’ll achieve by having this call once per controller will add up quickly.

private Transform _transform;

public void SetPosition(Vector3 position)

{

    	_transform.position = position;

}

Now it probably makes sense to pass the position before we initially play the sound, so we’ll pass a Vector3 parameter to the Play method of the AudioData and set the position right before the Play method call of the controller.

public AudioSourceController Play(Vector3 position)

{

    	AudioSourceController controller = AudioPoolManager.Instance.GetController();

    	controller.SetSourceProperties(Clip, Volume, Pitch, Loop, SpacialBlend);

    	controller.SetPosition(position);

    	controller.Play();

    	return controller;

}

We could also add an Overload to the Play method to pass in a Transform instead:

public AudioSourceController Play(Transform parent)

{

    	AudioSourceController controller = Play(parent.position);

    	controller.SetParent(parent);

    	return controller;

}

We are calling a SetParent method on the AudioSourceController.

public void SetParent(Transform parent)

{

    	_transform = parent;

}

The reason we handle the Transform separately and don’t just set the position and be done with it is that we can now check whether the Transform has moved and update the sound’s position in space accordingly. For this purpose we can use MonoBehaviours LateUpdate method which gets called at the end of every frame. This is so the sound updates its position after the entire game simulation happened and all the other GameObjects have been moved and rendered at their new locations.

This update method is also a great place to check if the sound is still playing. If not, and if it’s not in the pool, we want to call the Stop method. For this purpose we also declare a private boolean variable which is set to true when the Play method is called.

void LateUpdate()

{

    	if (_claimed && _source.isPlaying == false)

    	{

        		Stop();

    	}

    	if(_parentObject != null)

    	{

        		_transform.position = _parentObject.position;

    	}

}

At this point it makes sense to implement a Reset method to call every time the controller script stops. This will handle setting the parent Transform to null and resetting the claiming state.

private void Reset()

{

    	_parentObject = null;

	_claimed = false;

}

Updating the position of the sound means a call to the Unity API every frame. We should actually first check if the new position is any different from the old one, but checking the position is itself call to the Unity framework. Instead we will cache the current position in a private Vector3 field and update the sound’s Transform and cached position only if it has changed.

Utility Functions

Before we randomize volume and pitch of our sounds, let’s look at Unity’s documentation of the AudioSource. The volume in Unity is described as a floating point value between 0 and 1. This can be understood in terms of an amplitude factor. Multiply the amplitude by 1 and we playback a sound at unity gain. If the volume is 0.5, the amplitude is halved which results in a volume reduction of 6dB. To decrease the volume by another 6dB simply set the volume to 0.25 and so on. The volume in Unity’s AudioSource property behaves in a linear fashion as opposed to logarithmically; the way sound designers are used to interacting with volume. Another side effect is that if we randomize the volume factor by an amount of 0.1, this random factor means very little if our volume sits at around 1, but can have a huge impact when the initial value is lower. To solve this, we’ll create a helper function to randomize the volume by a certain Decibel amount instead.

A quick web-search spits out plentiful results for converting gain factors to decibels. The conversion algorithm i based on this one here, which results in the following code:

using UnityEngine;

public class AudioUtils

{

	public static float DecibelToLinear(float dB)

	{

    		if (dB > -80)

        			return Mathf.Clamp01(Mathf.Pow(10.0f, dB / 20.0f));

    		else

        			return 0;

	}

	public static float LinearToDecibel(float linear)

	{

    		if (linear > 0)

        			return Mathf.Clamp(20.0f * Mathf.Log10(linear), -80f, 0f);

    		else

        			return -80.0f;

	}

}

Here is a quick refresher about the Decibel unit.

There is a minimum threshold of -80dB to avoid unnecessary calculation at very low volumes and each conversion is wrapped in a Clamp function to make sure my output is always within a usable range. We are using static function so the Conversion can be called without having to hold a reference to an instance of the Utility class.

The same issue arises with the AudioSource’s pitch property. A pitch of 1 translates to normal playback speed, 0.5 translates to half the playback speed meaning -12 semitones and so on. Unity also supports negative pitch which means the sound will play backwards and if the pitch is set to 0 the sound will start playing but never progress which means it is taking up a voice that will play back that one sound forever. Another source for neat audio bugs ;)

Finding formulas to solve this is a bit harder. I believe I got the initial idea for this in “The Audio Programming Book” (MIT press). These problems often have been solved before. Do some research and borrow exchange ideas with other audio coders. The power of collaboration!

So here is:

private static float twelfthRootOfTwo = Mathf.Pow(2, 1.0f / 12);

public static float St2pitch(float st)

{

    	return Mathf.Clamp(Mathf.Pow(twelfthRootOfTwo, st), 0f, 4f);

}

public static float Pitch2st(float pitch)

{

    	return Mathf.Log(pitch, twelfthRootOfTwo);

}

We’ll stick that into the AudioUtils class as well and move on.

 
Randomisation

Now that we have a way to turn our volume and pitch values into a randomizable format we can add those properties to our audio data. First we need to use the new number format but we can add some random ranges at the same time.

public float Volume = 0f;

[Range(-20, 0)]
public float RandomVolume = 0f;

[Range(-24, 24)]
public float Pitch = 0f;

[Range(0, 12)]
public float RandomPitch = 0f;

We need to adjust the Play() method as well to translate from the AudioData to the format the AudioSource in Unity expects. To generate Random numbers we’ll use the UnityEngine.Random.Range(float min, float max).

float volume = AudioUtils.DecibelToLinear( Random.Range(-80, Volume + RandomVolume) );

float pitch = AudioUtils.St2pitch( Random.Range(-80, Pitch + RandomPitch) );

controller.SetSourceProperties(Clip, volume, pitch, Loop, SpacialBlend);

We can simply generate a random number between “0” and the random range by using the Random.Range() method in the AudioData.Play() call like this:

float volGen = AudioUtils.DecibelToLinear( Random.Range(Volume + RandomVolume, 0) );

float pitchGen = AudioUtils.St2pitch( Random.Range(Pitch - RandomPitch, Pitch + RandomPitch) );

controller.SetSourceProperties(Clip, volGen, pitchGen, Loop, SpacialBlend);

Now when we play the sound it will randomly generate a pitch and a volume and apply those to the AudioSourceController every time a new Play() call comes in.

Randomizing volume and pitch only gets us so far so let’s try to have some variation in samples as well. First we need a way to store multiple sound files in the AudioData. For this we’ll use a list of type AudioClip.

List<AudioClip> Sounds = new List<AudioClip>();

To be able to declare a variable of type List we need to include the System.Collections.Generic namespace.

using System.Collections.Generic;

Next we’ll need to get a random clip from the list but first we’ll check if the list is not empty. The index defaults to “0” which will return the first item in the list unless the list is longer than one item in which case a random index is generated. Many data structures start with an index of 0 and count up from there.

private AudioClip GetClip()

{

    	if (Sounds.Count == 0)

    	{

        		Debug.LogWarning("AudioData does not contain any AudioClips.");

        		return null;

    	}

    	int index = 0;

    	if (Sounds.Count > 1)

    	{

        		index = Random.Range(0, Sounds.Count - 1);

    	}

    	return Sounds[index];

}

Now we can get rid of the single Clip field and use GetClip() to update the AudioData.Play() method.

controller.SetSourceProperties(GetClip(), volGen, pitchGen, Loop, SpacialBlend);

You will notice that the AudioClip we assigned to our AudioData previously has been replaced with a list of sound represented like this:

If we click on the “foldout” arrow we can set the length of the list and drop a different sound into each AudioClip field.

And that’s that! We now have a bunch of randomization options which will increase the mileage of our audio assets and reduce the “fakeness” feeling of our game tricking the brain into buying into our world building efforts a bit more.

Here are a couple of ideas on where one might improve on this system now: It would be nice to keep track of the last played index and make sure it is not repeated or even a list of previously played indexes to check against. The AudioSource still has a lot of interesting properties we are not accessing yet like minimum and maximum distance or distance falloff curve (Unity Documentation on the AudioSource component) which would add a lot of functionality and control to the tool. It’s also worth reading through the manual section on the AudioSource component as there is a lot of valuable information in there. Also the AudioPoolManager should maybe create a minimum size pool on Awake() so the cost of creating new GameObjects does not have to be paid during game play and maybe even limit the amount of objects it can create so there will never be more than – say 64 voices in a scene at any given time. To make sure we can play this many sounds at once we also have to check the Project’s audio manager (Unity Documentation)

Next time we will take a look at doing work over time using Coroutines – first and foremost fading sounds in and out. We’ll also investigate custom inspector to make it easier to use out AudioData class, add drag and drop support and other neat features. Something that might be interesting but may be a bit over scope for the next tutorial would be a RTPC system (real time parameter control) to change a sound’s behaviour as it is playing.

Hopefully you found this article interesting and you will experiment and play around by adding more properties and functionality to the AudioData class.

love

Maris
 

Please share this: