- 3 months ago
In this Unity tutorial video, I will show you how to pass audio track data to shaders and how to use it to make a nice Audio reactive effects, by creating a simple Music Visualizer.
Resources!
Project files https://www.patreon.com/posts/133980076
Do you want to support my work? The best way to support this channel is to buy my game on Steam. https://store.steampowered.com/app/2527340/Cosmic_roads/
Resources!
Project files https://www.patreon.com/posts/133980076
Do you want to support my work? The best way to support this channel is to buy my game on Steam. https://store.steampowered.com/app/2527340/Cosmic_roads/
Category
📚
LearningTranscript
00:00in this video you will learn how to pass the audio data to your shaders
00:03and how to use it to make good looking effects in unity like the simple music visualizer
00:12by the end of the video you will be able to make your shader effects to interact with audio
00:17for the tutorial i'm using unit 6 and my project is set up in urp
00:23all right let's get started all right so i have this scene now first let's create a quad
00:30then i don't need this collider so let me remove it now if it was the first episode of how to write
00:39shader series i've told you that 99 of the time you can get away with unity shader graph
00:45unfortunately this scenario falls into that one person so we have to write the shader and i'll
00:52tell you why in a bit but first let's create a shader so create shader and let's go unlead shader
01:01i'm going to call it music visualizer
01:08and let's open up our shader all right now we have the shader here now if you've never written any
01:14shaders in unity before i do have an entire video series about that i recommend you go watch that
01:20first and then come back here because here i'm going to go a little bit faster all right so first
01:27i'm going to change the category to custom and i don't need this property for now yeah this is all cool
01:37and instead of cg program i'll go hlsl program then okay these two names i can work with them so
01:46that's all cool that's all cool and then i don't need fog for now and instead of including unity cg
01:55we will go packages slash unity render pipeline universal then shader library and core dot hlsl
02:07then i like to call this app data attributes so i'll rename it to attributes
02:12yeah okay no problem then i like to call this vertex position in local space
02:26okay okay i don't mind then i like to call this v2f structure wearings
02:33yep then we don't need fog so delete and here i like to call this vertex position in clip space
02:49and next let's move it on top
02:54then we don't need this main texture and sampler
02:58color then in the vertex shader for our clip space position simply go transform object to h clip
03:12and then pass in our attributes position and local space
03:17and for the uv simply use our attributes uvs
03:26and again i don't need fog so let me remove it
03:31then for the fragment shader instead of fixed for i'll simply go float for
03:38then i like to call this integer in so i'll simply rename it to in make sure you write capital i because
03:51otherwise it will be a keyword and i don't need this we don't need fog and let's define a vector 3
04:00call call for color and for now let's visualize our uv so float3
04:08input.uv and 0 and instead of returning cold directly i'll go float4
04:15down my color and the alpha which will be 1 in this case and then instead of ncg i'll simply go and hlsl
04:24all right save okay i have this shader now let's create material from that
04:33let's also call it music visualizer material
04:40and i'm going to apply it onto this quad and our shader is working and you can see that for the uvs
04:47i have origin 0 0 here and it goes up to 1 1 here
04:51all right now first i want to create multiple bars and for that i'm going to create a new
04:57function which will return a float i'm going to call it bars
05:02and it will take our original uv as an input add one float input for height
05:10and then i'll simply go return step of uv dot y and height
05:18and the step function will basically go if this input is less than this edge value
05:28we will get zero otherwise we will get one so in this case if i pass in height of 0.3 i will get white
05:36value from 0 up to 0.3 and then i will get black values about that okay let's let's just simply call
05:45our function so simply bars of our original uv and the height of 0.3
05:56okay save and let's head back to unity
06:00so you can see that upper uvs y is zero in the bottom and it goes up to one so
06:08up to 0.3 i will have white values and about that we will have black values now this is simply a single
06:16bar i want multiple of them and i want to control that from the inspector as well so for that i'm
06:24going to go up to this property block and going to define one property so i'm going to go underscore bar
06:33bars and let me put in the label of bar count
06:42and its type will be integer in here
06:47and let's set the default to 64. and this will simply create an input slot in the inspector
06:56our hlsl program doesn't know what this is so let's define it here as well and put that let's make
07:04some room here and here we should define it in a single c buffer blocks i'll go c buffer start
07:12then unity for material and c buffer end and here i'll go float passing the exact name bars
07:27yep then in my bars function first i'm going to go multiply another step and then let me
07:37add some tiny gaps i'll go 0.2 and then my uvs x so this will go if the uvs x value is less than this value
07:52i i'll get the black value at the left side and at the right side i will have white value so it will
07:58create a gap here okay let's just save and see so you can see that that created a gap
08:08now this is single bar we want multiple bars so for that i'm simply going to multiply this x with my
08:17bars property so now my uvs x will go from 0 up to 64 but what i want is i want this this to go from
08:310 up to 164 times so i can do that by simply going fraction of the entire thing and fraction function
08:41will simply return the fractional part of a number so if i pass in 10.03 it will return 0.03
08:50okay let's save and see so now i have 64 bars and if you don't believe me let me go
09:02and dial down the bar count to five and you can see that one two three four five we have five bars so
09:10oh it's working all right now i have the bars but i don't want them to be all white i need to color them
09:21so in my fragment shader here i'll simply go call multiply equals float three and let me just pass in some
09:32random gradient color so let's try sign off uv's x plus 0.02 and then let's go cosine of
09:46uv's y plus two and for the blue channel i don't know let's try 0.3 let's save and see
09:58yep okay i think this is a nice color but anyways you can always tweak this okay all right now that's
10:11it for our shader for now all right now in unity i have this audio file which has two channels let's
10:18just play it and to use this first let's create an audio source in our scene
10:28okay let me also call it music visualizer
10:38and for the audio resource simply assign our audio file then let me check this loop property so it
10:49will loop continuously and make sure you set the play on awake to true so as soon as i hit play audio
10:56starts playing and yeah that's pretty much for our audio source and now we have to read the frequency
11:06data from this audio and pass into our shader and for that we have to write a c sharp script so i'll go
11:14create mono behavior script and let me call it i don't know music visualizer
11:26controller
11:31and now unity will compile it so we have to wait okay it finally compiles let's assign it to our music
11:39visualizer and then double click to open our script in the script first let's grab the material reference
11:47i'll simply go serialize field so we can access it from the inspector then go private material material
11:57yeah that's fine then in unity select our music visualizer and there we have those material slot let's
12:06assign our music visualizer material there then in our script we also need the audio source reference
12:14and for that i'll go private audio source let's also call it audio source and in the awake method
12:27let's grab the reference for that so i'll go audio source equals get component of audio source
12:35and now to read the audio data i'm going to create a float array so here let's go private float array
12:45let's call it data for now and in the awake method i'm going to initialize it so data equals new float array
12:56with the size of 256 and this basically defines the sample size you want to read at any given point
13:07also you have to make sure that the area size is power of 2 so 2 4 8 16 32 etc also this area size must be
13:18greater than 64 and should be greater than 8 192 and now i think i will be in the way so let's just open this explorer
13:32so you can see better the code and now here let's go update
13:37in the update method we will populate our array with the audio samples and for that audio source has a nice method
13:52called get spectrum data that that was the first one and then we have to pass in our array so data which
14:03will be populated then we have to pass in the channel left or right or one or two so let's pass in the first channel
14:14and then we had to pass in the fft window type which basically means how you want to read the audio samples
14:22or how you want to access the audio frequency and let's let's use blackman
14:31it doesn't really matter so once this line executes we will have data in our data array
14:39now we have to pass that array data to our shader and there we have a problem
14:46well let me head back to my shader so in shader lab there is no way to define float array in the properties
14:53block so i can't define that property here
14:57here fortunately this property block is only there so we can access the property from the inspector
15:05the actual properties are defined in the hlsl blocks so in the c buffer block i'll simply go float
15:13array sorry here i have to go float
15:17of frequency let's call the data frequency and i have to assign the length of the area here as well
15:29so again 256 all right now that i have float array in my shader let's simply copy this name
15:39head back to our c-sharp script and then i'll simply go material dot set float array
15:49then my variable name which was frequency and then i have to pass in the data
15:59and just like that we can pass the array data to our shader in the shader we don't have to define
16:07the property in the property block and a lot of people don't know about this so if you knew
16:12let me know in the comments and that was the reason we had to write the shader because
16:18in share graph there is no way that you can define float array at least i don't know any way to do it
16:26all right now that we have our audio data in our shader instead of statistically defining this 0.3
16:33height to all our bars let's sample our array or actually let's access our array so for that i'm going
16:42to go float h for height equals let's go frequency and then i have to pass the index to read the data
16:53and for that i'm simply going to use
16:58my uv's x multiply my bars by the way i'm carelessly writing like this but you have to make sure that this
17:09bars value don't exceed the array size otherwise it will return an error and here this value will go
17:18from 0 up to 64 but this this is an index so it will be automatically converted to int so it will go
17:29from 0 up to 63 for all of this bars and just like that we have a different frequency in this h for
17:42every bar then instead of passing this 0.3 statically i'm simply going to pass h let's save and
17:51head back to unity and let's display and yep our shader is working correctly but now what i want is i want this bar
18:02to be in the middle and i want them to go in both up and down direction let's just stop and in the shader
18:13in the bar function let's go let's let's simply modify over uv so for uv's y i'm going to go
18:27minus equals 0.5 by doing this it will go from negative 0.5 to positive 0.5
18:34and i want that to go from minus one to one so instead of going minus equals i'll simply go uv dot
18:45y minus 0.5 wrap the entire thing in the brackets and simply go multiply two so with this my uv's y will
18:55go from minus one to one and basically what i want is i want to reflect the top part in the bottom so here
19:03i also want positive values and for that i can simply use absolute of this okay let's save and see
19:12okay it will be weird but let's just play so now i have my bars in the middle and the top part is
19:20reflecting at the bottom as well now you can see that here i don't see any bar that's because this
19:31h value is zero there so let's simply add some minor value on top let's save and see
19:43that's too big i think but let's see yeah that's too big so instead of 0.1 i'll simply go 0
19:522 maybe
19:58again too big again this is a tweaking part so feel free to tweak to your heart's content
20:04there we have a nice music visualizer i hope you can take it from here for example with box sdf you
20:10can make this effect glow with circle sdf you can get something like this
20:15and if you made it this far hit that like button if you're new here subscribe and if you wish to
20:24support me take a look at my patreon that's it from me and i'll see you in the next one
Recommended
19:47
|
Up next
18:21
16:53
11:14
11:28
11:33
13:05
17:26
17:50
17:51
5:13
0:30
19:23
5:43
5:06
Be the first to comment