00:00Hello, everybody.
00:01In this lesson, we're gonna be talking
00:02all about semantic layers, what they are, how they work,
00:05and what they do for your team,
00:06as well as some of the pros and cons
00:08of even having a semantic layer.
00:16So let's talk a little bit about what a semantic layer is.
00:19It is basically a layer between your raw data
00:21and your end user, which could be you,
00:24or it could be a client or some type of consumer.
00:26Now, when your client or you wants to use AI,
00:29maybe you're just asking a natural language question,
00:31hey, what does our revenue look like this month?
00:34You aren't gonna be hitting directly off of your raw data.
00:38What you're gonna be doing is you're gonna be sending
00:40that query or that prompt to your semantic layer,
00:42which is gonna translate it,
00:44and then it's gonna go to your raw data,
00:46get the correct information, and then give it back to you.
00:49It's basically an AI translation layer, right?
00:52It's translating what you're saying in English
00:54to actual queries or code to go and get the data
00:58that you need and return it properly.
01:00Now, what does this layer actually look like?
01:02In a lot of databases or on a lot of different systems,
01:04they could just be a YAML file,
01:06and it's gonna define your business metrics.
01:08It's gonna define different dimensions,
01:10and it could even define things
01:12like joins between your tables.
01:14So if I'm working in a database
01:15and I see a column that says amount,
01:17and it's in some transactions table,
01:18that may be the final amount that a customer paid.
01:21It could be the amount or the quantity
01:23of something that someone bought,
01:24which are two totally different things.
01:26And what you can do in the semantic layers,
01:27you can define exactly what that column is
01:30or what it means.
01:31If you use AI to hit directly off that raw data,
01:34it may not fully understand the context
01:36of what that amount column actually is.
01:39But when you define it and you have that layer,
01:41once you start querying that data,
01:43you're gonna get the same answer,
01:44and everyone's gonna be on the same page
01:45about exactly what that is.
01:47Now, like I mentioned,
01:48in the age that we're currently in,
01:49and with AI being integrated into basically everything,
01:52it's pretty important to have these semantic layers,
01:55dare I say, needed for most companies.
01:58Now, there are pros and there are cons
02:00to using semantic layers or to having them.
02:03And so let me talk about the pros first,
02:05which we've talked about a few of them,
02:06and I'm gonna focus on the cons next.
02:08The first pro is that you have
02:09that single source of truth.
02:11You've defined your metrics,
02:12you've defined all of these things
02:14so that whenever somebody's asking questions on the data,
02:17you're all gonna get the same thing.
02:18So if you ask how many sales do we make this quarter,
02:21it's gonna understand what that is if you've defined it,
02:23because quarters can be different.
02:26Are you talking about a financial quarter?
02:27Are you talking about a calendar quarter?
02:28If you have the semantic layer, it's going to be defined,
02:31and so you won't have two different numbers,
02:33whether it's looking at a financial or a calendar quarter.
02:36The next thing is that semantic layers
02:37really help with reproducibility,
02:39because that's one of the biggest issues
02:41with using AI in any type of database is reproducibility.
02:45The next thing is that it's gonna make self-service easier,
02:47it's not going to perfect it,
02:49but it's gonna make self-service easier.
02:50The issue has always been with self-service
02:53is that there's always things breaking.
02:55There's always things that are changing,
02:56and so people going and getting data themselves
02:59has always been an issue.
03:01I have seen within different organizations
03:03that I've been working with
03:04is that it has actually helped with self-service,
03:07although it has increased requests as well
03:10for more in-depth kind of digging into data
03:12when certain things don't look exactly right.
03:14And so a lot of those low-level queries,
03:16a lot of those low-level questions
03:18that would typically go to like a business analyst,
03:20a data analyst, or a data scientist
03:22for some of these requests,
03:23those are kind of being taken care of at some level,
03:26but the more difficult questions are needing
03:28a lot more work to kind of dig into these things
03:30to make sure they're really accurate.
03:32But now let's look at the cons,
03:34and there are cons to using semantic layers.
03:36The first big con is that you have to maintain
03:39these semantic layers a lot, and as dimensions change,
03:44as metrics change, as a lot of things change,
03:46which they do in business, you have to keep this updated,
03:49otherwise it goes out of date very quickly.
03:51Now along those same lines, this was a pro before,
03:54how you can update one semantic layer
03:56and you're good to go,
03:56but it depends on how your data does things.
03:59You may have data all over the place
04:01with all different departments,
04:02and you're gonna have to rebuild these out,
04:04and you may have to change these dimensions,
04:06you may have to change these metrics,
04:07depending on the department, or depending
04:09on who you're working with.
04:10Again, that's just a lot of maintenance,
04:12and a lot of things that you're gonna have
04:14to be thinking about for that end user.
04:16The last con is just getting buy-in from everybody, right?
04:20This is kind of like a political thing, right?
04:22You're working with other departments,
04:24you're working with other people within your company,
04:26and you have to get buy-in for these things,
04:29and that's a very real issue.
04:31And so everyone agreeing on definitions and metrics,
04:35and agreeing that we even need semantic layers
04:37at certain levels of our data,
04:40that requires buy-in.
04:41And that's not always the easiest thing to get.
04:44In fact, I would say that might be one of the harder things,
04:47because once you get approved,
04:48sure, you have somebody to maintain it,
04:49and then that's a con,
04:50but even just getting it implemented,
04:53actually creating it and everyone agreeing together
04:56is gonna be one of the biggest challenges
04:57that you face trying to implement something like this.
05:00The smaller your organization,
05:01the easier it's going to be,
05:03but as you get into larger companies,
05:05and as you start working with different departments,
05:07and you guys are all kind of sharing
05:09and using similar data,
05:10that's where it gets really, really tricky.
05:12In my opinion, semantic layers aren't really optional
05:15if you're gonna be using AI,
05:17especially within like a database,
05:19or you're using it for visualization,
05:20or you're trying to implement agentic AI
05:22inside of your code base,
05:23or inside of your database,
05:25or inside of your workflow, whatever it is.
05:26You have to have these semantic layers.
05:29So I don't think it's optional,
05:30but it doesn't mean it's gonna be super easy
05:32to get everybody on board.
05:34There are a lot of politics within business,
05:36and you're gonna see that
05:37when you start trying to implement AI
05:39and implement these semantic layers
05:41within your organization.
05:43These are things that are just gonna pop up,
05:45and you're gonna have to be aware of them,
05:46and you're gonna have to have some type of logic
05:48for why you're doing it certain ways.
05:50With that being said, I hope that that was helpful.
05:52I hope you understand semantic layers a little bit more.
05:55And if you like this video,
05:56be sure to like and subscribe,
05:57and I will see you in the next one.
Comments