Intro to Databricks (Free Edition) | UI Walkthrough - video Dailymotion

Alex The Analyst

Take my Full Databricks Course Here: https://www.analystbuilder.com/courses/modern-data-workflows-in-databricks  Databricks Free Edition allows you to practice and learn to work with big data and AI for Free! It&#039;s an amazing platform to use! In this lesson we create our account and walk through all the features that Databricks Free Edition has to offer.  Get a Databricks Free Account Here: http://signup.databricks.com/?provider=DB_FREE_TIER&amp;utm_source=youtube&amp;utm_medium=video&amp;utm_campaign=AlextheAnalyst ____________________________________________   RESOURCES:  💻Analyst Builder - https://www.analystbuilder.com/  📖Take my Full MySQL Course Here: https://bit.ly/3tqOipr 📖Take my Full Python Course Here: https://bit.ly/48O581R 📖Practice Technical Interview Questions: https://bit.ly/46pDqqL  Coursera Courses: Google Data Analyst Certification: https://coursera.pxf.io/5bBd62 Data Analysis with Python - https://coursera.pxf.io/BXY3Wy IBM Data Analysis Specialization - https://coursera.pxf.io/AoYOdR Tableau Data Visualization - https://coursera.pxf.io/MXYqaN  Udemy Courses: Python for Data Science - https://bit.ly/3Z4A5K6 Statistics for Data Science - https://bit.ly/37jqDbq SQL for Dat

Transcript

00:00What's going on, everybody? Welcome back to another video. Today, we're going to be starting

00:03our Databricks series. Now, Databricks is an amazing platform. I've been using it for many

00:13years, and it's built on top of Apache Spark, which means it is very good at handling large

00:18amounts of data. It's designed for entire teams of data engineers, data analysts, data scientists

00:23to work with data collaboratively. So that means ingesting the data, analyzing the data,

00:27and visualizing data all in one place. In this lesson, we're going to be diving into Databricks

00:31and look at some of the core functionality and features that Databricks Free Edition has

00:35to offer. And then in the next several lessons, we're going to get really hands-on and start

00:39digging into these features and start building things out. With that being said, let's jump

00:42on my screen and get started. Before we jump into the platform itself, I just want to show

00:46you that this is where you sign up. So I'm going to have this link down below. So you

00:50can just click on it. You can create an account and you can sign in. There is no credit card

00:54information that they take. You just literally sign in and you are good to go. But on this

00:58page, you can sign up for the Free Edition or log in, which we'll log in in just a second.

01:02But you can also learn a lot more about Databricks Free Edition and everything that they have to

01:07offer. You can see that they have built-in AI agents and they have AI built-in. And that is

01:12something that we're going to cover in this series. You can also visualize your data and create full

01:16dashboards. And of course, you can interact with your data with Python and SQL, either through

01:21notebooks or, you know, through a code editor. Let's go back up and let's go and sign up for

01:27the Free Edition. All we have to do is log in. You can do that with Google, with a Microsoft

01:31account, or just an email. I'm going to sign in with my Google account. And all I have to do

01:36is

01:36say where I'm from. So this is Alex. We have the Free Edition and I'm from the United States. Let's

01:40go

01:40ahead and click continue. And just like that, we are signed up for the Databricks Free Edition. It was

01:45about as seamless as it can possibly be. Now, there's a lot to cover. Databricks has so many

01:51different features. They have so many different things that you can do from building interactive

01:54dashboards to writing SQL queries and sharing that with your team and being able to work with your

01:58team. You can even build AI agents. There's a lot of things that you can do. And some of this

02:03might

02:03be a little bit intimidating if you've never used a platform like this before. If you're used to just

02:07working something like SQL or R or Tableau, it's a little bit more advanced than that. So there's a

02:13lot of things that they have that are all combined into one place. And so we're going to walk through

02:17a lot

02:17of these things that you see on this left-hand side to see exactly what you can do in Databricks.

02:22Now, I'm going to start at the very top because this is a workspace. And a workspace is basically

02:27the place that you work and you can collaborate in as well. So you can give your teammates access to

02:32your workspace. So if you have data in here, if you have code in here, if you have visualizations,

02:36whatever it is, you can share that with them and they can get access to all of your work. Next,

02:40let's go to catalog. You can think of catalog as like a schema. So if you have a schema in

02:45a database,

02:46you're going to have all of your tables and your views and your stored procedures and everything

02:49underneath it. And this is very similar. So if I go into a workspace or if I go down here

02:54into the samples, we have different databases. And so we can click into these databases and we can look

03:00at all these different tables. And so this is where you're going to have access to view all of your

03:03data

03:03and files and tables within Databricks. Next, let's take a look at jobs and pipelines. This is where we

03:10start getting into some automation. So here we have an ingestion pipeline, an ETL pipeline,

03:14and a job. And each of these does a slightly different thing. If you create an ingestion

03:20pipeline, that's going to be like the extraction of the ETL process. You're extracting data to bring

03:24it in. Then we have our ETL pipeline, which is the entire process of transforming it, loading,

03:29and actually putting it into a table within Databricks. And then we have our jobs. That's

03:34going to orchestrate it. We're going to be able to say, here's when we actually run these pipelines

03:37so that we can time it. We can do it, you know, daily, weekly, monthly, whenever we want to run

03:42these pipelines. So we don't have any created, but when we do, they'll be down here at the bottom.

03:48Next, let's look at compute. Now compute is very simple in the free edition because you don't have

03:53any options here. We just have our serverless starter warehouse. This is what you get for free

03:58with the free edition. Now, if you go to the full Databricks product, then you're going to be able to

04:03kind of customize your compute to your needs. But for this free edition, we have the owner,

04:08we have the size, which is a two X small, and then we have whether it's active or not. Now,

04:13right now we're not running anything. We're not looking at any tables. We're not running any

04:16queries. So it's not active at the moment, but the second that we open something up and start

04:21working with real data, it will activate the serverless compute and we'll be able to use it

04:25for free. One small nuance to this is there actually is one other type of compute. You just

04:29can't see it. And that's for when you were in Python in a notebook, it's called a generic compute.

04:34It's just for that one specific use case. For the most part, you're going to be using this

04:37serverless compute, but I thought it was worth mentioning. Notice when I clicked on compute,

04:41it also took me right down here to the SQL warehouse. This is our SQL warehouse that we

04:46are using. Next, let's go down to the marketplace. Now the marketplace is basically just, hey,

04:52here's a lot of companies that work with us, that we partner with, we have connectors to,

04:57and we have partnerships with, and it allows you to work with them a lot easier. So here we go,

05:01partner connect integration. So if you want to connect to Fivetran, to Power BI, to Tableau,

05:07DBT, Prophecy, you can easily connect to these. I know at my previous job, I connected it with

05:12Power BI a lot. And so these are great connectors where you can just kind of search and you can

05:17say,

05:17hey, do they connect to this tool that we use? And most of the time they do have that connector.

05:22Now the marketplace has other things as well. If we come over here to products, we have things like

05:26tables or files, which is data. You can also search for models and notebooks and all sorts

05:31of other things. So you can come in here and we can search for files. And so these are all

05:36free resources and files that you can search for and use. If you can't find data here in the

05:41marketplace, you can always just go to something like Kaggle and get free data and bring that into

05:45Databricks as well. Now let's come over here and go to the SQL editor. You can see we have kind

05:50of

05:50these larger overarching sections up here, but then we have SQL and then we have data engineering.

05:56Then we have AI and machine learning right now. We're in the SQL section. So right in here,

06:01we're able to come in and we're able to write SQL, just like any other platform.

06:05The neat thing about this though, is that they have AI integrated into it. It helps a lot with

06:10writing some of the base queries that you're going to write. Now we're going to have a whole lesson

06:13on Genie. And so we're going to dive into that and see how you can generate code and do a

06:17lot of

06:17different things with Genie, which is their AI system within Databricks. And Genie just allows you to

06:22get insights from your data, just using natural language, but you'll be able to write SQL right

06:26here in this editor, as well as have multiple tabs and choose what workspace you are working in right

06:31now. We're just on the default, but if you had different workspaces, you'd be able to just click

06:35on those and select where you want your queries to be pointed at. Next, let's take a look at our

06:39queries. So once we get in and we start developing all these queries, we want to save them.

06:44And organization is a big piece of that. I know that I used to have like 20, 30 different kind

06:48of queries that I was saving on my file explorer back in my old job. And then I started working

06:53in

06:54Databricks. I'm like, Hey, I can just save those within this. And then I don't have to email someone

06:58my query. And then they pull it up and copy and paste. Like it's all here within Databricks,

07:02which is really great. Next, let's go look at dashboards. Now these are some sample dashboards

07:07and we're going to have a whole lesson on analyzing and visualizing data within Databricks. So we'll be

07:12able to build something like this. Let's click into one of these dashboards right here. It says my

07:17warehouse is starting this because it's connecting to the data to be able to visualize our data.

07:22So let's get rid of this filter really quick. So this is what our dashboard looks like. And you're

07:26able to create all these different visualizations. You're able to drill down into the data. And this

07:31is all connected to data that is already in Databricks. So it's all in one place. Next, let's go take

07:36a

07:36look at Genie and they have a whole section just for this. And they also have an entire section that

07:42is

07:42just devoted to this warehouse. Now, this is something that a lot of platforms are trying to build

07:47to, but Databricks already has it, which is they have all the data. They have the ability to query

07:52and analyze and visualize data all in one place, but being able to use AI with it to get a

07:57lot of

07:57those things done faster. And so you can ask it questions and it's going to prompt you because

08:00this is kind of a, uh, you know, starter section and you can ask it questions and it's going to

08:06run

08:06these queries and it's going to get you that information really quickly. In one of our future

08:09lessons, we're going to be diving into just Genie. Focus only on what it can do, how it helps you

08:14analyze your data faster and how it can be really useful to you as a user. The next thing I

08:19want to

08:19take a look at is alerts. Now an alert is basically like a trigger. When a condition is met, it's

08:24going

08:24to do something. And so you can customize these alerts and set these conditions and they'll email

08:28you or they'll message you when these conditions are true. Now in this section up here, this is where

08:34we're going to be spending the majority of our time. There are other things like the job runs,

08:39which is where you can create your jobs or just part of these, uh, jobs and pipelines. And then of

08:44course,

08:44the data ingestion as well, which allows you to connect to your data. So if you have a data source

08:49that you want to point at, maybe it's Google analytics, you can just click on this and connect

08:53to your data. Or if you just want to upload a file, you can do that as well. Down here,

08:58we have our AI

08:59and machine learning section. So you can click on the playground. This is another area where you can

09:03interact with the AI within Databricks and you can select some custom parameters as well as the prompts

09:09that you are using. We can also look at experiments. This is where you can build out AI

09:13agents and machine learning models and you can actually test them out. And so this is a fantastic

09:17place where you can actually learn how to use a lot of these foundational models and you can take

09:21the data and you can deploy it and you can work through these issues and really learn hands-on

09:25how to work with them. Naturally, they're going to have AI integrated into all this. So you can work

09:29with the Databricks assistant to get help with all of your coding. So as you're going along, if you get

09:33stuck, you can always get help with your code. As I said before though, we are going to be spending

09:37most of our time right up here. Of course, we'll need to ingest our data, but after that,

09:42we're going to be spending a lot of our time visualizing, see how we can work with our data

09:46and how we can use their integrated AI to be able to do our work faster. At the end of

09:50this entire

09:50series, we'll be building a full project using a lot of the things that we're going to be looking

09:54at in this series. I think building and getting hands-on is the best way to learn. So I cannot

09:59wait

09:59to get started on that. Before we go, I want to give a huge shout out to Databricks for sponsoring

10:03this series. Databricks is an amazing platform that I've been using for many years. And I did

10:07not even know that they had a free edition like this. And so when they said, we have a free

10:12way

10:12for you and everybody else to learn Databricks, I was like, I absolutely want to be part of this.

10:16I was so excited. So thank you again for sponsoring this series. Again, click on the link in the

10:20description to get started using Databricks free edition. And I will see you in the rest of the

10:24series as we get really hands-on and start learning Databricks. If you liked this video,

10:28be sure to like and subscribe, and I will see you in the next video.

10:42I'll see you in the next video.

Intro to Databricks (Free Edition) | UI Walkthrough

Category

Transcript

Comments

Recommended