- 7 months ago
Welcome to AI-3018, where we explore Azure AI Foundry, Microsoft's latest AI platform designed for building, fine-tuning, and deploying AI models at scale. Whether you're an AI enthusiast, developer, or business professional, this session will introduce you to Azure AI Foundry's capabilities, tools, and real-world applications.
🔍 What You’ll Learn in This Session:
✅ Introduction to Azure AI Foundry – Features & Benefits
✅ How to Build & Fine-Tune AI Models with Foundry
✅ Using Prompt Flow & Custom Models for AI Development
✅ Integrating Azure AI Foundry with Microsoft Copilot & Power Platform
✅ Deploying AI Solutions for Enterprise Applications
✅ Best Practices for Responsible & Ethical AI Development
🛠️ Who Should Watch This?
Developers & AI Engineers exploring Azure AI capabilities
Business users & IT professionals integrating AI into workflows
Students & researchers looking to understand AI model training
Organizations wanting to scale AI-driven solutions with Azure AI Foundry
📌 Key Highlights:
✅ Hands-on Demo – Navigating the Azure AI Foundry portal
✅ Step-by-Step Guide – Building & deploying AI models
✅ Real-World Use Cases – AI-powered automation & data insights
✅ Best Practices – Optimizing AI models for performance & accuracy
💡 Kickstart your AI journey with Azure AI Foundry & take your AI projects to the next level!
Explore Our Other Courses and Additional Resouces on: https://www.youtube.com/@skilltechclub
🔍 What You’ll Learn in This Session:
✅ Introduction to Azure AI Foundry – Features & Benefits
✅ How to Build & Fine-Tune AI Models with Foundry
✅ Using Prompt Flow & Custom Models for AI Development
✅ Integrating Azure AI Foundry with Microsoft Copilot & Power Platform
✅ Deploying AI Solutions for Enterprise Applications
✅ Best Practices for Responsible & Ethical AI Development
🛠️ Who Should Watch This?
Developers & AI Engineers exploring Azure AI capabilities
Business users & IT professionals integrating AI into workflows
Students & researchers looking to understand AI model training
Organizations wanting to scale AI-driven solutions with Azure AI Foundry
📌 Key Highlights:
✅ Hands-on Demo – Navigating the Azure AI Foundry portal
✅ Step-by-Step Guide – Building & deploying AI models
✅ Real-World Use Cases – AI-powered automation & data insights
✅ Best Practices – Optimizing AI models for performance & accuracy
💡 Kickstart your AI journey with Azure AI Foundry & take your AI projects to the next level!
Explore Our Other Courses and Additional Resouces on: https://www.youtube.com/@skilltechclub
Category
🤖
TechTranscript
00:00okay so now let's start with the new module which is getting started with
00:13Azure AI Foundry undoubtedly I should mention this guy said this is one of my
00:18favorite Azure AI service and if you are a developer or a data scientist I'm sure
00:24you're also going to love this in this particular module we are going to start
00:28with what is exactly Azure AI Foundry most of the time people know Azure AI
00:34Foundry portal because that's what they have used for deploying large language
00:38models but in this case we'll understand is your AI Foundry with all the
00:42aspects with all the components which it includes and then after this we'll move
00:47forward to grounding a generative AI app with your own data so there are various
00:51things which you can do with Azure AI Foundry one of the thing is providing a
00:55grounding data we'll talk about that after this we have a topic which we need
00:59to understand in depth which is RAG retrieval augmented generation this is one
01:06of the most important topic if you want to associate your own data with your large
01:11language models and then this is going to be very important if you do not want to
01:16waste your time in fine-tuning and retraining of your model after this we
01:20will understand another important topic of Azure AI Foundry which is known as
01:24prompt flows basically these are workflows which are created with the help
01:29of your prompt based logic and after this we are going to use retrieval
01:34augmented generation in prompt flow so basically we will combine RAG and prompt
01:39flow and that is what we are going to discuss finally the last topic of this
01:43module will be deploying a prompt flow now all these things we are going to
01:47understand as a conceptual thing right now and if you are interested in any of
01:52the practical labs associated with this we will surely going to create this for
01:56you in the coming modules now let me just make it clear once again I hope you
02:02have seen my previous videos where we discussed one thing that when you are
02:06trying to develop generative AI applications you have two different ways the
02:10first way is your microsoft copilot studio which is actually software as a
02:14service and it's a low code kind of an environment that is perfect for you if
02:19you are not a pro coding kind of a developer or data scientist but if you
02:24are looking for a pro code kind of a platform as a service then you have the
02:29service which is known as Azure AI Foundry. Azure AI Foundry is a platform
02:34that empowers developers to innovate with AI and shape the future it provides a
02:40collaborative environment with enterprise grade security where you can explore
02:44build taste and deploy AI tools and machine learning models through the
02:50integration with AI services. Azure AI Foundry is a platform as a service as I
02:55mentioned which means that this is actually giving you a fully professional
03:00development portal where you can actually deal with multiple models which is
03:05not only including Microsoft and open AI models it also includes models from
03:10hugging face meta deep seek and many more and in this case you will be able to
03:15fine-tune your model you will be able to integrate those things with the prompt
03:19flow you will be able to orchestrate the conversation flow with the help of
03:23prompt flow and you can add your own data there are a lot many things which you
03:27can do which we are going to discuss in depth in this particular module on top of
03:32this we have additional services available in Azure AI Foundry which are
03:36actually helping you to do the evolution of your GPT models and you will be able
03:42to taste the performance scalability reliability and the responsible AI safety
03:48so all the things are included as a one package in your Azure AI Foundry service now
03:54let's talk about what is Azure AI Foundry and what kind of component it has
03:59inside that the first thing which you need to understand is there are two
04:03components which you need to understand AI hubs and AI projects AI hub in Azure AI
04:09Foundry are the top level resources that provide a centralized setup and
04:13management for AI projects basically this is going to help you to group your
04:19multiple projects under one umbrella if you have used Azure portal and if you
04:24have created cloud resource deployments in that I'm sure you are familiar with
04:28something called resource group AI hub is very similar to that one AI hub can
04:33have multiple AI projects inside that this AI hubs offers features like data
04:39upload artifact storage connections with Azure services base model endpoint
04:44compute resources and governance control you can perform multiple tasks on the AI
04:49hub level like you can create and manage connections to resources such as data
04:55store github Azure AI search indexes and many more you can create and manage
05:00compute distances also on which you can run experiments prompt flows and custom code
05:06you can set up a security by creating members and assigning them to a specific
05:11roles and you can also set up governance by defining policies to manage behavior such
05:17as automatic compute shutdown all these configurations you will be able to
05:21control and manage at AI hub level inside one AI hub you can have multiple AI projects your
05:28projects are organizational containers within a hub that allow for AI
05:33customization and orchestration they help you to organize your work save the
05:39state across different tools like prom flow and collaborate with others your
05:43projects can use shared resources from AI hub and have a dedicated storage container for
05:50uploading files and sharing with project members within an AI project you can
05:55deploy foundation models from the model catalog which is available in a short AI
06:00foundry projects you can taste the models you can deal with the models inside the
06:05playground you can also augment prompts by connecting a custom data sources and
06:10connections and you can build copilots with prom flow you can evaluate model and
06:16application performance by using built-in or custom metrics and you can also
06:21manage the deployments of your model and apps all the things you can do inside AI
06:25projects just for your kind information anytime you want to use as your AI foundry
06:31you at least have to create one project inside one AI hub now let's talk about
06:38what is your AI foundry portal well this is one centralized environment where you can
06:44deploy and taste your model from the model catalog so basically this is going to
06:49help you to create a project allow you to browse the model from the model catalog
06:54and easily deploy those models and then you can taste the model by using the chat
06:59playground which is available inside Azure AI foundry let me tell you before
07:04Azure AI foundry portal there were multiple portals which were available which we
07:08normally used to call studio so we were having or I can say we still have Azure AI
07:14language studio vision studio document intelligence studio kind of things Azure AI
07:19foundry portal is actually bringing all of this together so somewhere if you are
07:24interested in generative AI and you are trying to explore generative AI models from
07:30various providers then Azure AI foundry is that one place which you have to try as you
07:36can see in the slide right now I'm showing you a screenshot where you have a
07:40section for model catalogs this model catalog section is going to give you
07:44open AI models like GPT-40, GPT-3, DALI, DAVINCI it's also going to give you
07:51Microsoft models it's going to give you LAMA models and very recent the new
07:56edition of the model which is available DeepSeq R1 is also available in the model
08:02catalog you can check the performance of the model you can compare the
08:06model you can deploy the model and you can actually fine-tune and customize these
08:11models from the model catalog now let's say I have selected one of the model
08:16like GPT-4 model it's actually going to give you the description of that
08:20particular model which version it is right now available inside that what kind
08:24of token configurations are required for that it's also going to give you an
08:28option to deploy the model fine-tune model or you can just make sure that what
08:34kind of deployments are configured with this or you can just add on your own data
08:40with the configuration of that particular model this one is a screen which is
08:45showing you the chat playground where we are actually associating a deployment of
08:49the model so this is my sample deployment which I have done a chat
08:52playground is basically going to give you three sections the left side section
08:57which is showing you system message is allowing you to customize the AI system
09:01message associated with that the right side part is actually showing you a
09:05playground where you will be able to type some kind of prompt as a user
09:10question and then your LLM model is going to respond based on that like we are
09:15asking a question what is AI and is actually giving you the response
09:19associated with that you also have a third section which is helping you to
09:22customize this and that is something which is a top bar you can see that
09:26inside the playground itself you have a section where you will be able to
09:30export you will be able to see the view code you can associate with the prompt
09:34flow or you can evaluate or maybe you can deploy to a web application if you want
09:40to use this thing as a proper chat bot all these options are available in the
09:45playground and we are surely going to try all those things in our next video
09:49which is going to be the lab of this module now let's say you have created a
09:54generic base generative AI assistant that use a general knowledge on which it
09:59was trained basically if you ask a question is going to give you an answer
10:03from that general knowledge but the model responses are not interesting and it
10:08doesn't know anything about your own data when this kind of things are there you
10:12have to make sure that you are going to integrate your own data and that is
10:16where the grounding is something which is coming in the picture so you can see in
10:21this slide we are talking about grounding a generative AI with your own data
10:25this is one of the most common requirement for most of my clients when I
10:29teach them generative AI the obvious question which much organization ask is
10:35how can I associate this large language based models with my own data and the
10:40answer is you can do this thing with the grounding content now in order to
10:45understand this thing let's understand the concept of groundedness first
10:49groundedness refers to whether the response of a language model is based on
10:54some factual information or not the main limitation of LLM is that they do not
10:59have an access to data which is protected by farewell and their knowledge is
11:03frozen in a specific period of time and that is something which is based on the
11:08time of training of that model for example consider the two scenarios which are
11:13shown in this particular slide in the left side you have ungrounded scenario in
11:17which the model is going to generate an answer to a questions about a product
11:21based on the data which it was trained which in the case of many large language
11:27models essentially means the whole of the internet the model use the text which
11:31it was trained with the generated coherent grammatically correct and
11:35appropriate answer but it's not based on a validated data source it might
11:41reference to a real product that happened to be an included in the training
11:45data or it might be even just an invent a product the goal of the model is to just
11:50provide an answer to the question even if the answer is not grounded in reality
11:55this is not going to focus on that now if you check the example here in the
12:01left side section we are asking a question what product should I use to do X now
12:06a grammatically correct but uncontextualized response is going to be
12:10generated with this particular model while on the other hand if I have a
12:14grounded scenario a specific data source is going to be used to provide the
12:19context for the interaction so basically in this case your LLM is actually going
12:23to have some kind of a product catalog but it can actually refer to a specific
12:28catalog in which it can find the product and it can relate with the person's
12:33intent when they are asking some particular questions in this kind of
12:37scenarios the answer will be grounded in reality and it's based on the product
12:41information from the catalog data and that's going to have a more
12:45contextualized response obviously grounded data and groundedness is going to
12:51improve your response quality and it's also going to improve your customer
12:55satisfaction that's the reason your LLM has to associate with the grounding data
13:01with that now the question is how exactly we can do this thing well the name of
13:06the topic which is helping you to do the thing is this which is known as
13:10retrieval augmented generation even though language models are trained on a vast
13:15amount of data they may not have an access to the knowledge which you want to
13:19make available to your users to ensure that your AI chat app is grounded on a
13:24specific data to provide accurate and a domain specific response you can use
13:29retrieval augmented generation this is a technique that you can use to ground a
13:35language model in other words it's a process for retrieving information that
13:40is relevant to the users initial prompt in general terms RAG pattern
13:45incorporates the multiple steps which are mentioned here you can see that there
13:49are three steps which are there step number one is retrieve grounding data based on
13:54the initial user entered prompt step number two is augment the prompt with
13:59grounding data and step number three is use a language model to generate a
14:04grounded response these three steps are going to be followed with the help of
14:09retrieval augmented generation and that's what this term means using RAG is a
14:14powerful and easy to use technique for many cases in which you want to ground
14:19your language model and improve the factual accuracy of your AI chat apps
14:24responses when we are using retrieval augmented generation we have to prefer to
14:30use embeddings for data retrieval now while the text based index
14:34will improve your search efficiency you can usually achieve a better data
14:39retrieval solution by using a vector based index that contains embeddings that
14:44represent the text tokens in your data source an embedding is a special format of
14:49data representation that a search engine can use to easily find the
14:54relevant information and this is extensively used in all the transformer based
14:58gpt models and this is extensively used in most of the modern transformer based large language models more specifically an embedding is a vector of floating point numbers now to understand this thing let's have a look at the slide we have an example here where imagine you have two different documents with a different kind of content in it the left side document document number one is having a statement that the children played joyfully in the park while the right side document
15:05which is document number two is having a statement kids happily run around the playground in this
15:35in this case this two documents contains text that are semantically related even though different words are used by creating vector embeddings for the text in the document the relation between the words in the text can be mathematically calculated
15:50imagine the keyword being extracted from the document and plotted as a vector in a multi-dimensional space as it is visible in this particular graph the distance between vector can be calculated by measuring the cosine of the angle
16:05between two vectors it also known as the cosine similarity in other words the cosine similarity computes the semantic similarity between the document and a query by representing words in their meanings with vectors you can extract relevant context from your data source even when your data is stored in a different format like text or images and many multiple languages when you want to be able to use vector search to search your data you need to create embeddings
16:35when creating your search index to create embeddings for your search index you can use azure open ai embedding models which are available in the azure ai foundry if you are interested in using this kind of an embedding models we have a lab available on this on our youtube channel which you can check out here on this particular link
16:56now let's now let's have a look at the next thing which is prompt flow and the components of the prompt flow so if your question is what are prompt flow the answer is prompt flow is a feature within the azure ai foundry that allows you to author flows basically is allowing you to create an executable workflows flows are nothing but this kind of executable workflows which are integrating with language models as you can see here in this particular slide
17:24we have some kind of an initial prompt from the user and then we want to add a conversational history on top of it and on top of this combined conversational history we want to add some relevant data and data sources
17:39we want to add relevant data from data source when you do this thing you are going to have a logical flow which is going to execute all these things sequentially and based on all these historical things is going to give you the response
17:51so basically it's going to combine the data and history for grounding context with initial prompt and then it's going to prepare one response this kind of a logical execution can be customized with the help of prompt flow and that's the reason this is going to help you to author flow with the components to retrieve data call the language model execute your own code in which you can actually mention the code in python and many more all these things will be possible with the help of prompt flow
18:19prompt flow now let's understand prompt flow components properly prompt flow is a feature of azure ai farmry portal so remember these author flows are only available within azure ai farmry portal these flows are executable workflows so it's going to help you to basically focus on three different things most of the time every prompt flow is going to focus on inputs outputs and in between input and output you're going to have multiple nodes which can be having multiple tools associated with that
18:49your inputs represent the data which is passed into the flow basically this is that initial data on which your prompt flow will start working it can be different data types like string integers or boolean after inputs you're going to have multiple nodes which represent tools that perform data processing task execution or some kind of algorithmic operations which may or may not require to have association with your existing LLM connection
19:19and actually write a custom logic with the help of Python language and finally you're going to have outputs which represent the data which is produced by the flow this output can be further used in the response which you're going to give it to your users when we talk about tools some common tools which are used as a nodes are LLM tools prompt tools and Python tools LLM tools are going to enable custom prompt creation utilizing your large language models
19:49to prepare prompts as a string for complex scenarios for integration with the other tools and your Python tools are going to allow you to execute a custom Python script in that which tool you're going to use LLM prompt tool or Python tool is something which depends on your custom logic which you want to build with your prompt flow now because we understood retrieval augmented generation and prompt flow both the common question which people ask is can we combine both of them the answer is 100% yes
20:19the key to use RAG pattern in the prompt flow is to use an index lookup tool to retrieve data from an index so that a subsequent tool in the flow can use the result to augment the prompt used to generate output from an LLM
20:35now when we do this thing this tool is going to help you to connect with your retrieval augmented generated index is going to look up into that index and then using that is going to get the context of that
20:47now as you can see here this whole process is designed here with the five step in the step number one you're going to add your data to your Azure AI project in step number two you're going to have an index your data with Azure AI search so basically up to this point you're searching and indexing is what which is going to come into the picture after this you're going to query your index data in a prompt flow with the index lookup tool this is a separate tool available
21:17you're going to associate with your Azure AI search index then based on that it's going to reference the retrieval context in the prompt and then it's going to send the prompt with the context to a large language model this is going to give you a more refined way to generate the response based on your own customized lookup data
21:35now last but not the least once we are happy with our prompt flow configuration logic we can deploy it to an endpoint
21:42now when we say we are deploying to an endpoint it's actually going to be working like a rest based API service it's going to give you a rest endpoint you can then consume this endpoint from your client application and you can implement generative AI functionalities in your application
21:57any developer who knows how to use rest based APIs can actually deal with this deployed endpoint of your prompt flow you are going to follow the structure like this your endpoint with your prompt flow logic is going to be deployed on the Azure cloud you will be processing all this request on the Azure cloud only but your client application is going to be connected with this particular service using that endpoint where you're going to make get post put delete kind of
22:27of a rest based request which will be further processed on through the cloud side this is how you're going to communicate between your client application and your custom logic which you have built and deployed using a prompt flow with this this module is coming to an end I hope you understood Azure AI foundry service and you understood the different components which are there but I honestly say this thing that you will be able to understand this thing with crystal clear clarity when you go through our lab videos which are available
22:57on skill tech club YouTube channel I strongly recommend you to go through those labs which are available for Azure AI foundry and those are going to be really important still if you have any doubts or questions your friend Maruti is always available please put a question in the comment of this particular video and I'll be more than happy to guide you on that thank you I'll see you in the next video
23:18the next video
Be the first to comment