Lab 12- Creating a webapp and power virtual agent bot with custom data using Azure OpenAI Service
Introduction:
Azure OpenAI on your data works with OpenAI’s powerful ChatGPT (gpt-35-turbo) and GPT-4 language models, enabling them to provide responses based on your data. You can access Azure OpenAI on your data using a REST API or the web-based interface in the Azure OpenAI Studio to create a solution that connects to your data to enable an enhanced chat experience.
One of the key features of Azure OpenAI on your data is its ability to retrieve and utilize data in a way that enhances the model’s output. Azure OpenAI on your data together with Azure Cognitive Search determines what data to retrieve from the designated data source based on the user input and provided conversation history. This data is then augmented and resubmitted as a prompt to the OpenAI model, with retrieved information being appended to the original prompt. Although, retrieved data is being appended to the prompt, the resulting input is still processed by the model like any other prompt. Once the data has been retrieved and the prompt has been submitted to the model, the model uses this information to provide a completion.
Objectives
- To create a storage account, container, and Azure cognitive search service in the Azure portal.
- To deploy gpt-3-turbo and Embedded model in Azure AI Studio and to add data in Chat Playground.
- To test Assistant setup in Chat playground by sending queries in chat session.
- To launch a new power virtual agent and start a conversation with the bot
- To launch a new app and start a conversation with the copilot app.
- To delete gpt-3-turbo and embedded model, Azure storage account, cognitive search service, and the new web app.
Exercise 1- Create an Azure Storage Account and Azure cognitive Search by using the portal
Task 1: Cognitive Services Usages Reader for the Azure OpenAI resource
Open your browser, navigate to the address bar, and type or paste the following URL: https://portal.azure.com/, then press the Enter button.
In the Microsoft Azure window, use the User Credentials to login to Azure.
Then, enter the password and click on the Sign in button.
In Stay signed in? window, click on the Yes button.
Type in Subscriptions in the search bar and select Subscriptions.
Click on your assigned subscription.
From the left menu, click on the Access control(IAM).
On the Access control(IAM) page, Click +Add and select Add role assignments.
Type the Cognitive Services Usages Reader in the search box and select it. Click Next
In the Add role assignment tab, select Assign access to User group or service principal. Under Members, click +Select members
On the Select members tab , search your Azure OpenAI subscription and click Select.
In the Add role assignment page, Click Review + Assign, you will get a notification once the role assignment is complete.
You will see a notification — added as Cognitive Services Usage Reader for Azure Pass-Sponsorship.
In Azure subscription page from the left menu, click on the Access control(IAM).
On the Access control(IAM) page, Click +Add and select Add role assignments.
Type the Cognitive Services Contributor in the search box and select it. Click Next
In the Add role assignment tab, select Assign access to User group or service principal. Under Members, click +Select members
On the Select members tab , search your Azure OpenAI subscription and click Select.
In the Add role assignment page, Click Review + Assign, you will get a notification once the role assignment is complete.
You will see a notification — added as Cognitive Services Usage Reader for Azure Pass-Sponsorship.
Go back to Azure portal home page, type in Azure OpenAI in the search bar and select Azure OpenAI.
Click on your Azure OpenAI service.
From the left menu, click on the Access control(IAM).
On the Access control(IAM) page, Click +Add and select Add role assignments.
Type the Cognitive Services Contributor in the search box and select it. Click Next
In the Add role assignment tab, select Assign access to User group or service principal. Under Members, click +Select members
On the Select members tab , search your Azure OpenAI subscription and click Select.
In the Add role assignment page, Click Review + Assign, you will get a notification once the role assignment is complete.
You will see a notification — added as Cognitive Services Usage Reader for Azure Pass-Sponsorship.
Task 2: Create an Azure Storage Account by using the portal
Sign in to the https://portal.azure.com/
Click on the Portal Menu, then select + Create a resource
In the Create a resource window search box, type Storage account and then click on the storage account.
In the Marketplace page, click on the Storage account section.
In the Storage account window, click on the Create button.
On Create a storage account window, under the Basics tab, enter the below details to create a storage account and then click on Review
SubscriptionSelect your Azure OpenAI subscriptionResource groupSelect your Resource group(that you have created in Lab 1)Storage account nameazureopenaistorageXX(XX can be a unique number) (here, we entered azureopenaistorage39)RegionEast USPerformanceStandard: Recommended for most scenarios (general-purpose v2 account)RedundancyLocally-redundant storage (LRS)
On the Review tab, click on the Create button.
This new Azure Storage account is now set up to host data for an Azure Data Lake. Click on the Go to resource button.
After the account has been deployed, you will find options related to Azure Data Lake in the Overview page. In the left-side navigation pane, navigate to Data storage section, then click on Containers.
On azureopenaistorageXX | Containers page, click on +Container.
On the New container pane that appear on the right side, enter the container Name as source and click on Create button.
On azureopenaistorageXX | Containers page, select source container.
On source container page, click on Upload button.
In the Upload blob pane, click on Browse for file, navigate to C:\Labfiles location and select TF-AzureOpenAI.pdf, then click on the Open button.
In Upload blob pane, click on the Upload button.
You will see a notification — Successfully uploaded blob when the uploaded is succeeded.
Task 3: Create an Azure Cognitive Search service in the portal
On the azureopenaistorageXX | Containers page, click on Home to go back to Azure portal home page.
In Azure portal home page, click on + Create Resource.
In the Create a resource page search bar, type Azure AI Search and click on the appeared azure ai search.
Click on azure ai search section.
In the Azure AI Search page, click on the Create button.
On the Create a search service page, provide the following information and click on Review+create button.
FieldDescriptionSubscriptionSelect your Azure OpenAI subscriptionResource groupSelect your Resource group(that you have created in Lab 1)RegionEastUSNamemysearchserviceXX (XXcan be unique number)Pricing TierClick on change Price Tire>select Basic
Once the Validation is passed, click on the Create button.
After the deployment is completed, click on the Go to resource button.
In the mysearchserviceXX Overview page. In the left-side navigation pane, under Settings section, select Semantic ranker.
On the Semantic ranker tab, select Free tile and click on the Select plan.
You will see a notification -Successfully updated semantic ranker to free plan
Task 4:Create an Azure Cosmos DB for MongoDB vCore cluster by using the Azure portal
Sign in to the https://portal.azure.com/
Click on the Portal Menu, then select + Create a resource
In the Create a resource window search box, type Azure Cosmos DB and then click on the Azure Cosmos DB.
In the Marketplace page, on the Azure CosmosDB click on the Create button.
On the Which API best suits your workload? page, select the Create option within the Azure Cosmos DB for MongoDB section
On the Which type of resource? page, select the Create option within the vCore cluster section.
On the Create Azure Cosmos DB for MongoDB cluster page, provide the following information and click on Next:Networking> button
FieldDescriptionSubscriptionSelect your Azure OpenAI subscriptionResource groupSelect your Resource group(that you have created in Lab 1)Cluster nameaoaicosmosdbXX (XXcan be unique number)LocationEastUSCluster tierselect the Configure option enter the below options and click on Save button• Cluster tier- M30 Tier, 2 vCores, 8-GiB RAM• Storage- 128 GiB• Select the check box — I UnderstandAdmin usernameaoaiadminPasswordpassword321!Confirm passwordpassword321!
In the Networking section, select Allow public access from Azure services and resources with Azure to this cluster and click on Review+create.
Once the Validation is passed, click on the Create button.
Click on the Go to resource button.
Exercise-2: Add your data using Azure OpenAI Studio
Task 1: Deploy gpt-3-turbo and embedded models in Azure AI Studio
Open your browser, navigate to the address bar, and type or paste the following URL: https://oai.azure.com/ then press the Enter button.
Note: If you are directed to the Azure OpenAI Studio home page, then skip steps from #2 to #4, else continue.
In the Microsoft Azure window, enter your Sign-in credentials, and click on the Next button.
Then, enter the password and click on the Sign in button.
In Stay signed in? window, click on the Yes button.
On the Welcome to Azure OpenAI Studio dialog box, under the Subscription field, enter the subscription assigned to you, and in the Resource field, select the existing Resource name that you’ve created in Lab #1, and then click on the Use resource button.
Within few minutes Azure AI Studio page will appear.
On the Azure AI Studio homepage, click on Create new deployment button.
In the Deployments page, click on +Create new deployment.
Select the Model version as Auto-update to default, in the Deployment name field, enter gpt-35-turbo, and click on the Create button.
You will see a notification — Successfully Created deployment when the deployment is succeeded. (You can also view the notification by clicking on the bell icon beside Azure AI | Azure AI Studio).
In the Deployments page, click on +Create new deployment.
In the Deploy model dialog box, under Select a model click on the dropdown select text-embedding-ada-002 field, under Model version select 2(Default) and under Deployment name enter text-embedding-ada-002. Click on the Create button.
You will see a notification — Successfully Created deployment when the deployment is succeeded. (You can also view the notification by clicking on the bell icon beside Cognitive Services | Azure OpenAI Studio).
- Important: We strongly recommend using text-embedding-ada-002 (Version 2). This model/version provides parity with OpenAI’s text-embedding-ada-002. To learn more about the improvements offered by this model, please refer to OpenAI’s blog post. Even if you are currently using Version 1, you should migrate to Version 2 to take advantage of the latest weights/updated token limit.*
Task 2: Add your data using Azure OpenAI Studio
Click on the Azure OpenAI home icon to go back to the home page.
In Azure OpenAI Studio Home page, under Welcome to Azure OpenAI Service, click on the Bring your own data
In the Add data page, click on the dropdown under Select or add data source, then navigate and click on Azure Blob Storage.
In the Add data page, under Select or add data source enter the following details and select Next.
SubscriptionSelect your Azure OpenAI subscriptionSelect Azure Blob storage resourceSelect your Azure Blob storage that you have created in Exercise 1 Task 2(azureopenaistorageXX)Select storage containersourceSelect Azure AI Search resourceSelect your Azure AI Search that you have created in Exercise 1 Task 3(mysearchserviceXX)Enter the index nameazure-indexIndexer scheduleOnce
Select the check box — Add vector search to this search resource.
Select an embedding model as text-embedding-ada-002, then click on the Next button.
- Note: In case, you encounter an error — Can’t manage CORS on this resource. Please select another storage resource, then syn your VM time, as mentioned in Lab #1, Task #1.
In the Add data page, on the Data management tab drop down the Search type and select Hybrid(vector+keyword).
Select the chunk size as 1024(default). Then, click on Next.
In Review and Finish pane, review the details that you’ve entered, and click on Save and close button.
The data will be added in your Chat Playground. This will take approximately 4–5 minutes.
Task 3: Explore text completion in the Chat Playground
In the Chat session section, enter the following prompt in the User message text box and click on the Send icon
What is Azure OpenAI Service?
In the Chat session section, select the references link and observe the details of search document on right side of the page.
Send the following prompt to the model by pasting them in User message text box and clicking on the Send icon.
How do I get access to Azure OpenAI?
In the Chat session section, select the references link and observe the details of search document on right side of the page.
Exercise 3: Deploy a web app with custom data
Task 1: Deploy a web app
Open your browser, navigate to the address bar, and type or paste the following URL: https://oai.azure.com/ then press the Enter button.
In Azure AI Studio Chat playground, click on the V chevron button beside Deploy to, then navigate and click on A new web app.
- Note: In case, you did not see Deploy to button on your VM, then use Ctrl+- or Ctrl+minus keyboard shortcut to zoom out and decrease the font size.
On Deploy to a web app window, select Create a new web app radio button and enter the following details:
NameAOAI-webappXXX(XXX can be a unique number) (here, we entered AOAI-webapp129)SubscriptionSelect the assigned subscriptionResource GroupSelect the resource group created in Lab 1LocationEast USPricing planBasic(B1)
Select the check box of Enable chat history in the web app
Select the check box of I acknowledge that enabling chat history will incur CosmosDB usage to my account
Select the check box of I acknowledge that web apps will incur usage to my account, then click on Deploy button.
Wait for the deployment to complete. The deployment will take around 10–15 minutes.
After successful deployment of the web app, you’ll will see a notification — Web app deployed. (You can also view the notification by clicking on the bell icon beside Azure AI | Azure AI Studio).
On the right side of Chat playground, click on Launch web app button.
Wait for 10 minutes, so that authentication configuration can be successfully applied on the app.
After 10 minutes, click on the Refresh button.
On Permissions requested dialog box, click on the Accept button
Now, web app will open in a new browser.
In the Azure AI web app page, enter the following text and click on the Submit icon as shown in the below image.
How do I get access to Azure OpenAI?
Similarly, paste the following text in the text box and click on the Send icon.
What is the expiry date of GPT-35-Turbo version 0301 and GPT-4 version 0314?
Refresh the webapp page and click on the Show chat history.
Under the chat history click on Accessing Azure OpenAI.
Exercise 4: Create a Copilot app with custom data
Task 1: Create a chatbot with custom data
In Azure AI Studio Chat playground, in the Add your data select the Remove data source.
In case, Remove grounding data? dialog box appears, then click on the Continue button.
In Chat playground pane , under the Assistant setup select Add your data and then select the +Add data source .
In the Add data page, under Select or add data source enter the following details and select Next.
SubscriptionSelect your Azure OpenAI subscriptionSelect Azure Blob storage resourceSelect your Azure Blob storage that you have created in Exercise 1 Task 2(azureopenaistorageXX)Select storage containersourceSelect Azure AI Search resourceSelect your Azure AI Search that you have created in Exercise 1 Task 3(mysearchserviceXX)Enter the index namecopilot-indexIndexer scheduleOnce
Note: In case, you encounter an error — Can’t manage CORS on this resource. Please select another storage resource, then syn your VM time, as mentioned in Lab #1, Task #1.
In the Add data page, on the Data management tab drop down the Search type and select Keyword, select the chunk size as 1024(default). Then, click on Next.
In Review and Finish pane, review the details that you’ve entered, and click on Save and close button**.**
The data will be added in your Chat Playground. This will take approximately 4–5 minutes.
In Azure AI Studio Chat playground, click on the V chevron button beside Deploy to, then navigate and click on A new copilot in Copilot Studio(preview)
- Note: In case, you did not see Deploy to button on your VM, then use Ctrl+- or Ctrl+minus keyboard shortcut to zoom out and decrease the font size.
Deploy to a copilot in Copilot Studio(preview) dialog box appears, then click on the Continue in Copilot Studio button.
you are prompted to Choose your country/region, then click on the dropdown and select your region, then click on Start free trail button.
In the Create a copilot pane, enter Copilot name as TF-Copilot and click on Create button.
Wait for the deployment to complete. The deployment will take around 15–16 minutes.
In the New features in Copilot Studio dialog box, click on the Skip button.
In your Copilot Studio | TF-Copilot window, navigate to the Test copilot pane enter the following text and click on the Submit icon as shown in the below image.
How do I get access to Azure OpenAI?
Similarly, paste the following text in the text box and click on the Send icon.
What is the expiry date of GPT-35-Turbo version 0301 and GPT-4 version 0314?
Task 2: Delete the deployed models
In Azure OpenAI Studio, on the left pane, under the Management section, click on Deployments.
Select gpt-35-turbo0301 deployment name and click on Delete deployment.
In the Confirm delete dialog box, click on the Delete button. You will see the notification — Successfully Deleted deployment (In case, you did not see the notification, then click on the bell icon beside Azure AI | Azure AI Studio).
In Azure OpenAI Studio, on the left pane, under the Management section, click on Deployments.
Select text-embedding-ada-002 deployment name and click on Delete deployment.
In the Confirm delete dialog box, click on the Delete button. You will see the notification — Successfully Deleted deployment (In case, you did not see the notification, then click on the bell icon beside Cognitive Services | Azure OpenAI Studio).
Task 3: Delete the Azure storage account, Azure cognitive search and Azure Web app
To delete the storage account, navigate to Azure portal home page, type Resource groups in the Azure portal search bar, navigate and click on Resource groups under Services.
Click on the assigned resource group.
Carefully select storage account, Azure Cognitive Search, Azure web app, CosmosDB that you’ve created.
- Note: Don’t select Azure OpenAI service.
In the Resource group page, navigate to the command bar and click on Delete.
- Important Note: Don’t click on Delete resource group. If you don’t see the Delete option in the command bar, then click on the horizontal ellipsis.
In the Delete Resources pane that appears on the right side, enter the delete and click on Delete button.
On Delete confirmation dialog box, click on Delete button.
Click on the bell icon, you’ll see the notification — Executed delete command on 4 selected items.
Summary
You’ve created a storage account, container, and Azure cognitive service in Azure portal, then you’ve deployed gpt-3-turbo model in Azure AI Studio. You’ve added data in Chat Playground and tested the Assistant setup by sending queries in a chat session. Then, you’ve launched a new app and started conversation with the chatbot. You’ve deleted the gpt-3-turbo model, Azure storage account, cognitive search service, and the new web app to effectively and efficiently manage the Azure OpenAI resources.
Important Note: Please do not delete the Resource Group. If deleted, you’ll not be able to proceed with the next lab or create a new Resource Group.
Please do not delete the Azure OpenAI Service (Azure-openai-testXX). The same service will be used throughout all the labs.
Hey there!
Recently I have started this series of Azure AI Workshop, to learn more about this visit my profile. Thank you for your invaluable time and stay tuned.
Want to stay connected and get more awesome content? Here’s where you can find me on social media:
- Follow my latest articles and insights on Ganesh Neelakanta.
- Connect with me for industry updates, discussions, and networking opportunities on LinkedIn : Ganesh Neelakanta
