How to Create Agent in Azure AI Foundry

 https://learn.microsoft.com/en-us/azure/ai-foundry/agents/overview?context=%2Fazure%2Fai-foundry%2Fcontext%2Fcontext 

pip install azure-ai-projects pip install azure-identity

import os from azure.ai.projects import AIProjectClient from azure.identity import DefaultAzureCredential from azure.ai.agents.models import CodeInterpreterTool # Create an Azure AI Client from an endpoint, copied from your Azure AI Foundry project. # You need to login to Azure subscription via Azure CLI and set the environment variables project_endpoint = os.environ["PROJECT_ENDPOINT"] # Ensure the PROJECT_ENDPOINT environment variable is set # Create an AIProjectClient instance project_client = AIProjectClient( endpoint=project_endpoint, credential=DefaultAzureCredential(), # Use Azure Default Credential for authentication api_version="latest", ) code_interpreter = CodeInterpreterTool() with project_client: # Create an agent with the Bing Grounding tool agent = project_client.agents.create_agent( model=os.environ["MODEL_DEPLOYMENT_NAME"], # Model deployment name name="my-agent", # Name of the agent instructions="You are a helpful agent", # Instructions for the agent tools=code_interpreter.definitions, # Attach the tool ) print(f"Created agent, ID: {agent.id}") # Create a thread for communication thread = project_client.agents.threads.create() print(f"Created thread, ID: {thread.id}") # Add a message to the thread message = project_client.agents.messages.create( thread_id=thread.id, role="user", # Role of the message sender content="What is the weather in Seattle today?", # Message content ) print(f"Created message, ID: {message['id']}") # Create and process an agent run run = project_client.agents.runs.create_and_process(thread_id=thread.id, agent_id=agent.id) print(f"Run finished with status: {run.status}") # Check if the run failed if run.status == "failed": print(f"Run failed: {run.last_error}") # Fetch and log all messages messages = project_client.agents.messages.list(thread_id=thread.id) for message in messages: print(f"Role: {message.role}, Content: {message.content}") # Delete the agent when done project_client.agents.delete_agent(agent.id) print("Deleted agent")

Run Llama 3 on your laptop

Ollama is a convenient platform for the local development of open-source AI models. 

Why should you use open-source AI? \

Today, we have access to powerful Large Language Models, such as GPT-4o or Clause 3.5 Sonnet. 

But they come with 4 major problems: 

Data Privacy. When “talking” to GPT-4 you always send your data to the OpenAI server. For most companies, this is the #1 reason NOT to use AI. 

Cost. The best-performing LLMs are expensive, especially for high-volume applications. 

Dependency. Using GPT-4 or Claude means you rely on OpenAI or Anthropic. Most businesses prefer independence. 

Limited Customization. Every business has unique needs and problems. Custom solutions are crucial for many. But customizing the biggest models is possible only through Prompt Engineering. 

Let’s compare it to the open-source models: 

Full Privacy. 

We run open-source models locally. Which means, we don’t send the data anywhere. They can work offline! 

Lower Cost. You can use many “local” models for free. You pay for more powerful ones, but they’re much cheaper than GPT-4. 

Independence & Control. You’ve got full control over the model. Once you download it to your computer, you “own” it. 

Customization. You can fine-tune, re-train, and modify open-source LLMs to fit your specific needs.

But of course, open-source LLMs have their own limitations: 

Worse Performance. Reasoning and general performance of open-source LLMs always lag behind GPT-4. 

Integration Challenges. Integrating them requires more expertise and effort. 

Hardware costs. LLMs require high computational power. To run them for high-volume applications, you need your own GPUs. 

Running local Llama 3 with Ollama. 

All you need: Download Ollama on your local system. 

Download one of the local models on your computer using Ollama. For example, if I want to use Llama3, I need to open the terminal and run: 

$ ollama run llama3 

If it’s the first time you use the model, Ollama will first download it. 

Because it has 8B parameters, it’ll take a while. 

 Once you download the model, you can use it through Ollama API. 

To install Ollama API, run the following command: 

 $ pip install ollama 

And with these steps, you’re ready to run the code from this article. 

 In this article, we’ll explore how to use open-source Large Language Models (LLMs) with Ollama. We’ll go through the following topics: Using open-source models with Ollama. 

The importance of the system prompt. Streaming responses with Ollama. The practical applications of the LLM temperature. 

The usage and limitations of the max tokens parameter. Replicating “creative” responses with the seed parameter

Getting the Simple Response

Now it’s time to test our model. Let’s ask a simple question to see how it works.

import ollama

model = "llama3"
response = ollama.chat(
    model=model, 
    messages=[
        {"role": "user", "content": "What's the capital of Poland?"}
    ]
)
print(response["message"]["content"])

## Prints: The capital of Poland is Warsaw (Polish: Warszawa).

1. import ollama to use Ollama API
2. model = "llama3 to define the model we want to use
3. ollama.chat() to get the response. We used 2 parameters:
   - model that we defined before
   - messages where we keep the list of messages

To get the response, we dig in the response object for ["message"]["content"].

OLTP and OLAP

When to use OLAP vs. OLTP Online analytical processing (OLAP) and online transaction processing (OLTP) are two different data processing systems designed for different purposes. OLAP is optimized for complex data analysis and reporting OLTP is optimized for transactional processing and real-time updates.

test

1. snowflake pvt link 1) Gary and Justin to followup with GIS team 2) Swamy to document the 3 options with Pros and Cons diucssed 3) What is the data classification and volume? What is the data stored in snowflake next week - setting up followup meeting 2. monday.con network integration Dev team want the bi-directional api communication wher lambda /EKS to call monday.com API and also monday.com to call Lamba/EKS

Cyber Security Standards - Risk Based Framework

Purpose

• The Risk Based Framework (RBF) is a risk classification system developed by the Enterprise Cyber Security (ECS) department of the Cyber Risk Management team. 
• ECS policy is intended to protect the firm in an evolving threat landscape, regardless of changes in technology or business practices. 
• Even if specific terminology or scenarios are not part of the text, it is expected that you will exercise sound reasoning and judgment to adhere to the intent of stated requirements, practices, and implementations in both letter and spirit.

Scope

• All systems that are listed in the IT Service Manager (ITSM) application (e.g., ServiceNow), are required to have an RBF classification. 
• All systems where the lifecycle stage is ‘Concept’, ‘Acquisition/Development’, or ‘Retired’ are not in scope.

Conversational AI

 

Conversational AI Applications 

Design 

Developing a bot, like developing websites and applications, should start with a design for a great experience. When humans interact with bots, we expect that what we say is understood, what we receive as a response is appropriate, and what we get as a service is delight‐ ful. We expect that, if we leave mid-conversation, the bot will remember where we left off.

Build

 A bot is a representational state transfer (REST) web service that communicates with the user by sending and receiving messages and events from conversational interfaces like chat rooms or Web Chat widgets. With Microsoft’s Azure Bot Service and Bot Framework, you can create bots in a variety of development environments and languages. You can start your bot development in the Azure portal or use one of the Bot Framework SDK templates for local develop‐ ment. The templates support the C#, JavaScript, and Python lan‐ guages with Java support in early preview at the time of writing. 

Test

To test conversational AI application, Microsoft provides the Bot Framework Emulator enabling developers to test conversations quickly and easily. We can also write unit tests using the Bot Frame‐ work SDK, which can focus on functionality testing of specific dia‐ logs. Once configured through the Azure portal, bot can be reached through a web chat interface, enabling broader testing by end users early in your development process. 

Publish 

When ready for your bot to be available on the web, either publish your bot to Azure or to your own web service or data center —wherever a normal web application can be hosted.

Connect 

Azure Bot Service does most of the work necessary to connect bots to a range of channels and devices. Configured through the Azure portal, you can connect your bots to Facebook Messenger, Slack, Microsoft Teams, Cortana, email, Telegram, Twilio, LINE, and other channels. You can also use Web Chat widgets to embed your bots in your websites or mobile applications. 

Evaluate

 Recordings of conversations between bots and users provide valua‐ ble business insights to help you evaluate your bot’s performance. At this phase, best practices include evaluating success metrics that you defined during the design phase, reviewing instrumentation logs, collecting user feedback, refining, and iterating. Bot Framework provides sample Application Insights queries and a Power BI dash‐ board to help you grasp the full breadth of your bot’s conversations with users and gain key insights into your bot’s health and behav

Virtual Assistants

Building on the Bot Framework SDK, Microsoft’s open source Vir‐ tual Assistant solution (available in C# and TypeScript) is a project template that encompasses the best practices for developing a bot on the Microsoft Azure platform. 

Building a conversational experience using the Virtual Assistant template requires the following resources: 

• Azure Bot Registration (to configure Bot endpoint and channel configuration) 
• Azure Web App (to host your Bot application) 
• Azure Storage Account (to store transcripts) 
• Azure Application Insights (telemetry) 
• Azure Cosmos DB (Conversational State and User State—this can be swapped for Azure Storage for development purposes) 
• Language Understanding 
• QnA Maker (including Azure Cognitive Search and an Azure Web App)

 

Complete Control for Developers  

With Virtual Assistant, all aspects of the user experience are owned and controlled by you. This includes the branding, name, voice, per‐ sonality, responses, and avatar. Microsoft provides five chat person‐ alities based on the Azure Cognitive Service QnA Maker, enabling you to tailor the bot’s personality. The source code to the Virtual Assistant and supporting Skills are provided as samples for you to customize. Your Virtual Assistant will be deployed within your Azure subscription. Therefore, all data generated by your Assistant (questions asked, user behavior, etc.) is entirely contained within your Azure subscription. See Cognitive Services compliance and privacy details and the Azure section of the Trust Center for more information.

Prebuilt and Reusable 

Skills Common Virtual Assistant scenarios are provided as reusable Con‐ versational Skills and include tasks like finding nearby points of interest, checking off an item on a to-do list, and replying to an email. Skills—delivered in source code form—are fully customizable and consist of language models for multiple natural languages, dialogs, and integration code 

Flexible Integration and Contextual Awareness 

The Virtual Assistant architecture is flexible and can be integrated with existing speech or NLP capabilities, back-end systems, APIs, and devices to enable personalization and adapt responses as required for the users’ context. The Assistant can also be aware of the device or channel through which the user is communicating, which can be used to optimize the experience (e.g., voice versus text) and enable the user to switch between different channels if needed. 

Business Insights Captured and Distributed Bot Framework

 provides a rich set of out-of-the-box metrics ena‐ bling any conversational experience to collect data at the levels you desire, to let you understand better what your customers are asking and where you might have opportunities to improve the service

References

 Aperture in Action: How We Solved PostgreSQL Performance Challenges

https://dzone.com/articles/aperture-in-action-how-we-solved-postgresql-perfor?utm_source=Sailthru&utm_medium=email&utm_campaign=DZone_Daily_Digest_10.16.23_BT_Snowflake_Spotlight&utm_term=dzone-daily-digest-active

How to Use Serverless Architecture

https://dzone.com/articles/how-to-use-serverless-architecture?utm_source=Sailthru&utm_medium=email&utm_campaign=DZone_Daily_Digest_10.16.23_BT_Snowflake_Spotlight&utm_term=dzone-daily-digest-active

Best Practices for Microservices: Building Scalable and Efficient Systems

https://cb.sailthru.com/click/33025582.48113/aHR0cHM6Ly9kem9uZS5jb20vYXJ0aWNsZXMvYmVzdC1wcmFjdGljZXMtZm9yLW1pY3Jvc2VydmljZXMtYnVpbGRpbmctc2NhbGFibGU_/64b567c4080fd50b97b29669B382db0f9

Image Identification and Classification with Amazon Bedrock, OpenSearch, and OpenCLIP

https://garystafford.medium.com/image-identification-and-classification-with-amazon-bedrock-opensearch-and-openclip-5442baca1846

Your Own Personal LLaMa

https://towardsdatascience.com/your-own-personal-llama-8cf24a9237cb

LlamaIndex update 2023–10–10

https://blog.llamaindex.ai/llamaindex-update-2023-10-10-3718a3d19fb9

AI Opportunities in Retail — A Comprehensive Technical Implementation Guide

https://medium.com/predict/ai-opportunities-in-retail-a-comprehensive-technical-implementation-guide-d26d9abba144

Deploy ML models at the edge with Microk8s, Seldon and Istio

https://medium.com/ubuntu-ai/deploy-ml-models-at-the-edge-with-microk8s-seldon-and-istio-5b97fe695578

Ten tips and tricks to employ in your Gen AI projects

https://lakshmanok.medium.com/ten-tips-and-tricks-to-employ-in-your-gen-ai-projects-7468f8dc6bf0

What’s a Vector Database?

https://medium.com/building-the-open-data-stack/whats-a-vector-database-c3e8b6bb0436

Leveraging LLMs on your domain-specific knowledge base

https://blog.ml6.eu/leveraging-llms-on-your-domain-specific-knowledge-base-4441c8837b47

Azure , AWS and GCP

 Compute:        

• AWS EC2
• GCP Compute Engine

Containerization and Orchestration:

• Azure Kubernetes Service (AKS)
• AWS Elastic Kubernetes Service (EKS)
• GCP Kubernetes Engine (GKE)

Serverless Computing:

• Azure Functions
• AWS Lambda
• GCP Cloud Functions

Database:

• Azure SQL Database
• AWS RDS
• GCP Cloud SQL

Big Data and Analytics:

• Azure HDInsight
• AWS EMR
• GCP Dataproc

Storage:

• Azure Storage
• AWS S3
• GCP Cloud Storage

Networking:

• Azure Virtual Network
• AWS VPC
• GCP VPC

Identity and Access Management:

• Azure Active Directory
• AWS IAM
• GCP Identity and Access Management (IAM)

API Management:

• Azure API Management
• AWS API Gateway
• GCP Cloud Endpoints

Load Balancing:

• Azure Load Balancer
• AWS ELB (Elastic Load Balancer)
• GCP Cloud Load Balancing