Tuesday, December 17, 2019

Azure!!

Azure Load Balancer

  • Load-balance incoming internet traffic to your VMs. This configuration is known as a public load balancer.
  • Load-balance traffic across VMs inside a virtual network. You can also reach a load balancer front end from an on-premises network in a hybrid scenario. Both scenarios use a configuration that is known as an internal load balancer.
  • Port forward traffic to a specific port on specific VMs with inbound network address translation (NAT) rules.
  • Provide outbound connectivity for VMs inside your virtual network by using a public load balancer.
Hash-based distribution

Traffic Manager 

Let’s say you want to balance load between two endpoint where your first endpoint is in Azure and 2nd endpoint is in on-prem . In this scinario we can use Traffic Manager . Traffic Manager allows you to control the distribution of user traffic for service endpoints in different datacenters. Service endpoints supported by Traffic Manager include Azure VMs, Web Apps, and cloud services. You can also use Traffic Manager with external, non-Azure endpoints.
When using a vanity domain with Azure Traffic Manager, you must use a CNAME to point your vanity domain name to your Traffic Manager domain name. DNS standards do not allow you to create a CNAME at the ‘apex’ (or root) of a domain. Thus you cannot create a CNAME for ‘contoso.com’ (sometimes called a ‘naked’ domain). You can only create a CNAME for a domain under ‘contoso.com’, such as ‘www.contoso.com’. To work around this limitation, we recommend using a simple HTTP redirect to direct requests for ‘contoso.com’ to an alternative name such as ‘www.contoso.com’.
Connection establishment using Traffic Manager

Routing Method :

  • Priority: Select Priority when you want to use a primary service endpoint for all traffic, and provide backups in case the primary or the backup endpoints are unavailable.
  • Weighted: Select Weighted when you want to distribute traffic across a set of endpoints, either evenly or according to weights, which you define.
  • Performance: Select Performance when you have endpoints in different geographic locations and you want end users to use the “closest” endpoint in terms of the lowest network latency.
  • Geographic: Select Geographic so that users are directed to specific endpoints (Azure, External, or Nested) based on which geographic location their DNS query originates from. This empowers Traffic Manager customers to enable scenarios where knowing a user’s geographic region and routing them based on that is important. Examples include complying with data sovereignty mandates, localization of content & user experience and measuring traffic from different regions.

Application Gateway

Azure Application Gateway is a web traffic load balancer that enables you to manage traffic to your web applications.
Traditional load balancers operate at the transport layer (OSI layer 4 – TCP and UDP) and route traffic based on source IP address and port, to a destination IP address and port. But with the Application Gateway you can be even more specific. For example, you can route traffic based on the incoming URL. So if /images is in the incoming URL, you can route traffic to a specific set of servers (known as a pool) configured for images. If /video is in the URL, that traffic is routed to another pool optimized for videos.
imageURLroute



Deploy R Models to Azure

Tools
  • R Studio
  • VSTS

Approach-1

Take the Model and put in Docker Container
and create Keburnetes Cluster and copy images to the cluster

  • Create Docker Image
  • Deploy Image to Service  

  • These two steps can be done by Right click in VSTS and Deploy the Model to Service
Model file and Scoring File 


Approach-2

  • Use YMAL file config to specify the dependencies and stuff
  • Create Tags and Finish.. lower left corner you can see its being deployed..

How the scoring file looks like?

  • uses tensor flow model or some other model.
  • load that up
  • find the nodes and graphs that are relevant

  • init function and run function()
  • Custom code to pre-process data and normalize logic.
  • azureml-defaults to keep track of model

Where are these Azure containers and Services living?

  • Create AZURE ML work-space.

  • It automatically create blob storage account, azure container registry been created
  • Put the Docket image is put in registry 

Approach-3

  • Create VM , VNET, Provide IP address Public or Private (Direct Connect/ Express Route)
  • Install Dependencies libraries

High Level Steps in all approaches

  • Develop Code in R studio along with Data Sets
  • Create R Packages
  • Create Zip File
  • Create Wrapper File (again ZIP)
  • Upload the wrapper file to Azure ML Work Space.

Highlights 


  • 30% of Data Scientists are using R Data as Primary Data Analysis Tool
  • RStudio Server (open source) running in containers
  • Rstudio Connect Cluster








Azure Logging

OMS Log Analytics and Custom Logs


Steps

  • Create LOG
  • Collect LOG
  • Define Custom Fields
  • Search the LOG
The Custom Log
  • Delimiter
  • No Circle logging
  • ASCII or UTF-8 
Azure PORTAL

General
  • Search for Log Analytics or Monitor
  • Create New Log Analytics

  • Provide Name, Subscription, Resource Group, Location, Pricing Option etc..

  • Go to Advanced Settings
  • Go to Data
  • Chose File from Custom Log
  • Select Delimiter ( New line)
  • Set Path
  • Chose path for every log file
  • Add Name and Description
  • Takes 15 minutes to create the stream
  • Go to Log Search...... select the custom log that is created


Azure Monitor

Wednesday, November 13, 2019

Angular Minimal options


>> ng new --minimal --routing --style=scss my-site


>> styles.scss
>> routing.ts


Angular App Tour ( foundations)
-----------------------------------------

1) Modules >> organize the code

@Declarations

@imports

@providers

@bootstrap

2) Components

>>>> ng serve  will start the Node server locally in 4200 port


>>> Angular CLI re-complies anytime file is saved

Component contains

selecctor: 
template:
styles:


3) Routes

const routes: Routes = [
         {
              path: '  ',
              component: HomeComponenet
          },



]

Sunday, October 20, 2019

Monday, October 7, 2019

Docker and Windows 10

  1. https://hub.docker.com/editions/community/docker-ce-desktop-windows
  2. Click on Get Docket button to download Docker
  3. Docker for Windows Installer.exe  will be downloaded
  4. Docker Tool box needed for other than windows 10 Professional and Enterprise.
  5. https://docs.docker.com/toolbox/toolbox_install_windows/
  6. C:\Program Files\Docker Toolbox
    1. Docker cli client for running docker engine to create images,containers
    2. Docker machine so we can run docker commands
    3. Docker compose to run docker --compose command
    4. Kitematic --> docker gui
    5. Docker quick start
    6. Oracle VM Virtual box
  7. It creates Desktop icon for Docket Quick Start terminal
  8. Launch it
  9. run docker --version  
  10. Docker version 18.03.0-ce, build 0520e24


Coding Tips

AWS Base Architecture


WAR Remediation Steps




  • Apply Jumpstart standards to AWS Account 
  • Deploy CIS Benchmarks throughout AWS account
  • CIS Benchmark Alerts o GuardDuty
  • VPC Flow Logs
  • CloudTrail
  • CloudWatch Logs
  • S3 Buckets collecting the logs, etc.  
  • Resiliency – Utilize Route 53
  • Elastic Load Balancer and instances in Multi AZs. 
  • Monitoring - Compute resources & alerts 
  • Systems Manager – Patching, Cloud Watch Agent 


Security
Current state from 31% to 83% CIS compliant
  • SCP policies
  • GuardDuty
  • Centralize logs
  • Config rules
  • Security Hub (CIS compliance)

Identity Access Management
  • Clean up unnecessary IAM users and roles

Logging and Monitoring
  • Install  SSM Agent in the front-end instances to easily manage, patch, and run commands to multiples instances via AWS Systems Manager.
  • Installed CloudWatch agent in the front-end instances and sending the application logs to Cloudwatch
  • Configured CloudWatch Alerts and SNS Topic 
  • Monitoring the health of the front-end instances


Reliability
  • Help to define a plan for DR (disaster recovery) (RPO and RTO)
    • Potentially deploy AWS Backups to centrally manage and automate backups across AWS services
  • Provide a Network topology diagram

Cost Optimization
  • Use Reserved Instances for baseline workloads and capacity reservation
Use On-Demand Instances for short-term or unpredictable workloads

AWS - System Manager Setup (Apache LOGS)

Launch EC2 instance as ROOT

Edit the /etc/awslogs/awslogs.conf file and add the following.

  • sudo yum update -y
  • sudo yum install -y awslogs

  • vim /etc/awslogs/awscli.conf
  • sudo service awslogs start
  • sudo chkconfig awslogs on





vim /etc/awslogs/awslogs.conf

[/var/log/httpd/error_log]
datetime_format = %b %d %H:%M:%S
file = /var/log/httpd/error_log
buffer_duration = 5000
log_stream_name = {instance_id}
initial_position = start_of_file
log_group_name = /var/log/httpd/error_log

[/var/log/httpd/access_log]
datetime_format = %b %d %H:%M:%S
file = /var/log/httpd/access_log
buffer_duration = 5000
log_stream_name = {instance_id}
initial_position = start_of_file
log_group_name = /var/log/httpd/access_log

[/var/log/httpd/ssl_access_log]
datetime_format = %b %d %H:%M:%S
file = /var/log/httpd/ssl_access_log
buffer_duration = 5000
log_stream_name = {instance_id}
initial_position = start_of_file
log_group_name = /var/log/httpd/ssl_access_log

  1. Restart the awslogsd service
service awslogsd restart

AWS Click Stream Analytics with Kenisys

AWS - How to Perform EC2 Backup: Step-By-Step Guide