Sunday, September 5, 2021

azure function with docker

 {

    "$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "subscriptionId": {
            "type": "string"
        },
        "name": {
            "type": "string"
        },
        "location": {
            "type": "string"
        },
        "hostingPlanName": {
            "type": "string"
        },
        "serverFarmResourceGroup": {
            "type": "string"
        },
        "alwaysOn": {
            "type": "bool"
        },
        "storageAccountName": {
            "type": "string"
        },
        "use32BitWorkerProcess": {
            "type": "bool"
        },
        "linuxFxVersion": {
            "type": "string"
        },
        "dockerRegistryUrl": {
            "type": "string"
        },
        "dockerRegistryUsername": {
            "type": "string"
        },
        "dockerRegistryPassword": {
            "type": "securestring"
        },
        "dockerRegistryStartupCommand": {
            "type": "string"
        }
    },
    "variables": {},
    "resources": [
        {
            "apiVersion": "2018-11-01",
            "name": "[parameters('name')]",
            "type": "Microsoft.Web/sites",
            "kind": "functionapp,linux",
            "location": "[parameters('location')]",
            "tags": {},
            "dependsOn": [],
            "properties": {
                "name": "[parameters('name')]",
                "siteConfig": {
                    "appSettings": [
                        {
                            "name": "FUNCTIONS_EXTENSION_VERSION",
                            "value": "~3"
                        },
                        {
                            "name": "DOCKER_REGISTRY_SERVER_URL",
                            "value": "[parameters('dockerRegistryUrl')]"
                        },
                        {
                            "name": "DOCKER_REGISTRY_SERVER_USERNAME",
                            "value": "[parameters('dockerRegistryUsername')]"
                        },
                        {
                            "name": "DOCKER_REGISTRY_SERVER_PASSWORD",
                            "value": "[parameters('dockerRegistryPassword')]"
                        },
                        {
                            "name": "WEBSITES_ENABLE_APP_SERVICE_STORAGE",
                            "value": "false"
                        },
                        {
                            "name": "APPINSIGHTS_INSTRUMENTATIONKEY",
                            "value": "c5ac382d-ff86-40f6-bba3-5d808c42edb5"
                        },
                        {
                            "name": "APPLICATIONINSIGHTS_CONNECTION_STRING",
                            "value": "InstrumentationKey=xxxxxxxxxxxx;IngestionEndpoint=https://westus2-0.in.applicationinsights.azure.com/"
                        },
                        {
                            "name": "AzureWebJobsStorage",
                            "value": "[concat('DefaultEndpointsProtocol=https;AccountName=',parameters('storageAccountName'),';AccountKey=',listKeys(resourceId('55954b78-5022-4a56-9fdc-54207b3a74ab','swamy-resource-grp','Microsoft.Storage/storageAccounts', parameters('storageAccountName')), '2019-06-01').keys[0].value,';EndpointSuffix=','core.windows.net')]"
                        }
                    ],
                    "use32BitWorkerProcess": "[parameters('use32BitWorkerProcess')]",
                    "linuxFxVersion": "[parameters('linuxFxVersion')]",
                    "appCommandLine": "[parameters('dockerRegistryStartupCommand')]",
                    "alwaysOn": "[parameters('alwaysOn')]"
                },
                "serverFarmId": "[concat('/subscriptions/', parameters('subscriptionId'),'/resourcegroups/', parameters('serverFarmResourceGroup'), '/providers/Microsoft.Web/serverfarms/', parameters('hostingPlanName'))]",
                "clientAffinityEnabled": false
            }
        }
    ]
}
Posted by at No comments:

Saturday, August 28, 2021

azure function app with storage

 {

    "$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "location": {
            "type": "string"
        },
        "storageAccountName": {
            "type": "string"
        },
        "accountType": {
            "type": "string"
        },
        "kind": {
            "type": "string"
        },
        "accessTier": {
            "type": "string"
        },
        "minimumTlsVersion": {
            "type": "string"
        },
        "supportsHttpsTrafficOnly": {
            "type": "bool"
        },
        "allowBlobPublicAccess": {
            "type": "bool"
        },
        "allowSharedKeyAccess": {
            "type": "bool"
        },
        "defaultOAuth": {
            "type": "bool"
        },
        "networkAclsBypass": {
            "type": "string"
        },
        "networkAclsDefaultAction": {
            "type": "string"
        },
        "routingChoice": {
            "type": "string"
        },
        "isHnsEnabled": {
            "type": "bool"
        },
        "isContainerSoftDeleteEnabled": {
            "type": "bool"
        },
        "containerSoftDeleteRetentionDays": {
            "type": "int"
        },
        "isShareSoftDeleteEnabled": {
            "type": "bool"
        },
        "shareSoftDeleteRetentionDays": {
            "type": "int"
        }
    },
    "variables": {},
    "resources": [
        {
            "name": "[parameters('storageAccountName')]",
            "type": "Microsoft.Storage/storageAccounts",
            "apiVersion": "2021-06-01",
            "location": "[parameters('location')]",
            "properties": {
                "accessTier": "[parameters('accessTier')]",
                "minimumTlsVersion": "[parameters('minimumTlsVersion')]",
                "supportsHttpsTrafficOnly": "[parameters('supportsHttpsTrafficOnly')]",
                "allowBlobPublicAccess": "[parameters('allowBlobPublicAccess')]",
                "allowSharedKeyAccess": "[parameters('allowSharedKeyAccess')]",
                "defaultToOAuthAuthentication": "[parameters('defaultOAuth')]",
                "networkAcls": {
                    "bypass": "[parameters('networkAclsBypass')]",
                    "defaultAction": "[parameters('networkAclsDefaultAction')]",
                    "ipRules": []
                },
                "routingPreference": {
                    "routingChoice": "[parameters('routingChoice')]",
                    "publishMicrosoftEndpoints": false,
                    "publishInternetEndpoints": false
                },
                "isHnsEnabled": "[parameters('isHnsEnabled')]"
            },
            "dependsOn": [],
            "sku": {
                "name": "[parameters('accountType')]"
            },
            "kind": "[parameters('kind')]",
            "tags": {
                "name": "function-app-storage-acc"
            }
        },
        {
            "name": "[concat(parameters('storageAccountName'), '/default')]",
            "type": "Microsoft.Storage/storageAccounts/blobServices",
            "apiVersion": "2021-06-01",
            "properties": {
                "containerDeleteRetentionPolicy": {
                    "enabled": "[parameters('isContainerSoftDeleteEnabled')]",
                    "days": "[parameters('containerSoftDeleteRetentionDays')]"
                }
            },
            "dependsOn": [
                "[concat('Microsoft.Storage/storageAccounts/', parameters('storageAccountName'))]"
            ]
        },
        {
            "name": "[concat(parameters('storageAccountName'), '/default')]",
            "type": "Microsoft.Storage/storageAccounts/fileservices",
            "apiVersion": "2021-06-01",
            "properties": {
                "shareDeleteRetentionPolicy": {
                    "enabled": "[parameters('isShareSoftDeleteEnabled')]",
                    "days": "[parameters('shareSoftDeleteRetentionDays')]"
                }
            },
            "dependsOn": [
                "[concat('Microsoft.Storage/storageAccounts/', parameters('storageAccountName'))]",
                "[concat(concat('Microsoft.Storage/storageAccounts/', parameters('storageAccountName')), '/blobServices/default')]"
            ]
        }
    ],
    "outputs": {}
}
Posted by at No comments:

Tuesday, August 24, 2021

azure function- arm

 Function App

by Microsoft
Details
Subscription    xxxxxxxxxxxxxxb
Resource Group    xxxxxxxxxxxxx-grp
Name
poc-swamyt
Runtime stack
Java 8.0
Tags
name: function-app
Hosting
Storage (New)
Storage account
storageaccountoctopaa67
Tags
name: function-app
Plan
Plan type
App service plan
Name
ASP-xxxx-test-app-svc-8941
Operating System
Linux
Region
East US 2
SKU
Premium V2
Size
Small
ACU
210 total ACU
Memory
3.5 GB memory
Monitoring
Application Insights
Enabled
Name
xxxx-app-insights
Region
West US 2

{
    "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "sites_poc_swamyt_name": {
            "defaultValue": "poc-swamyt",
            "type": "String"
        },
        "serverfarms_ASP_octopus_test_app_svc_8941_externalid": {
            "defaultValue": "/subscriptions/xxxxx-xxx-xx-xx-xxx/resourceGroups/xxx-resource-grp/providers/Microsoft.Web/serverfarms/ASP-xxxx-test-app-svc-8941",
            "type": "String"
        }
    },
    "variables": {},
    "resources": [
        {
            "type": "Microsoft.Web/sites",
            "apiVersion": "2021-01-15",
            "name": "[parameters('sites_poc_swamyt_name')]",
            "location": "East US 2",
            "tags": {
                "name": "function-app"
            },
            "kind": "functionapp,linux",
            "properties": {
                "enabled": true,
                "hostNameSslStates": [
                    {
                        "name": "[concat(parameters('sites_poc_swamyt_name'), '.azurewebsites.net')]",
                        "sslState": "Disabled",
                        "hostType": "Standard"
                    },
                    {
                        "name": "[concat(parameters('sites_poc_swamyt_name'), '.scm.azurewebsites.net')]",
                        "sslState": "Disabled",
                        "hostType": "Repository"
                    }
                ],
                "serverFarmId": "[parameters('serverfarms_ASP_xxxx_test_app_svc_8941_externalid')]",
                "reserved": true,
                "isXenon": false,
                "hyperV": false,
                "siteConfig": {
                    "numberOfWorkers": 1,
                    "linuxFxVersion": "Java|8",
                    "acrUseManagedIdentityCreds": false,
                    "alwaysOn": true,
                    "http20Enabled": false,
                    "functionAppScaleLimit": 0,
                    "minimumElasticInstanceCount": 0
                },
                "scmSiteAlsoStopped": false,
                "clientAffinityEnabled": false,
                "clientCertEnabled": false,
                "clientCertMode": "Required",
                "hostNamesDisabled": false,
                "customDomainVerificationId": "xxxxxxx",
                "containerSize": 1536,
                "dailyMemoryTimeQuota": 0,
                "keyVaultReferenceIdentity": "SystemAssigned",
                "httpsOnly": false,
                "redundancyMode": "None",
                "storageAccountRequired": false
            }
        },
        {
            "type": "Microsoft.Web/sites/basicPublishingCredentialsPolicies",
            "apiVersion": "2021-01-15",
            "name": "[concat(parameters('sites_poc_swamyt_name'), '/ftp')]",
            "location": "East US 2",
            "dependsOn": [
                "[resourceId('Microsoft.Web/sites', parameters('sites_poc_swamyt_name'))]"
            ],
            "tags": {
                "name": "function-app"
            },
            "properties": {
                "allow": true
            }
        },
        {
            "type": "Microsoft.Web/sites/basicPublishingCredentialsPolicies",
            "apiVersion": "2021-01-15",
            "name": "[concat(parameters('sites_poc_swamyt_name'), '/scm')]",
            "location": "East US 2",
            "dependsOn": [
                "[resourceId('Microsoft.Web/sites', parameters('sites_poc_swamyt_name'))]"
            ],
            "tags": {
                "name": "function-app"
            },
            "properties": {
                "allow": true
            }
        },
        {
            "type": "Microsoft.Web/sites/config",
            "apiVersion": "2021-01-15",
            "name": "[concat(parameters('sites_poc_swamyt_name'), '/web')]",
            "location": "East US 2",
            "dependsOn": [
                "[resourceId('Microsoft.Web/sites', parameters('sites_poc_swamyt_name'))]"
            ],
            "tags": {
                "name": "function-app"
            },
            "properties": {
                "numberOfWorkers": 1,
                "defaultDocuments": [
                    "Default.htm",
                    "Default.html",
                    "Default.asp",
                    "index.htm",
                    "index.html",
                    "iisstart.htm",
                    "default.aspx",
                    "index.php"
                ],
                "netFrameworkVersion": "v4.0",
                "linuxFxVersion": "Java|8",
                "requestTracingEnabled": false,
                "remoteDebuggingEnabled": false,
                "httpLoggingEnabled": false,
                "acrUseManagedIdentityCreds": false,
                "logsDirectorySizeLimit": 35,
                "detailedErrorLoggingEnabled": false,
                "publishingUsername": "$poc-swamyt",
                "azureStorageAccounts": {},
                "scmType": "None",
                "use32BitWorkerProcess": false,
                "webSocketsEnabled": false,
                "alwaysOn": true,
                "managedPipelineMode": "Integrated",
                "virtualApplications": [
                    {
                        "virtualPath": "/",
                        "physicalPath": "site\\wwwroot",
                        "preloadEnabled": true
                    }
                ],
                "loadBalancing": "LeastRequests",
                "experiments": {
                    "rampUpRules": []
                },
                "autoHealEnabled": false,
                "vnetRouteAllEnabled": false,
                "vnetPrivatePortsCount": 0,
                "localMySqlEnabled": false,
                "ipSecurityRestrictions": [
                    {
                        "ipAddress": "xx/32",
                        "action": "Allow",
                        "tag": "Default",
                        "priority": 2147483641,
                        "name": "Global Innovation Lab"
                    },
                    
                    {
                        "ipAddress": "Any",
                        "action": "Deny",
                        "priority": 2147483647,
                        "name": "Deny all",
                        "description": "Deny all access"
                    }
                ],
                "scmIpSecurityRestrictions": [
                    {
                        "ipAddress": "xxxx/32",
                        "action": "Allow",
                        "tag": "Default",
                        "priority": 2147483641,
                        "name": "Global Innovation Lab"
                    },
                    
                    
                    {
                        "ipAddress": "Any",
                        "action": "Deny",
                        "priority": 2147483647,
                        "name": "Deny all",
                        "description": "Deny all access"
                    }
                ],
                "scmIpSecurityRestrictionsUseMain": false,
                "http20Enabled": false,
                "minTlsVersion": "1.2",
                "scmMinTlsVersion": "1.0",
                "ftpsState": "AllAllowed",
                "preWarmedInstanceCount": 0,
                "functionAppScaleLimit": 0,
                "functionsRuntimeScaleMonitoringEnabled": false,
                "minimumElasticInstanceCount": 0
            }
        },
        {
            "type": "Microsoft.Web/sites/hostNameBindings",
            "apiVersion": "2021-01-15",
            "name": "[concat(parameters('sites_poc_swamyt_name'), '/', parameters('sites_poc_swamyt_name'), '.azurewebsites.net')]",
            "location": "East US 2",
            "dependsOn": [
                "[resourceId('Microsoft.Web/sites', parameters('sites_poc_swamyt_name'))]"
            ],
            "properties": {
                "siteName": "poc-swamyt",
                "hostNameType": "Verified"
            }
        },
        {
            "type": "Microsoft.Web/sites/snapshots",
            "apiVersion": "2015-08-01",
            "name": "[concat(parameters('sites_poc_swamyt_name'), '/2021-08-24T12_56_50_4269006')]",
            "dependsOn": [
                "[resourceId('Microsoft.Web/sites', parameters('sites_poc_swamyt_name'))]"
            ]
        }
    ]
}
Posted by at No comments:

Monday, March 1, 2021

Cloud Comparisons by Resource

 AWS                                                                Azure                                            Google



(1) ADF Trigger invokes API through ADF Web Activity based on schedule

(2) App Service invoke IM Datalink Service and process the data using transformation logic

(3) App Service Generate File (FIX) format and send to COMET SFTP Server

(4) Pager Duty integrated with APIM(Amber) so all notifications can be centralized for connectiviy failures

(5) Developer invoke the API to perform end to end testing on developer machine

(6) AZDO is being used for both Application code and infra code deployments in CI/CD fashion

Posted by at No comments:

Azure: Setup OAuth2.0 between ADF and APIM for secured and authenticated traffic

  •  Create Service Principal - to be presented to APIM for Auth token
    • Create App Registration
    • Under Token configuration, add optional claim to ensure only app call end point not the user/human
    • No explicit permission needed
    • Add the Application URI
    • No explicit scopes needed
    • No RBACs as resources level as this is for API not users

    • SPN Name: dept-proj-env-oauth
    • SPN API Permissions: none
    • SPN RBAC permissions: none
    • SPN RBAC permission scope: none

  • Create Policy for APIM to validate request
    • Write the policy, 
    • Route the backend if request successful
  • ADF pipeline to use api://imdev-dev part of the REST request ( which is also application URI for SPN)

Posted by at No comments:

Thursday, August 6, 2020

Azure Databricks

 It is an Apache Spark based analytics paltform optimized for Microsoft axzure



key features

  1. Spark SQL lirabry
  2. Streaming services for OIT et
  3. MLIB machine leaning libs
  4. Graph Computation


Spark core API supports

  1. R
  2. SQL
  3. Python
  4. Scala
  5. Java

Azure Databricks as a pltform contains.

  1. Datbricks workspace
  2. Datbrcisk workflows
  3. Databricks Runtime
  4. Databricks I/O
  5. Datbricks Serverless
  6. Dataricks Enterprise Security (DBES)
Storage solutions supported
  1. Blob storage
  2. Data lake
  3. SQL DW
  4. Apache kafka
  5. Hadoop
Applications Supported
  1. ML
  2. Streaming
  3. Data 
  4. Power BI
  5. Others
Users
  • Data Scientist
  • Data engineers
  • Analysts
  • othes



Posted by at No comments:
Labels:
Subscribe to: Posts (Atom)