Azure File Sync is a new Azure feature, still in preview, that allows to sync a folder between your local file server and Azure Files. This way your files are accessible both locally at your file server and publicly at Azure Files using an SMB 3.0 client. Also the files can be protected online using Azure Backup.
The idea of this post is to have the files of two file servers to sync to Azure Files using Azure File Sync and in addition use the DFS Namespace feature to achieve common name and availability. This is not something officially supported, it is just an idea on using two different technologies to help for a service.
The requirement before starting the Azure File Sync is to create an Azure File share. We have covered this at a previews post, check here
Once the Azure Files share is ready, proceed with the Azure File Sync resource. At the Azure Portal press New and search for it and create it.
At the Deploy Storage Sync blade select a name for the Resource, subscription, resource group and location.
When the Azure File Sync is ready we need to create a Sync group. Sync group is something like the DFS Replication Group. It is a group that consists of an Azure File Share and many local file servers that syncs a folder.
Press “+Sync group” it will open the new “Sync group” blade. There provide a name for the Sync group and select the storage account and the Azure File Share created before.
The Sharegroup is ready with the cloud endpoint. The next step is to add the first local file server.
Register the local servers
Navigate to https://docs.microsoft.com/en-us/azure/storage/files/storage-sync-files-server-registration for information on how to download the agent, install it and register the server. After that press “Add server endpoint”
At the “Add server endpoint” blade, select the registered server and add the path to the folder that has the data you want to sync. With Cloud Tiering you select a percent of the volume of the local server. When the capacity of the volume reaches this number then Azure File Share makes the files that are less frequently accessed cloud only. The file icon on the server get transparent and if anyone double clicks the file then it is downloaded instantly.
Register the second server the same way as the first and finally the share group will have two server endpoints. At my example the second server had no data, just the folder, and the Azure File Sync synced all files from server A.
Create a DFS Namespace
The next step is to create a DFS Namespace, just the namespace with the two local servers. Add the folders of both servers and you are ready.
Also if you browse the Azure File Share, all files are accessible
Notes from the field
Adding or changing a file at the first server, almost instantly replicates to Azure File Share and to the second server.
Altering a file at both servers instantly it will keep the last accessed by timestamp as is and the other file will be renamed by adding the server name at the file name, as the example “enaneoarxeio-AzureFS2.txt” where AzureFS2 is the server name.
You can add an Azure Backup and have a Cloud Backup of all your files.
Create Azure File Shares at your ARM template using PowerShell
Using Azure Resource Manage template deployment, you can create a Storage account but you cannot create File Shares. Azure File Shares can be created using the Azure Portal, the Azure PowerShell or the Azure Cli.
Mainly, the idea is to run a PowerShell script that will create the File Shares. This script will be invoked inside the ARM Template. In order to use a PowerShell script from a template, the script must be called from a URL. A good way to provide this is using the Git repository. One major thing to consider is the Storage Account key must be provided to the PowerShell script securely, since the PowerShell script is at a public URL.
The PowerShell script will run inside a Virtual Machine and we will use a CustomScriptExtension Extension to provide it. To use this, at the Virtual Machine Resource of the JSON file add a resources section.
The Custom Script Exception is located at the Virtual Machine resource. Lets assume that the last part of the Virtual Machine resource is the “diagnosticsProfile” so after the closure of the “diagnosticsProfile” we can add the “resources”. Inside the “resources” add the “extensions” resource that will add the “CustomScriptExtension”, like below.
The Template Part
This will be the addition at the Virtual Machine resource:
"storageUri": "[concat(reference(concat('Microsoft.Storage/storageAccounts/', variables('diagnosticStorageAccountName')), '2016-01-01').primaryEndpoints.blob)]"
"commandToExecute": "[concat('powershell -ExecutionPolicy Unrestricted -File ','azurefiles.ps1 -SAName ',parameters('AzureFilesStorageName'),' -SAKey ', listKeys(resourceId(variables('AzureFilesStorageAccountResourceGroup'),'Microsoft.Storage/storageAccounts', parameters('AzureFilesStorageName')), '2015-06-15').key1)]"
The extension must be depended from the Virtual Machine that will run the script and the Storage Account that will bu used for the file shares.
At the custom script properties add the public RAW url of the PowerShell script.
Next lets see the Storage Account key and execution part. At the connandToExecute section, we will provide a variable that will pass the Storage Account key & Name inside the script for execution. The variable will get the Storage Account key from the Storage Account using the permissions of the Account running the Template Deployment.
Of course to make the template more flexible I have added a variable for the Resource Group and a parameter for the AzureFilesStorageName, so the template will ask for the Storage Account name at the parameters.
The PowerShell script is tested at Windows Server 2016 VM. You can find it below:
Install-PackageProvider -Name NuGet -MinimumVersion 188.8.131.52 -Force
Set-PSRepository -Name PSGallery -InstallationPolicy Trusted
Install-Module Azure -Confirm:$False
$storageContext = New-AzureStorageContext -StorageAccountName $SAName -StorageAccountKey $SourceSAKey
$storageContext | New-AzureStorageShare -Name #####
The post Create Azure File Shares using ARM template & PowerShell appeared first on Apostolidis IT Corner.
Azure Web Farm using IIS & Azure File storage
This post is my view of a complete guide, from A to Z, including both the Windows Server part and the Azure Portal part on how to build a Web Farm using IIS & Azure File storage. Following this guide you will have a functional two server IIS Web Farm using Azure File storage.
To create a Web Server Farm the mail requirement is a high available common storage. I see that when deploying on Azure, a lot of people are using DFSR for common storage. This solution has two major issues, the first is the cost, because you need at least two Windows Servers and second the replication speed that many times doesn’t cover the web servers needs.
Azure File storage provides a very fast, high available and cheap solution to create a web server farm on Azure.
Azure File storage supports both SMB 2.1 & 3.0 protocol. Every share can be up to 5TB, storing unlimited number of files and each file can be up to 1TB. Of course we can create as many shares as needed up to 500TB that is the Storage Account limit. Also every share provided 1000 IOPS.
For this guide I used one Windows Server 2016 for Domain Controller, two Windows Server 2016 for web servers and one Standard Storage Account for Azure Files. Of course you can create an IIS Web Farm with workgroup server, but I will cover this at an other post.
I will skip the part of creating the Azure VMs, the domain and joining the servers on it, so I assume that there is a domain and the other two servers are member servers of the domain.
Just a note, when creating the Azure VMs that will be part of the IIS Farm remember to add them to the same Availability Set. This will inform the Azure Fabric to have the VMs to different update & fault domains
Step 1. Add the Web Server role
From the Server Manager, click Add roles and features and add the “Web Server (IIS)” role and at the next screen add all the features your app needs.
Step 2. Add Application Request Routing
For more feature and control over the IIS load balance add the Application Request Routing (ARR) using the Web Platform Installer. The Web Platform Installer uses the Internet Explorer engine so I disable the “IE Enhanced Security Configuration” just for the installation and then enable it. You can do it from the Server Manager, at the Server section.
Next, open the IIS Manager and at the Actions Panel (on the right) click the “Get New Web Platform Components”
Search for “arr”, and add the Application request routing 3.0
Step 3. Create an Azure Storage Account for Azure File Storage
To create the Azure File storage shares, go to the Azure Portal, click the + button and create a Standard Storage account (you can just search for storage account)
Select General purpose and Standard performance (Premium does not support Azure Files)
once the storage account is created, open it and click the “Files” button
Click the “+ File share” to add a file share
Give it a name and quota limit
After clicking OK the share will be created and it will be listed at the shares blade.
Click on it to get the connection string and then click Connect to get the access key. At this point copy the “Connecting from Windows” command. We will use this to mount the share to the web servers.
Finally click the File Share and at the new blade press “+Add Directory” and create two directories, one Configuration and one Data
Step 4. Configure Windows Firewall & NSG
Open the port 80 (and nay other requires ports) at the Windows firewall and at the NSGs
for the windows firewall, go to the Servers’s control panel, Windows Firewall and click “Advanced Settings”
go to the Inbound Rules, add New Rule and select Port
Select TCP 80 (and any other needed for your application/site)
Allow the rule for all profiles, provide a name and press finish to create the rule
For the NSGs , go to the Azure Portal, open the NSGs of the web servers and add an http allow rule like below
Step 5. Create the IIS Shared Configuration
Go back to the first web server, open Computer Manager and add a new user with username and password the same as the connection string of the Azure Files share. You will need this for the IIS Service to access the Azure File storage share.
Add the user to the IIS_IUSRS group
Then open the IIS Manager and click the “Shared Configuration”
at the Action pane click “Export Configuration”
add the Azure File storage Share UNC path and “Connect As” using the share’s credentials
Enter a complex encryption key and press ok to export the configuration to the Azure File storage “configuration” share
Now we can enable the Shared Configuration, add the Configuration Share path and the same credentials and click apply.
it will ask for the encryption key and finally the Shared Configuration will be enabled. You will need to restart the IIS Manager console
at this point we can change the path of application pools, ftp applications etc to the Data folder share of the Azure File storage. Just note that you will need to use the UNC path “\\azurefilesol.file.core.windows.net\iisfarm\Data” and not any mapped drive.
Repeat the above steps to the second web server. At the Shared Configuration don’t export the configuration, just enable the Shared Configuration.
Finally we can create the Farm, from the IIS Manager, right click at the “Server Farms” and click “Create Server Farm…”
give a farm name,
add both servers and press Finish
Press NO at the URL rewrite rule auto creation
And the farm is ready
as mentioned before, deploy your Web Site, Web Application etc to the Data file share, always using the full UNC path “\\azurefilesol.file.core.windows.net\iisfarm\Data”. All the website data will reside at the Azure File storage.
If you need more than 1000 IOPS create more Azure Files storage shares and divine your website/application data.
Run the following command to allow access to the Azure File storage. Change the file://”path” as needed
C:\Windows\Microsoft.NET\Framework64\v4.0.30319>caspol -m -ag 1. -url file://azurefilesol.file.core.windows.net/iisfarm/* FullTrust
Step 5. Azure Load Balancer
Finally, create a Load Balancer to distribute incomming traffic to the farm. For this guide I will create an External Load Balancer, but also an Internal Load Balancer is supported.
First, find and create the Load Balancer from the Azure Portal
provide a name, for external load balancer select Public, select Dynamic or Static IP, resource group, location and press Create
Then go to the new load balancer and create a health probe
create an http probe
provide a name, select Availability Set for association and add both the Web Servers
finally create a load balancing rule to load balance the TCP port 80 at the farm
once the load balance rule is created you can browse the public ip / name of the load balancer
Now, browse to the load balancer’s IP and you are good to go!