Using Azure PowerShell and PowerShell 3 or 4? You need to update PowerShell SOON (probably).

I recently saw a GitHub Pull Request (https://github.com/Azure/azure-powershell/issues/5149) for the Azure PowerShell cmdlets (the PR was merged into Version 5.3.0 of the Azure cmdlets). Besides from the continuing improvements that each version brings, I noted one very interesting sentence: 'PowerShell version 3 and 4 will no longer be supported starting in May 2018. Please update to the latest version of PowerShell 5.1 '

What does this mean? Well – it means that after May this year, if you are running either PowerShell V3 or V4, new versions of the Azure cmdlets may not longer work – and are not supported in any case. That is not to say that either the sky is going to fall in! Older versions of the cmdlets should continue to work and most of the newer cmdlets should work too. Note the ‘should’. But why take the risk. You have several months before the lack of support begins. But you should start to plan now if you are still using PowerShell V3 or V4 to manage Azure.

So what should you do? The answer should be fairly obvious – if you are using Azure and the Azure cmdlets, just upgrade to the latest version of PowerShell (i.e. 5.1). This new version of PowerShell should work just fine on all supported versions of Windows. Of course, if you are still using XP to manage Azure then you may have some issues trying to upgrade, although an OS upgrade to Windows 10 would fix this problem.

The upgrade of PowerShell should be a no brainer. I suspect many (most?) readers here are already running later versions!  There should be no issue, but if you are using Exchange, tread carefully to ensure that the version of PowerShell you are thinking of upgrading to is going to work with and is supported by your version of Exchange.  This is probably not going to be an issue if you using hosted Exchange (O365).

It seems to me that this is the start of removing all support for PowerShell V3 and V4. V5 and V5.1 are sufficiently better to make the upgrade most welcome. Loads more cmdlets, improvements in workflows etc are all goodness that comes from the Upgrade.

What is your take?

The Great Azure Cmdlet Renaming – Coming Soon

As many readers of this blog know, I am a fan of the Azure module – it makes it simple and relatively straightforward to automate just about all operations in Azure. I've written scripts to do all sorts of things, including creating LInux and Windows VMs, created web sites, worked with networking, etc. I have a lot of scripts out there – and soon, I am going to have to look at each and every one, and probably change most if not all of them. The Azure module is about to change in a very big way – and this could become work for some!

To understand the renaming that is about to happen, we must start at the beginning. In developing Azure, Microsoft created a REST API, the Azure Service Management (ASP) API. A set of cmdlets were created that interfaced with the API to enable automation via PowerShell of the Azure components like VMs. I use these cmdlets most days and they rock. There is room for improvement, but they work and work well.

But the Azure team created a second API, the Azure Resourced Manager (ARM) API. And the Azure module was changed to allow Azure ARM based cmdlets. The ARM API has a much different look and feel – and provides some features not available in the ASM version, particularly templates and resource groups. As an aside, I can only surmise that if thee Azure team were to go back in time, knowing what they know now, they may never have built the ASM API, going straight to ARM. But they didn't.

The impact to the user of the Azure module is how you call cmdlets targeting ARM vs. ASM. For a start, the two cmdlet sets were never meant to be loaded or used together. When you load the Azure module, you get a module of cmdlets, etc, that are targeted at the ASM API. To get to view the ARM based cmdlets, you use the Switch-AzureMode cmdlet, and  using the –Name parameter, you specify whether to use the expose  AzureResourceManager or AzureServiceManagement API based cmdlets. Both cmdlet sets feature cmdlets with the same name that do more or less the same thing, but are in compatible. You can't really mix and match between the cmdlet sets. With the latest Azure module, it looks like this:

 

As you can see, there a different number of cmdlets in each set, and when you use Switch-AzureMode, it removes one set of cmdlets and load a totally different module (where did that come from – we never loaded or installed an AzureResourceManager module! The magic of the Azure module makes this all work – and since scripts tend to only use one or other API set, no real harm done.

But it is confusing and from a pure PowerShell point of view, the basic design was flawed – it does need cleaning up. And, going forward, that is what Microsoft is going to do. In a post entitled Deprecation of Switch-AzureMode  in Azure PowerShell, Joe Levy spells out the need for the change and how it will happen.

The key changes are broadly as follows:

• The Azure module will be renamed to “ASM”. And all ASM cmdlets will be prefixed with “ASM”, so Azure/New-AzureVM will become ASM/New-AsmVM or simply New-AsmVM.
• The existing AzureResourceManager cmdlets will keep their existing names, so New-AzureVM will map to the ARM version.
• The existing AzureResourceManager module will be broken into many modules by service and behaviour. Examples of the module names would be the following: AzureCompute, AzureNetwork, AzureStorage, …
• Azure PowerShell and all of its modules will be distributed via MSI and PS-Get. MSI cadence will decrease because the MSI will be a wrapper for a PS-Get script.

So if you have written any scripts using ASM cmdlets, these will need to be updated to use the new cmdlet names. If you have written scripts using the ARM modules, these should not need to change much, but some change is inevitable (erg removing any Switch-AzureMode statements you have!).

If you have done ANY scripting with Azure and PowerShell, you should read the article carefully and start to work out your strategy for the updates that will be coming. The plan, at the moment at least, is that this will all happen over the next couple of months. The ARM decomposition should be done in August, and the Azure module rename will happen in September.

del.icio.us Tags: PowerShell,Azure,Azure Cmdlets

And Another Azure PowerShell Release

The other day, I joked via twitter that if you blink, you would miss yet another new Azure feature. Well nearly! Just 6 days ago, I noticed (and tweeted) that there was a new release of Azure PowerShell. Excitingly, the version was 0.9.0. Lots of changes (a total of 469 separate commits) and loads of new cmdlets. Release 0.9.0 has 625 cmdle5ts and 83 aliases (in service management mode), and 336 cmdlets in Azure Resource Manager mode.

Well, just 5 days later, Microsoft released an updated version of the cmdlets – I am assuming this is a drop especially for Ignite. The new build, version 0.9.1, installs nicely over the earlier build. The new version has 8 additional cmdlets in service management mode, but a total of 448 cmdlets in resource management mode (or 112 new cmdlets!). The new MSI weighs in at 19.9mb!

The main thrust of both 0.9.0 and 0.9.1 have been to add new cmdlets to Azure Resource Manager mode.  This involves using Azure Templates about which is a subject for another blog post!

del.icio.us Tags: Azure,Azure PowerShell cmdlets

New Version of Azure PowerShell Module

As I was writing another blog post last night, I discovered that the Azure PowerShell module has been updated – the new version is 0.8.16. The Azure module now boasts 606 cmdlets (and 35 aliases).  The updated module has some new cmdlets (notably for Azure Insights), bug fixes and some changes to the Azure Websites cmdlets that partly reflect the removal of Web sites and their replacement (web apps).

You can get the updated module from https://github.com/Azure/azure-powershell/releases. This page features a download link to both the (updated) web platform installer and to a standalone MSI for just the module.  The page also has some more details on what's new.

Interestingly – the updated module still supports for Azure web sites (e.g. New-AzureWebsite) despite the GUI (the Azure portal and the Azure preview portal) having dropped these in favour of the newly announced Web Apps functionality that in effect subsumes the older Web Sites feature). The Azure portals no longer show Web sites, whereas the cmdlets still support the feature. One noticeable and important difference here – the web site cmdlets do not appear to support the gallery function. So while creating a web site is pretty trivial, creating it via a gallery (e.g. a WordPress site complete with MySQL) does not appear possible (yet).

del.icio.us Tags: Azure,Azure web sites,Azure web apps,Azure PowerShell cmdlets,PowerShell Blog Week

Azure Cmdlets and the Debug Switch

So here is is, late on a Saturday and I'm off in the morning to teach an Office 365 class. In a fit of overexcitement, I decided to try to build out the lab environment on my laptop before travelling to the class. The class is run as a long-day boot camp and in the past the start of the course has been challenging with issues relating to building out the environment. I hoped to avoid that this week!

This class is one of the first from Microsoft to utilise Azure. The student's "on premises" data centre is in Azure, and the course looks at adding Office 365 into the mix. It's a fantastic idea – the student just runs a few scripts and hey presto – their live environment (DC, Exchange, etc, etc, etc) is all built by script as part of the first lab. And amazingly – this just takes around two hours according to the lab manual. What could possibly go wrong?

Well – the last time, those two hours turned into 6 as we had some errors at the start with the first script not running properly. Of course, I was able to troubleshoot and fix the issues although it did take time. So I thought it might be a clever idea to re-test the scripts this time and maybe avoid the issues.

I downloaded the first VM and installed it onto my Windows 10 client, then started running the build scripts. All was going well, till I tried to run the script to create the DC. For several hours, each time I try to create the VM, i would get error: "CurrentStorageAccountName is not accessible. Ensure the current storage account is accessible and in the same location or affinity group as your cloud service." All the normal methods (i.e. looking at Google and reading the articles found) did not help. None of the suggested fixes were appropriate to me. After two hours I was stuck.

Then, by this time on the third page of Google results, I came across a trick that should have occurred to me earlier. Use the-Debug Switch on the call to New-AzureVM (the cmdlet that was failing). I was able to see the HTTP traffic which relates to the underlying REST management api  leveraged by the Azure PowerShell cmdlets.

What I saw was the client asking for my storage account details – but claimed that the storage account did not exist. On close inspection, I could see the storage account name being requested was almost, but not quite, the correct ID. In creating the storage account, the script asked for user input (partner ID and student id), then created a storage account with a unique per/student name 0 a nice touch. In my case, I entered a partner number beginning with a 0 (zero). And looking closely at the script, part of it strips off the leading zero and used the zero-less storage account name for later operations – and of course that fails. To fix things, I just removed everything created thus far from Azure, and re-ran the scripts utilising better input and away I want.

There are two lessons here: first (as I tell all my students in almost every PowerShell Class): all user input is evil unless tested and proved to be to the contrary. If you are writing a script that accepts user input, code defensively and assume the user is going to enter duff data. The script should have checked to ensure the partner id entered did not start with a zero – it's a production script after all. Of course, I probably should have, and eventually did use a partner number not starting in Zero. So the underlying cause is user error. Still a good lesson is that, given a chance, most users can do dumb things and you need to write scripts accordingly.

The second one is the value of the –Debug switch when using any Azure PowerShell cmdlet. There can be quite a lot of output, but the ability to see what the cmdlet is doing can be invaluable. In my case, it took only seconds to work out the problem once I'd seen the debug output from the call to New-AzureVM. I recommend you play with this, as you get more familiar with Azure PowerShell – it can certainly help to in troubleshooting other people's scripts!

del.icio.us Tags: Azure,Azure PowerShell,-Debug switch,debugging Azure PowerShell scripts

Azure Premium Storage

Before Christmas, Microsoft added a bunch of new features to Azure. One that I've only really just noticed was Premium Storage. Azure provides several types of cloud storage (blogs, queues, tables and files) – with the files storage still in preview. Premium storage too is in Preview.

The basic idea if Azure Storage is that you can store your data in the cloud: whether that data is the VHD drive for an Azure VM, a message queue used to hook up different parts of an application, or whatever. Azure storage is a fundamental building block you use to create cloud computing systems based on Azure.

The new premium storage feature now allows you to store this data on SSD disks. This new storage option provides higher performance and lower disk latency. Not only that, but, at least during the preview, Microsoft offered three types of SSD: P10, P20, and P30. These disks are 128gb, 512gb and 1tb respectively. The bigger disk provide more IOPS and greater throughput.

I look forward to playing a bit more with Premium Storage! I suppose it goes without saying: you can easily use Azure PowerShell cmdlets to manage this storage. I hope to generate a few scripts to demonstrate this!

For more details on Azure Storage, See Sirius Kuttiyan's recent blog post. For fuller details on Azure Storage pricing, see: http://azure.microsoft.com/en-gb/pricing/details/storage/. Premium storage is still in preview, and is offered at a bit of a discount. The 128GB  P10 disk, for example, is £5.47/month, while the 1TB P30 is £37.54/month.

del.icio.us Tags: Azure,Azure Storage,Premium Storage

Adding an Endpoint to an Azure VM

In a recent blog article, I showed you how you can create an Azure VM. If you are familiar with Hyper-V and the Hyper-V cmdlets – the approach is a little different (since you configure Azure differently). One aspect of creating a VM is how you, in effect, open ports to an Azure VM.

With Azure, you create an end point that, in effect, creates a public port and an internal port for your VM. You create one port, which is the public port (on the Azure service) and an internal port for the VM. The approach to doing this is pretty straightforward, although it's a pipelined pattern:

Get-AzureVM -ServiceName $ServiceName -Name $VmName  |
  Add-AzureEndPoint -Name "Http" -Protocol "tcp" -PublicPort 80 -LocalPort 80  |
    Update-AzureVm

This pattern has you first get the Azure VM object, to which you add an Azure endpoint. Then the updated object is piped to Update-AzureVm which adds the new endpoint to the VM. Of course, if the end point already exists, this pattern would throw an exception. With Add-AzureEndPoint, you specify the protocol, and public and external ports. This 'one-liner' creates a public port and an internal (local) port to enable the VM to serve HTTP traffic.

I've created a simple script, over on http://pshscripts.blogspot.com, that implements a New-HttpVmEndpoint function which you can use to add a new HTTP endpoint to a virtual machine. This script omits some error handling you might wish to add in. For example, you might want to check whether the end point already exists. or whether the VM and VM service exist. You could obviously extend that script to add other endpoints (eg HTTPS).

del.icio.us Tags: Azure,PowerShell,VM Endpoint,script

PowerShell Patterns and Azure Cmdlets

Get-Process Power* As anyone learning PowerShell quickly finds out, there are a number of basic patterns of usage that work well across a huge range of PowerShell cmdlets/modules. One of the most basic pair of patterns I try to teach, by way of example here, are early vs late filtering. The following two commands accomplish the same result:

Get-Process Power*
Get-Process | Where-Object name -like Power*

I wrote a little script to measure the performance difference between these two (http://pastebin.com/4N2YYqnZ). Running each of these 1000 times, the result was that early filtering was 3 times as fast as later filtering. So as a general rule – always early filter. BUT: sadly some cmdlets that emit useful objects do not support early filtering, so the later filtering pattern always works. I try to teach both of these patterns and why they are different since so many scripts can make use of these patterns.

Which brings me to Azure. In much of my PowerShell use, there is a cmdlet to set some value for some object or perform some operation on an object. For example, to change the number of CPUs in a Hyper-V VM, you use the Set-VM cmdlet, like this:

To do this in Azure is a bit more complicated. To change the number of processors in an Azure VM, you need to change a different property on the Azure VM object. Plus, the say you do it is different. With Azure, each VM runs in a given instance. The instance determines the number of CPUs and memory each Azure VM gets. So I would expect to do something like this:

Set-AzureVM –name xxx –service xxx –instancesize 'extrasmall'

Unfortunately, that cmdlet does not exist, and that wouldn't be the right way to do it anyway. The good news is that in Azure, there is a specific cmdlet to change the instance size (Set-AzureVMSize). So Azure VMs, the most common pattern is like this:

Get-AzureVM –VM psh1 –Service psh1 -Verbose |
  Set-AzureVMSize -Instancesize $VmSize   -Verbose |
     Update-AzureVM -Verbose

A different pattern. Unlike Set-VM, Set-AzureVMSize takes an Azure VM object (not the name of the VM). Secondly, Set-AzureVMSize does not persist the updated instance size value – it just updates an in memory object relating to the VM. To persist the value you have to write it back to Azure using Update-AzureVM. While you can do it step at a time, and avoid using the pipeline – in this case, using the pipeline makes seems easier.

In the Azure VM cmdlets, we see a great deal of use of the Pipeline based patterns. For example, to create an Azure VM, you first create a VM config object (Technically, New-AzureVMConfig produces an object of type Microsoft.WindowsAzure.Commands.ServiceManagement.Model.PersistentVM) which you then pipe to New-AzureVM (see http://tfl09.blogspot.co.uk/2015/01/managing-azure-vms-with-powershell.html). This general pattern of getting or creating an in-memory object, updating that in-memory object typically via the pipeline and finally persisting it to Azure is easy to apply.

When you start to use Azure you quickly find that while many of the features of Azure are similar to those in Windows – in this case, Azure VMs – the methods you use to manage the features does differ. You  have different patterns of usage you need to learn and understand.  When I explained this to a class this week, I got the obvious question: why? Why is it different?

Since I was not on the dev team, I can't really answer that. But, I suspect the main technical reason is that the Azure VMs are, in effect, a wrapper around the REST API exposed by Azure. I am guessing it was easier to write the cmdlets using this pipeline pattern (or the cmdlet's approach of passing a VM object vs a VM name). Having said this, incorporating the Get-AzureVM and the Update-AzureVM cmdlets inside the Set-AzureVMSize cmdlet would have not been that difficult. But that wasn't what MS implemented.

So go with the flow; learn the patterns and enjoy Azure!

del.icio.us Tags: PowerShell,Azure

Updated Azure PowerShell Module

I've been playing a lot with the Azure cmdlets and noticed that there's an updated version of the module. My recent Azure related posts have been based on the version of the tools Ioaded before the New Year:

If you got to the download page, http://go.microsoft.com/fwlink/p/?linkid=320376&clcid=0x409, and run the platform installer, you see an updated version of the tools is available:

The download is around 13 mb and took around 45 seconds to download and install. After installation:

Sadly, there appear to be no release notes, or readme.txt file in the installation folder. Given the huge amount of change, better release notes would be useful. AND it would be nice to have the latest module available via PowerShellGet. We can hope!

del.icio.us Tags: Azure,cmdlets,0.8.13

Azure PowerShell DSC Extension

The PowerShell team have just published a new DSC extension to cater for Azure. You can find more information from the PowerShell team's blog post. he blog post shows yow you can use this extension. Before you use these cmdlets, you will need to have the Azure PowerShell SDK (0.8.6 or later) installed. See here to download the SDK.

del.icio.us Tags: Azure,PowerShell,DSC

More PowerShell Cmdlets for Azure

I just notices that Cerebrata have brought out a set of cmdlets for Azure. For an introduction, see: http://www.cerebrata.com/products/azure-management-cmdlets/introduction. From here you can also download the cmdlet.

These cmdlets are, to say the least, NOT cheap. UK Price is over £80 pounds. I suppose if you are doing a lot of work with Azure and PowerShell this is good value. There are volume licensing options. Depending on your need, you may decide to use the Microsoft cmdlets and make up the difference yourself.

del.icio.us Tags: PowerShell,Azure

PowerShell and Azure

CloudComputingAdmin.com has a nice article discussing managing Windows Azure IAAS using PowerShell. The article discusses installing the latest version of the PowerShell cmdlets (0.7.4) and shows how to do some common operations, including creating an Azure VM and removing an Azure VM. If you are looking at or using Azure, then you should investigate these cmdlets.

del.icio.us Tags: PowerShell,Azure,cmdlets

Azure PowerShell Cmdlets

Further to my article I posted last week, Over the weekend I read a nice article on 'Getting the Windows Azure PowerShell Cmdlets. It's over on Michael Collier's blog, and explains how to get the cmdlets. To do this, the article explains, you navigate to the Windows Azure downloads page, click the link to “Install” the cmdlet, and let the Web Platform Installer (WebPI) do its thing.

Once installed, you have the Azure module which has hundreds of PowerShell cmdlets. You can use these to access and manage most of the Azure services. I really need to sit down and spend a few days getting to grips with these cmdlets!

del.icio.us Tags: Azure,Cmdlets,PowerShell