hckrnws

I'm still fuming from when Azure cancelled my organisations PowerBI A1 instance account, and refused to explain why. We re-opened the account and got the report working. They cancelled it again.

A scientific reporting project trying PowerBI dead in the water.

I was called from leave, from the beach in Thailand, to try and untangle this mess. I couldn't. I'm the "linux guy" but all the "microsoft people" were at their wits end and didn't know what to do. A colossal kafkaesque maze mess of Microsoft support agents with cheerful "have a nice day and is their anything else we can help you with" emails canning the project, over and over. AND REFUSING TO EXPLAIN WHY.

=============================

Greetings for the day!

I have received an update from Account Research Team: As part of our strong commitment to the protection of our customers and our interest in preserving the quality and integrity of the Azure Marketplace, we perform supplementary reviews of accounts which may exhibit irregular or suspicious activity. Your account was selected for one of these reviews and after careful consideration, this account will remain closed.

Please understand that we keep security checks like these in place in order to protect the quality and integrity of the Azure Marketplace

=============================================================

They can't have any suspicious activity if there's no activity! taps head meme

$DAYJOB is spending a lot of money very quickly in a modernisation project of our core product. It's being split into a micro-service architecture using Azure.

We've basically shipped 5% of the features but already to the point where compute and logging is costing "too much". I'm not sure who made the decision to jump to Azure without doing even back-of-the-envelope estimates on this type of thing.

My real annoyances is that the support in Azure is some of the worst bottom of the barrel crap I've experienced.

- You have to open a support ticket to raise your limits on your subscriptions - and prepare to wait weeks with zero communication from microsoft if your region is contended.

- I've had people from the third party support companies directly contact me on teams chasing for me to close their tickets... HIGHLY unprofessional in my opinion. Tickets they have not been able to complete because it's "impossible":

- You cannot delete 'views' of previous commits that may be leaking secrets in Azure DevOps. GitHub has an entire support category for this. Not for ADO, apparently it's impossible according to support. Oh well, leaked secrets in the GUI for our repository forever, despite the commits no longer existing.

- Even their architect experts are useless; we want to separate our SaaS from our company's Entra ID and even though it's one of the "legitimate" reasons listed in their docs (ISV), all of their so-called experts don't think this is wise, or at least is too bothersome for them to work out how entitlements work. Yes, let's pollute our corporate tenant with service principles etc. of our hosted software.

Overall it has been a garbage experience, meanwhile $DAYJOB is hurtling down this path for strategic reasons (boils down to the CEO being told by customer CEOs that you need to be in the cloud to be considered a real company, despite buying our services for decades).

Azure could be pretty good to be honest - it just has bad support and even worse technical sales/architect people.

If you're reading this and thinking "it can't be that bad:"

no, it really is.

On paper, Azure has everything you need and more.

In reality, using Azure is death by one hundred thousand cuts that's smoothed over by an army of support engineers, TAMs, and PMs (seriously!) providing the best support I have ever seen. Seriously, they are everywhere. They're probably reading this comment!

Enumerating them like this is SUCH A GREAT IDEA.

Here are some of the ones I experienced when I spent a year working seriously with Azure:

- I hope you like HTTP 409 CONFLICT

- Azure AD goes down ALL OF THE TIME. Everything in Azure and Office 365 relies on Azure AD. Somehow this is fine.

- One minute, your VM will take two minutes to create. Another minute, it will take 45. You have no way of knowing beforehand.

- Literally EVERYTHING about Azure ARM. You will not get nested templates right. Just give in.

- Try using Azure DevOps via CLI. You will love Jira more quickly than you thought possible.

“2 min or 45”

IME any vm operation via portal takes at least 15 minutes. Stop/start/restart/redeploy. It’s so consistent it feels like they just slept for 14 minutes then do the thing.

This isn't even going into the terrible security record Azure has had in the past two years. The only major cloud provider with cross-tenant security vulnerabilities, and tons of them at that!

Wouldn't you be able to do a similar list for any given provider, given sufficient motivation and rage? Or do you think Azure is distinct

You would, but as someone who has interacted with all three big cloud providers at some level, and has seen demos using all three, and has colleagues that have had to do work and/or demos with all three... Azure is by far the worst.

Their security is the worst - they have had more than 10 critical security vulnerabilities, most of them cross-tenant, some of them trivial to exploit and should have never gotten into production, just over the past 2-3 years.

Their UX and docs are extremely bad. Not that AWS/GCP are perfect, but at least they're actually usable - the docs can be read and understood, and the UI and API don't take years to answer every single call.

As an anecdote, demos from their own Solutions Architects leave the impression everything is a slow disjointed mess. As another anecdote, I don't know any single Platform/Infra/SRE/DevOps/Cloud person that actually likes Azure, while I know lots of people who evangelise AWS/GCP for various reasons.

The main reason to go to Azure is because your bosses' boss was convinced to buy it at a golf course.

> The main reason to go to Azure is because your bosses' boss was convinced to buy it at a golf course.

This is why SharePoint got installed in so many gov/large enterprise in the past, and I have to assume why Azure has any kind of market share today. If it wasn’t a golf course, it was after being wined and dined in Vegas, at least it was when I was last working in government..

To this day enterprises are looking for "SharePoint Developers" and everyone they hire hates every mouse click so they outsource further and further. "Azure Developers" are mostly developers from outsourcing centers forcefully trained by the enterprise. Training sponsored by workplace and conducted by Big Four. Here and there some whisper their wonderment why such expensive "cloud training" is needed. That's the IT workforce shortage the companies are talking about.

Thanks for that insight, I've not used it much, only minor things, so I didn't realize it was like that.

> Their UX and docs are extremely bad.

Their UX makes me feel stupid. I just can't find things and I keep wondering what I'm missing. A bit of a relief to hear this.

> The main reason to go to Azure is because your bosses' boss was convinced to buy it at a golf course.

I have said this so many times about all this horrible garbage our mgmt picks.

Less "golf course" and more "discounts on discounts on discounts"

I already follow that account. A reason is that we use Azure at work. A previous job used AWS, IDK what the next one will use, so it goes.

I don't see Azure as uniquely bad, but since I have to use it, I want to know about the particular ways in which it is bad. So I follow that account. By all means, do an AWS one.

Certainly. There's the cloud providers everybody hates and the cloud providers nobody uses.

and then there's AWS.

> cloud providers nobody uses

They're sweet and perfectly fine most of the times. Most self-healing clusters with sophisticated permission models are perfectly replacable by one Debian/RedHat instance.

I'm so glad I use AWS instead. Oh no wait. 200 different shits.

The UX, latency and API design consistency were absolutely terrible when I was forced to use Azure

Inconsistent API design is the hallmark of growth by acquisition

> Inconsistent API design is the hallmark of growth by acquisition

I don't doubt the rule but in MS's case it's commonly about constant warring between the interdepartment fiefdoms.

That seems reasonable, but I think you can trace those fiefdoms back to acquisitions. From what I understand even the NT kernel team was originally the result of Microsoft scooping up all the engineers from DEC

Their APIs are not very reliable and getting proper support takes ages even though you have the correlation ID in hand and the employer spends millions of dollars every month on Azure cloud resources.

The Azure portal is terrible. If I need to check something across our 5000+ subscriptions looking for a resource is an absolute chore. The way opening things expands and scrolls horizontally pisses me off.

Functions are the worst thing I always had to deal with. It's weird... it's some kind of .net runtime running somewhere. Most if not ALL Function documentation revolves around clickety click bullshit with VSCode. Say you want to provision with terraform you need to pack a zip file with the exact file structure explained from instructions buried among 300 screenshots of VSCode guides. Its also a pain in the ass to troubleshoot.

Security is a paid feature. They consider logs and critical security features extra addons. Granted other providers to some extent do the same but it is ridiculous with Azure. But the UI is much more pleasant to use than amazon or gcp which hardly makes up for their other failings.

There is this terrible IT thing where companies want to use one vendor for as many things as possible. MS really shines there by tying in M365, Azure and AD/Entra Id together. Outside of networking, that's like 90% of IT. And there are a ton of people that made their careers out of MS expertise. This is why Azure is king with big corporations.

Concur with #200. My number 1 issue with all of Microsoft's cloud offerings, especially Power Platform, is how difficult they make it for developers to get things done in large organizations.

Every tutorial and piece of documentation seems to assume you are an administrator in your tenant, and if you aren't, good luck figuring out who administers the particular combination of functionality you're fighting today.

Note that the problem here isn't the organisation's controls, it's the complete lack of discoverability of process, and integration into those processes, that is exhausting. Even a hyperlink into a templated SharePoint list with admin distribution lists as required info to activate a policy would be a game changer.

(Could the org maintain a central repo of this information? In theory. In practice it will grow stale as people/features come and go. Azure knows who administers what, and who flipped what policies on, and can tie process back to those policies correctly.)

Meanwhile it seems everything in Microsoft 365 defaults to "anyone internal can access/use the resource." As a result, I've seen large organizations leaking some of their most sensitive information to regular users.

My favourite Azureisms:

Begging and pleading for spot quota, being repeatedly told it's unavailable because the region is full, while the spot price is at the minimum and has been for months

Having quota silently removed because we weren't using it for two weeks, then having to resume begging for months to get it back.

Flat out refusal to let me pay by card on a new subscription - bank transfer only - on the same account as other subscriptions paying 10k/mo bills by card.

To use the Mastodon web application, please enable JavaScript. Alternatively, try one of the native apps for Mastodon for your platform.

Yeah, I wanted to follow this account (always interesting to keep track of feedback, even overwhelmingly negative) and had to tell Safari to “disable content blockers” to log in temporarily to do that.

That's funny, I used to keep a similar blog on my experience on Microsoft's Cloud. On Thumblr instead of Mastodon. Precously submitted to HN: https://news.ycombinator.com/item?id=12788098

I am still miffed that they renamed Azure AD to Entra ID. Definitely not a needless and confusing change.

The reason for this I heard by Microsoft was to stop having to explain to non technical people that Azure AD doesn't have feature parity with a real Active Directory.

As a Portuguese native Microsoft FTE, I found the name change jarring. “Entra” means “Enter”, with a subtext of permissiveness that seems overly positive for an authentication mechanism. But then again, Continental Portuguese is seldom considered as a semantic idiom, and Brazilian Portuguese (or even Spanish) tend to have a lot more influence in branding everywhere on the planet.

"Entra" means "Enter" in Spanish as well.

Did anyone catch the number of Continually Moving Controls to Unique Locations, Each One More Baffling Than The One Before?

Bicep might be a better option to do IAC on azure.

Bicep is just a set of macros for the JSON templating. I get why it was invented, I tried to use it about a dozen times, have always fallen back to either Terraform or raw Azure templates (which I actually prefer since they’re so, so easy to handle programmatically and work essentially like serialized API call data).

I find it easier than terraform since when using bicep you don't need to track state.

But then you have two problems

There is a lot to complain about on Azure.

But a lot of his posts are wrong (he just could not figure it out - a documentation problem likely) or complaining not about Azure, but Terraform.

If you tell me which ones are wrong I'd be happy to look into them again. :)

We can't guarantee that all of the posts are 100% correct, although we try to confirm all of them before posting. Most of these are things we encountered during our daily work, and yes, many of them are possibly just documentation errors. But when you can't figure out how to do something in a reasonable amount of time... that's not great either.

We use Terraform as our primary Azure client, but I'd argue most of the problems with Terraform are because of fuck-ups in the underlying Azure API or Azure resource providers.

To use an example: The AppGW TLS cert/KeyVault issue.

If it works manually, but not via Terraform, it's the fault of Terraform (or possibly the underlying Go SDK, small disclaimer). The ARM API is used by the Azure portal, as well as all the SDK's.

Of course it's totally possible it doesn't work manually. It did last time I checked, and it's a pretty foundational functionality - so I'd be surprised if it didn't.

> If it works manually, but not via Terraform, it's the fault of Terraform (or possibly the underlying Go SDK, small disclaimer). The ARM API is used by the Azure portal, as well as all the SDK's.

that's not true. sure the portal uses the same APIs, mostly. it also uses its own hidden APIs for some functionality. so it is entirely possible for it to work via the portal, but not work via any API mechanism.

e.g. uploading PKCS12 certs to (IIRC) automation accounts. the UI requires a password, and also sends the cert to some back end middleware to change the cert itself. while via the REST API you can upload the cert without a password, and it remains unmodified. (without the modification process some valid PKCS12 certs will not show up on the runners, and therefore can't be used for auth.)

Note: Azure support was not helpful in resolving this matter, since if it worked via PowerShell (the cert generation and upload), then it was considered an issue with our code (ignoring the fact that our code (and libraries used) remained unchanged throughout). though changing API behavior even when specifying an API version in the request is typical for Azure.

The hidden API's are a corner case.

They are not generally used for deployment. My point is valid.

I'm not quite sure what you mean by "not used for deployment" when they are for some cases, like the example I gave. trying to "deploy" an automation runbook that needs authentication. it'd be part of a greater solution, sure, but it'll still be part of the deployment

You're wrong about that anyways.

It appears to be a documentation miss in the REST API doc.

https://learn.microsoft.com/en-us/powershell/module/az.autom...

The AZ module is using the REST API QED - it supports the password.

In this case probably what you need to do to sniff out how it is working with the REST API is to run it in powershell with -verbose -debug and see what it is doing (or configure powershell for the fiddler proxy).

My point - just to remind you (I can almost hear you typing) - is that it's more likely to be the fault of Terraform. Undocumented Portal API's from what I have personally seen are all "read" not "write".

Yes, the docs suck.

I think you misunderstand. the problem isn't to do with the password necessarily. I want to not have a password, which is fine. however the cert I generate and upload does not work (rather, started to stop working). if the same cert is given a password (as the portal requires it), then, uploaded via the portal, the portal changes the certificate via a hidden API (I can see it via dev tools). that cert then works fine.

that's the issue.

I believe I also tried uploading the cert I generated via PowerShell (both with and without a password), and the same problem was there. It was narrowed down to 1. certs generated by PowerShell are fine. 2. The portal uses that hidden API to transform the cert before uploading it. which neither PowerShell or the REST API do.

It might be possible to do through the API, but there is no documentation available on what you need to do. There are only PowerShell examples, which you could look into and try to replicate on your own using the API.

The shit part about this: - Can't do it through the Azure Portal - Can't do it through Terraform - Docs only give you PowerShell examples - Docs don't explain you WHY you need to do this, they just expect you to suck it up - Why is Application Gateway only supporting user-managed identities and not system-assigned identities for this feature?

Sorry, I don't really care about which part of the Azure ecosystem is at fault here. It's a shitty experience.

A documentation problem is an Azure problem. A problem with support is an Azure problem. It's all part of a package.

About the terraform part, I am not sure about how it works for Azure, but GCP providers are developed /maintained by Googlers. Assuming it's the same, terraform problems are actually Azure problems (IMHO).

Disclaimer: Googler but working nowhere close to Google's Terraform efforts.

I don't believe that's the case here. The Azure RP AFAIK is maintained by Terraform folks. Possibly there's a few MSFT contributors.

Bad documentation as a root cause is still a bad experience running on the platform, so I'm not sure how that would make the posts wrong.

I can concur. At an old job we had a slack channel filled with terraform issues like these. We were an AWS shop.

Terraform tends to be one of those things that can quickly become a nightmare if you take up it’s offer to manage everything in your infrastructure.

I've been feeling this lately. A new guy came in and Terraformed the crap out of a very consistent and reliable AWS infrastructure.

I don't know whether to blame the tool or the implementation, but the injection of unpredictability is extremely unwelcome! We're only using it on the non-production account so far, fortunately.

I prefer boring systems. The more boring, the better.

What do you use instead, CF?

After having used Azure for a year, I believe every single post on this thread.