Azure Fundamentals Part 2 Summary

This is a summary of Azure Fundamentals part 2: Describe core Azure services. This learning path consists of five modules. The first one is the “Introduction to Azure fundamentals”, which is repeated in every Azure Fundamentals learning path — we’ve already covered this in Azure Fundamentals Part 1. The remaining four modules cover the core services: compute, storage (databases and unstructured data), and networking.

Database and Analytics Services

This section is a summary of the Explore Azure database and analytics services module.

Managed Database Services

Azure Cosmos DB:

  • is a globally distributed, multi-model database service
  • can scale throughput and storage across any number of Azure regions
  • provides fast, single-digit millisecond data access (latency)
  • stores schemaless data in atom-record-sequence (ARS) format
  • can be accessed using various different APIs including SQL, MongoDB, Cassandra, Tables, or Gremlin

Azure SQL Database:

  • is a relational database based on the latest stable version of MS SQL Server
  • is fully managed (PaaS) so things like upgrading patching, backups and monitoring are all taken care of
  • offers 99.99% availability

Azure SQL Managed Instance is also a managed SQL Server-based offering with 99.99% availability. However, this has better compatibility (i.e. feature coverage) with the SQL Server engine than Azure SQL Database (the example given is that Azure SQL Database supports only one default collation, which makes it a little hard to support foreign characters outside of the Latin set).

The Azure Database Migration Service provides guided migration from existing on-premises databases (e.g. SQL Server, MySQL, etc) to Azure-managed databases (e.g. Azure SQL Database). It works as follows:

  1. You first get an assessment report to identify any hurdles with migration.
  2. After any issues are resolved, the Database Migration Service carries out the migration for you.
  3. You just have to change the connection string in your applications.

Azure Database for MySQL:

  • is a managed version of MySQL based on MySQL Community Edition 5.6, 5.7 and 8.0
  • offers 99.99% availability
  • supports point-in-time restore as far back as 35 days
  • can scale as needed
  • can protect data in transit and at rest

Azure Database for PostgreSQL, similarly, is a managed PostgreSQL instance that offers high availability, can scale, and supports point-in-time restore up to 35 days. What’s interesting here is the deployment options:

  • Single server: 99.99% availability, can scale vertically
  • Hyperscale (Citus): horizontally scales queries across multiple machines using sharding; good for workloads of around 100GB or more

Big Data and Analytics

Azure Synapse Analytics is an enterprise data warehouse and big data analytics service. You can query data using either serverless or provisioned resources.

Azure HDInsight is a managed analytics service for enterprise based on open source software. It can process massive amounts of data and is suitable for things like ETL, data warehousing, machine learning and IoT. It supports different cluster types such as:

  • Apache Spark
  • Apache Hadoop
  • Apache Kafka
  • Apache HBase
  • Apache Storm
  • Machine Learning Services (R-based analytics)

Azure Databricks:

  • Analytics and AI based on Apache Spark
  • Supports languages: Python, Scala, R, Java, SQL
  • Supports libraries: TensorFlow, PyTorch, scikit-learn

Azure Data Lake Analytics is an on-demand analytics job service.

  • You set the dial for how much power you need
  • You pay for jobs only when they’re running
  • No hardware deployment

Compute Services

This section is a summary of the Explore Azure compute services module.

Virtual Machines

Azure virtual machines (VMs):

  • Include virtual processor, memory, storage and networking (IaaS)
  • Provide full control over the OS and applications, but you need to maintain them
  • Useful for lift-and-shift cloud migration

Azure virtual machine scale sets are a group of identical, load-balanced VMs that autoscale (increase/decrease in number) in response to demand or a defined schedule.

Azure Batch runs large-scale parallel and high-performance computing (HPC) batch jobs across a pool of VMs. It also identifies failures and requeues work.

Containers

Virtual machines provide an abstraction for CPU, memory and storage, making these easy to change. However:

  • You install the OS and applications (more maintenance effort)
  • They support only one OS at a time, which is not that great where different runtime environments are required
  • Starting up or taking snapshots can be slow

Containers provide a lighter-weight abstraction. Whereas VMs virtualise the hardware, containers virtualise the OS.

  • A container bundles a single app and its dependencies.
  • You deploy a containerised app to a container host.
  • The container host provides a standardised runtime environment which abstracts away the OS and infrastructure requirements.
  • The same container works in different environments (e.g. dev and prod).
  • Cluster orchestration can handle the problem of where to deploy containers, and take care of starting them, stopping them, and scaling them out.

Azure supports Docker, and provides the following services to manage containers:

  • Azure Container Instances: a PaaS service making it fast and easy to deploy containers without having to manage any VMs.
  • Azure Kubernetes Service: complete orchestration service suitable for distributed architectures with lots of containers

Hosted Apps

Azure App Service provides a managed environment for hosting different kinds of apps.

  • You pay depending on how much hardware is devoted to your host
  • Covers deployment, management, securing endpoints, scaling, and high availability (load balancing and traffic manager)

The following types of apps are supported:

  • Web apps: ASP .NET, ASP .NET Core, Java, Ruby, Node.js, PHP, Python, running on Windows or Linux.
  • API apps: REST APIs with HTTP or HTTPS endpoints, Swagger support, and the ability to publish to Azure Marketplace.
  • WebJobs: schedule/trigger a program (.exe, Java, PHP, Python, Node.js) or script (.cmd, .bat, PowerShell, Bash) in the same context as a web application – good for background tasks.
  • Mobile apps: backend for a mobile app, providing services such as databases, authentication via social logins, push notifications, or custom backend logic (C# or Node.js).

Serverless Functions

Azure provides Azure Functions and Azure Logic Apps, both serverless (i.e. they run on servers but they are hidden away) and triggering based on an event (e.g. timers, HTTP requests, queues, etc).

Azure Functions have the following features:

  • Automatic scaling
  • Micro-billing (you pay only for the time the code is running)
  • Stateless (restarted each time) or stateful (Durable Functions – context is passed to the function on execution)
  • Can run locally or in the cloud

Azure Logic Apps:

  • Execute workflows built on predefined logic blocks to automate business scenarios
  • Are created using a visual designer in the Azure Portal, or Visual Studio. They are stored as JSON using a defined schema.
  • Use connectors (over 200 in-built, or you can write your own) to interact with enterprise apps
  • Run only in the cloud (not locally)

Windows Virtual Desktop

Instead of shipping laptops to remote employees, IT administrators can use Windows Virtual Desktop to provide them with a cloud-based (virtualised) version of Windows.

  • They can remote in from Windows, Mac, iOS, Android or Linux
  • They can also access it directly from most modern browsers
  • It separates the compute environment from user devices, making it less likely for employees to leave confidential data on personal devices
  • Windows 10 Enterprise Multi-Session allows more than 2 users on the same VM
  • Uses reverse connect technology, so it does not open any ports for RDP

Azure Storage Services

This section is a summary of the Explore Azure Storage services module.

You can use several different data storage services after first creating an Azure Storage account.

  • The storage account will contain your data objects (e.g. blobs).
  • It also serves as a unique namespace for your data.
  • Data is secure, highly available, durable, massively scalable, and accessible over HTTP or HTTPS.

Azure Disk Storage is an IaaS service providing virtual disks for Azure VMs. Disks come in different sizes and performance levels (e.g. HDDs vs SSDs).

Azure Blob Storage:

  • Object (think “file”) storage solution for the cloud
  • Can store massive amounts of data
  • Data is unstructured, so you can put any type of data (e.g. videos, backups, etc)
  • Storage Account contains Containers (think “folders”), which in turn contain Blobs (think “files”)

Understanding Blob access tiers:

  • Hot access tier is for frequently accessed data.
  • Cool access tier is for infrequently accessed data stored for at least 30 days. Slightly lower availability and higher access costs are a tradeoff for lower storage costs.
  • Archive access tier is for rarely accessed data stored for at least 180 days, with flexible latency requirements. Data is stored offline, and this tier carries the highest costs to rehydrate and access data.
  • All access tiers can be set at the blob level, whereas only the hot or cool tier can be set at the account level.

Azure Files:

  • Fully managed file shares in the cloud
  • Accessible via Server Message Block and Network File System (preview) protocols
  • Can be mounted concurrently by cloud or on-premises deployments of Windows, Linux and macOS
  • Files can be accessed from anywhere using limited-time URLs using Shared Access Signature (SAS)

Networking

This section is a summary of the Explore Azure networking services module.

Virtual Networks

Azure virtual networks are an IaaS service providing cloud-based equivalents of networks in a data centre. Features include:

  • Isolation and segmentation – private IP address space, subnets
  • Internet communications – via public IP or load balancer
  • Communicate between Azure resources – virtual networks and service endpoints (which link Azure services to virtual networks)
  • Communicate with on-premises resources
    • Point-to-site VPN: computer connects to Azure virtual network
    • Site-to-site VPN: connect on-premises network to Azure network via VPN gateway
    • Azure ExpressRoute: dedicated private connectivity to Azure
  • Route network traffic – route tables, Border Gateway Protocol (BGP)
  • Filter network traffic – network security groups, network virtual appliances
  • Connect virtual networks – virtual network peering (connect networks even across regions)

VPN

VPN = virtual private network, a secure connection between 2 or more trusted private networks over an untrusted network (e.g. the internet).

Azure VPN Gateway:

  • Connects on-premises data centres to Azure virtual networks via site-to-site connection
  • Connects devices to virtual networks via point-to-site connection
  • Connects virtual networks to other virtual networks via network-to-network connection
  • You can deploy only one VPN gateway in each virtual network, but it can connect to multiple locations
  • Policy-based or Route-based: both use pre-shared key and rely on Internet Key Exchange (IKE) on v1 or v2 of IPSec

Policy-based VPN:

  • Supports IKE v1 only
  • Uses static routing: IP address configuration controls how traffic is encrypted/decrypted
  • Use this only where necessary (compatibility with legacy devices)

Route-based VPNs:

  • IPSec tunnels are modelled as a network interface
  • Support static or dynamic routing (using routing tables, BGP)
  • Are resilient to topology changes (e.g. creation of new subnets)
  • Support IKE v2
  • Use any-to-any (wildcard) traffic selectors

To deploy a VPN gateway you need the following in Azure:

  • A virtual network (address space must not overlap with the on-premises network)
  • A dedicated subnet for the VPN gateway
  • A public IP address (dynamic, but it won’t change until you delete the VPN gateway)
  • A local network gateway (represents the on-premises network from Azure’s point of view)
  • A virtual network gateway
  • One or more connection objects, connecting the on-premises VPN device’s IP address to the virtual network gateway’s IP address

To deploy a VPN gateway, you also need to have the following on-premises:

  • A VPN device supporting policy-based or route-based VPN gateways
  • A public-facing IPv4 address

VPN high availability scenarios:

  • Active/standby: 2 instances, brief downtime during maintenance
  • Active/active: additional tunnels and devices are used as fallback
  • ExpressRoute failover: use VPN gateways as fallback if ExpressRoute fails
  • Zone-redundant gateways: deploy VPN gateways or ExpressRoute across availability zones. Needs different gateway instances and Standard (not Basic) IP addresses.

ExpressRoute

Azure ExpressRoute:

  • On-premises data centre connects to Azure via a private connection (i.e. not over the internet) via a connectivity provider
  • It provides better speed, reliability, security, and more consistent latencies compared to going over the internet
  • It covers two layers of the OSI Model:
    • Layer 2: Data Link Layer (node-to-node communication on the same network)
    • Layer 3: Network Layer (addressing and routing on a multi-node network)
  • Provides connectivity to MS cloud services (e.g. Office 365, Dynamics 365 and Azure services)
  • ExpressRoute Global Reach: connect private data centres together via ExpressRoute
  • Uses BGP, dynamic routing

ExpressRoute connectivity models:

  • Colocation at cloud exchange: provider (e.g. ISP) provides Layer 2 and Layer 3 connections between your infrastructure and the MS cloud
  • Point-to-point Ethernet connection
  • Any-to-any networks: Layer 3 connection between your WAN and Azure

Technology and Anxiety

We’re about a year into the COVID19 pandemic by now, and topics like stress and anxiety have gone from taboo to mainstream. They have frequently appeared in the news and social media over the past several months.

While COVID19 has been a horrifying experience in many ways, I think many reasons for the anxiety we experience every day have been around before the pandemic. COVID19 has just exacerbated certain issues by changing the rules, forcing isolation upon us and making us more dependent on technology to communicate; but at the same time making remote work a reality for many people, saving commute time, and reducing environmental impact.

If you work in the IT industry, you are probably no stranger to stress, anxiety and burnout. In fact, I have recently seen a bigger trend of developers burning out, taking a step back from social media, and trying to live in a more disconnected manner.

I am, of course, no exception to all this myself; and reading about other people’s experiences has inspired me to write my own thoughts on the subject.

Disconnecting

When we went from primitive modems to ADSL, we heralded an era in which internet connectivity was “always on”. Fast forward to today, when we have mobile data, and we receive notifications any day, any time, whether we’re awake or asleep, from work or from friends, with no room to breathe at all. Is it really any wonder that we sometimes feel overwhelmed?

Mobile Devices and Social Media

The Netflix film “The Social Dilemma” is an excellent eye-opener showing how social media platforms are engineered to capture our attention, addict us, and sell us as a product. Yet, what’s in it for us? Giving everyone a voice is hardly an advantage when you’re scrolling through a deluge of politics, marketing, and petty squabbles.

The best thing you can do for your own sanity is get off social media:

  • Remove Facebook etc from your phone.
  • For anything you can’t remove, turn off notifications.
  • If you need to access social media, login from a computer in an Incognito session, do what you have to do, and then logout. (Of course it’s more tedious than using your phone. That’s the point.)
  • Check social media only at specific (few) times of day.
  • Use Do not disturb when you’re asleep, taking a break, or otherwise don’t want to be interrupted by your phone.
  • Stop getting hammered by recruiters. Pursue them yourself if/when you need them.
  • Avoid anything with marketing content. Install an ad blocker in your browser, and watch films on a medium that doesn’t require you to watch advertisements half the time.

Do the same with work-related apps.

  • Don’t install Slack etc on your phone if you can avoid it.
  • If you can’t, turn off notifications outside working hours.
  • If possible, turn off notifications in general. If you’re usually at a computer in front of Slack anyway, there’s no reason why your phone should be buzzing about things you’re already seeing. Keep the app handy for when you really need it (e.g. you’re on the train and need to tell your boss you’ll be late).
  • Tell your colleagues to give you a call if they need you for an emergency while you’re out of the office. That way you don’t need to constantly check your phone or rely on notifications.

Context switching all the time is a sure way to burn you out. It’s already hard enough having to deal with email, chat and other notifications when you’re on a computer.

Infinite Scroll

Part of that social media addiction is a feature where you can scroll infinitely and keep getting more content to consume. Aside from being a bad thing in the stress/anxiety space, I would also argue that it is terrible from a user experience perspective.

  • Content eventually gets lost in an endless stream of random stuff.
  • You have no idea where you are because you can’t rely on pagination or a scrollbar.
  • Things reload automatically, sometimes while you’re trying to read something.
  • Once it reloads, the order is often messed up, and it becomes impossible to locate something you were interested in.

The end result is that you’re addicted to a very shitty way of reading stuff you don’t even care about.

Social Obligations

COVID19 has forced us to rethink the way we socialise. Any contact with people outside our regular household could bring the virus into that household, so is it worth the risk? It depends. People need to eat, I get it. There are also people who don’t care. But people scrambling to buy and deliver Christmas presents in the midst of a pandemic? Seriously… priorities.

Again, I think the lesson here extends beyond COVID19. Before COVID19, was it really uncommon to have to attend so many events (several of which we’d rather have avoided) and do so many things that it would result in very tangible fatigue?

Here, I’ll borrow a tip from Tania Rascia’s 2020 into 2021 roundup (which is excellent related reading, by the way) and suggest you choose your battles, or rather, social obligations by saying “HELL YEAH!” or “no”. Originally a suggestion by Derek Sivers, this means it’s worth investing your time and soul into things you’re really enthusiastic about. Drop it if you have doubts.

Hobbies

Being addicted to social media (or other stuff) means you’ll likely go back to it by default when you’re bored. So how do you stop needing it? Gradually, and by replacing it with things that are actually fulfilling.

Ideally, find hobbies where you don’t need a computer (e.g. reading a physical book, some kind of sport, playing a musical instrument, knitting, hiking, etc). Some computer-related hobbies are also okay (e.g. video games, anything artistic). You’re looking for things where you don’t need to use the mind too much (you do enough of that in your day job), yet which are either physically or creatively stimulating.

How can something be creatively stimulating and yet not be a burden on the mind? You know how science and art are often thought to be two extremes of a distant spectrum? I think it couldn’t be farther from the truth. Take music, for instance. The science part of it gives it structure, yet on its own it would be uninteresting. As an art without the science, it would be random and uncompelling, like what you get if you put a monkey (or me) in front of a piano, and leave it free to bang around.

Now, I believe programming jobs are a combination of science (the tools you use to solve a problem) and art (the way you arrange those tools to solve the problem in a specific way). The problem is that after your first handful of “A-ha!” moments, it becomes more of the same, and the creative passion goes away. That’s why, outside of work, it’s good to pursue interests that are of a purely creative nature. Science makes us better at life, but it is art/creativity that makes it truly enjoyable.

Hobbies are also best when you use as many of the senses as possible. Sure, you could experience a distant land on a computer, seeing the panorama, listening to the sounds, and perhaps running your finger across a touch-screen. Seriously? Go on a hike. Let your eyes focus on distant objects. Listen to sounds that are natural and not electronic. Feel the breeze on your skin. Listen to the waves. Taste some of the local produce. There, all five senses. Beautiful. You see, computers let you experience a lot of things you never could, but in a limited way. When you build something with your hands, it’s a completely different feeling from building software.

Productivity

If you really like what you do, you probably take an interest even outside of work. Constantly reading, learning, experimenting, achieving, and sharing what you learn with others. It’s something we’re encouraged to do in our industry, and for good reason – many of the things we learn and the solutions we find are thanks to selfless souls who took the time to share their wisdom for no personal gain.

There is, however, one problem: resources are limited. More specifically, our time is limited. When you have to spend a good part of your day at work, look after your family, take care of your home, shop for groceries (not to mention other necessities), get enough sleep, do some exercise, have a hobby or two, and do more work-related stuff in your free time… well then you might be juggling a bit too much… ergo stress, burnout, anxiety.

I’m not saying you shouldn’t have a blog or side projects, or take an interest in programming outside of your job. But to achieve a good balance, you need to prioritise, and do it in moderation.

Burning through the Backlog

People who sell Agile for a living love to extend Agile practices beyond the realm of software development. For instance, in his talk “Agile Everywhere!“, Henrik Kniberg talks about he optimised the way his family wash the dishes. While this talk is entertaining and probably also insightful, it can be pretty tedious to live life as a sequence of tasks in a neverending backlog.

In the software industry (and in everyday life), I sometimes feel that we’re just constantly trying to finish off as many things as possible. There are two main reasons for that. First, although I (and Peopleware) love to think otherwise, the sad truth is that in most companies, software development has been reduced to a factory, a conveyor belt of tasks. “But,” you might argue, “software is knowledge work! It’s a creative endeavour! Surely good quality software can’t be written in such conditions!” Well, just take a look at the state of websites, web applications, and software in general today.

Secondly, oftentimes, the pressure is actually self-imposed, especially for high achievers. In a competitive industry where there are no real standards for measuring competence (unlike, say, the engineering or accounting professions) or what job titles even mean, it’s really not clear what one must do to stand out from the crowd, and sometimes people will invest huge amounts of effort even in their own time to try to get somewhere.

The problem with this is that, like work itself, learning and keeping up to date is a process that never ends. Like I wrote in 2014, don’t bite more than you can chew. Prioritise your personal goals, keeping them to a minimum and selecting only those which feel really worth it. Spread them out over longer periods of time rather than trying to burn through something you’re learning as fast as possible.

The “HELL YEAH!” or “no.” advice from earlier applies here as well, since your own excitement is not a bad way to decide how to spend your own time. However, you can also take a hint from Lean practices and try to get some feedback before embarking on a complex side project or writing a book that perhaps nobody cares about. The only time it’s worth doing something that nobody cares about, or reinventing the wheel, is when you’re doing it as a learning exercise; and in that case, you still need to treat it as a proof of concept and scope the problem to avoid wasting huge amounts of time on it.

Content

Those of us who are active in the community tend to feel pressure to constantly produce content. Whether that means answering questions on Stack Overflow, writing blog articles regularly, or speaking at conference, it often feels as if we’re disappointing people if we don’t give them something to reap.

This, of course, is just one of those self-imposed goals, and a pretty harmful one, for two main reasons. First, there is very little return in it. I’m not saying that sharing knowledge doesn’t have a lot of benefits (helping others, learning by teaching, and increasing your visibility to name a few), but when you fast forward a few years and several hundred articles later, and you look back, you can’t help wondering whether you should have spent that time in better ways… with family for instance?

Because in actual fact, unless you’re Jon Skeet, or you’ve contributed something pretty awesome to a major open source project (which becomes more and more difficult to do as software becomes more complex), chances are that people who can make a difference in your career aren’t going to notice your contributions. And even if they do, in an industry dominated by politics and Codility-style tests, your ability isn’t necessarily going to be an influential factor in your career development.

The second problem with forcing yourself to produce content regularly is that… you end up doing exactly that. Making content for the sake of making content. And that’s why the internet is full of people writing the same stuff over and over again, or bullshit “N Things” articles. Like I wrote in “The Sixth Anniversary“, it’s far better to slow down and write something only when you have something really interesting to share.

Write Things Down

It’s pretty common to have an idea, or remember something you need to do, and to be too busy to pursue it right away (or anytime soon). Multiply that by one or two dozen, and you’re suddenly anxious because you don’t want to forget all these things you can’t do right now.

Write them down. Chances are that most of them can wait, and by writing them down, your struggling mind doesn’t have to juggle all these things at the same time. Even better, schedule them at different points in your calendar to figure out which you actually need to do first, and which can wait. Once again, spread them out, and avoid overloading your calendar at any given time. Try to set a maximum number of items you think you can tackle on a daily and weekly basis.

Coping with Work

A couple of years ago, when I used to work in Ireland, a high-level executive visited our office one time and talked about his career. I asked him how he coped with stress, and he answered, “How many things that bother you today will you still be thinking about in 5 years’ time?” Sure enough, that makes you realise that although some things may worry us, sometimes we make a bigger deal than we need to.

What’s the Worst That Could Happen?

Think about it. COVID19, Brexit, US elections. Missed deadlines, abuse of power and public funds, economic and political instability dragging on for months and years, and millions of people sick and dead thanks to people who messed up (many of whom are top political leaders), and no one’s accountable.

The point here is that although you should always strive to do a proper job, many decisions carry very little risk, and most mistakes are easily corrected. Conversely, in software development, we’re often known to waste a lot of time arguing about little details, a phenomenon known as bike shedding. Are you building systems that people’s lives depend on (such as medical, aircraft or space-related software)? In that case, by all means, analyse every little detail and lose sleep if it makes the result that much more reliable. Otherwise, making a mistake or looking stupid probably isn’t a big deal. Go ahead, take a risk. You’ll learn from it in any case.

Another aspect to this is when you need to worry. In my experience, I’ve found that it’s quite common to worry about something that might happen in future, only to come back later and find that it won’t happen at all. Well, there’s no point in prematurely worrying about some problem that might occur, just like there’s no point in building features just in case (You Ain’t Gonna Need It (YAGNI)).

Commuting

One of the biggest lessons we learned from COVID19 is that we were right about remote work. It works. Not just that, but there are other benefits.

Let’s just focus on commuting. Avoiding the commute saves time, money, traffic, environmental impact, and stress. Just physically getting to work on a day-to-day basis, along with millions of others on the same quest, is bound to make you anxious as you deal with things like overcrowded, late or faulty public transport, crazy drivers, and road closures. It’s not uncommon for people to be already stressed out by the time they arrive at the office, when their day hasn’t even started yet.

Add to that the fact that commuting can eat up anything from 20 minutes to 4 hours (even as much as 6, in extreme cases) a day, and you have basically no time left for yourself.

There is lots to be said about remote work and commuting, but the takeaway here is that you really should consider working remotely if it’s an option. Simply avoiding the commute works wonders for one’s inner peace.

Conclusion

Basically, if you want to worry less:

  • Aim to do less. Choose a few goals that are really important, and spread them out so that they’re achievable and leave you time to enjoy other things in life.
  • Block out anything that you don’t really need and that makes you miserable. Social media, politics, news, even friends or family who bring out the worst in you. You don’t owe anything to anyone.
  • Get some creative and stimulating hobbies. Build something with your hands. Use your senses. Get some exercise. Stop being a zombie on a phone, especially when you’re with other people.
  • It’s okay if you’re not productive all the time. On the contrary, you might find things work out better if you take a step back and have time to think about what you’re doing.
  • Focus on quality rather than quantity.
  • Write things down to get them off your mind. Most things can wait.
  • Are you doing something high-risk? Then worry, spend time asking for advice, and take great care. Otherwise, get on with it and don’t look back.
  • Work remotely if you can.

Azure Fundamentals Part 1 Summary

This is a summary of Azure Fundamentals part 1: Describe core Azure concepts. This learning path is made up of the following modules:

This is actually a bit of a mess. The first module is repeated in every Azure Fundamentals learning path. The second module is actually a repetition of the first, and the third covers part of the first module in more detail.

My goal here is to gather the most important points to serve as a decent summary for anyone preparing to take the Azure Fundamentals exam.

Cloud Computing

Cloud computing is the delivery of computing services over the internet.

  • You typically pay for what you use
  • Someone else manages certain resources for you (e.g. underlying physical hardware)
  • Compute power and storage are the main cloud resources
  • You can add/remove resources as needed

Basically, you’re renting computing and storage resources from someone else’s datacentre. Thus you don’t need to concern yourself with things like building security or cooling. The pay-as-you-go model is handy because you can provision (and de-provision) resources quickly and as needed:

  • No upfront costs
  • No need to buy and manage idle resources
  • Pay for additional resources when needed
  • Stop paying for resources when they are no longer needed

On a financial level, cloud usage shifts IT expense from CapEx to OpEx:

  • Capital expenditure (CapEx): up-front expenditure on infrastructure, which incurs depreciation over time
  • Operating expenditure (OpEx): pay for what you use, just like electricity

Advantages of cloud computing include:

  • High availability
  • Scalability (vertical and horizontal)
  • Elasticity (autoscaling)
  • Agility
  • Geo-distribution
  • Disaster recovery

Cloud Service Models

Azure and similar cloud providers offer a large range of services. These services abstract underlying resources at different levels, and are generally categorised as follows:

  • Infrastructure as a Service (IaaS): Azure manages the hardware, but you manage the OS, networking, etc. This category offers most control/flexibility but you have to take care of more things yourself (e.g. Azure virtual machines).
  • Platform as a Service (PaaS): Azure manages the hosting environment (e.g. VMs, networking). You just deploy your application (e.g. Azure App Service).
  • Software as a Service (SaaS): Azure manages all aspects of the application environment, including the application itself. You just bring your data (e.g. Office 365).

Serverless computing sits in the PaaS category, and offers a way to execute code in an event-driven manner that scales automatically, without needing to manage infrastructure. Servers are hidden (e.g. Azure Functions).

Public, Private and Hybrid Cloud

  • Public cloud: Azure sells services over the internet to anyone. Technically the physical server resources are shared.
  • Private cloud: Resources are dedicated to one organisation. This can be on-premises or hosted by the cloud service provider (i.e. Azure).
  • Hybrid cloud: Uses both public and private cloud, sharing some resources between them.

How Azure Works

  • Azure uses virtualisation
    • A hypervisor sits between hardware and OS
    • This allows a single physical server to run several VMs, at massive scale
  • Azure has datacentres all over the world
    • Each datacentre has many racks filled with servers
    • Each server includes a hypervisor to run multiple VMs
    • Servers are connected by network switches
  • One server in each rack includes a fabric controller
    • Fabric controller receives instructions from an orchestrator
    • Orchestrator manages everything that happens in Azure, including responding to user requests
  • API requests (e.g. to deploy a VM, from the Azure Portal) will go to an orchestrator, which talks to a fabric controller, which provisions/deprovisions resources as needed.

The above is explained graphically in a video at the What is Azure? page.

Miscellaneous

The Azure Portal is a web interface to view and manage your Azure subscription and resources. It has instances in every datacentre (so it’s close to users) and runs with high availability – updates incur no downtime.

The Azure Marketplace contains third party solutions to be run on Azure.

Azure Services

There is a long list of services on Azure, split up into a number of categories – you can see them at the Tour of Azure services. The more important of these are covered in subsequent learning paths, so let’s just take a high-level look at the categories at this stage:

  • Compute: VMs, container services, serverless functions, etc
  • Networking: virtual networking, load balancing, VPN, and other networking and security services
  • Storage: unstructured storage including blob, file (file server), queue and table (schemaless NoSQL) storage. These are all durable and highly available, secure, scalable, managed, and accessible via HTTP or HTTPS.
  • Mobile: backend services for mobile apps such as offline data sync, push notifications, connectivity to on-premises resources (e.g. SQL Server), and corporate sign-in.
  • Databases: several managed data store offerings ranging from managed RDBMSes (Azure SQL, MySQL, PostgreSQL, MariaDB), Redis, as well as CosmosDB (globally distributed NoSQL).
  • Web: App Service (managed hosting for web apps), Notification Hubs (push notifications), SignalR service, API management and more.
  • Internet of Things (IoT): connect, monitor and manage IoT devices.
  • Big Data: Synapse Analytics (managed enterprise data warehouse), HDInsight (Managed Hadoop clusters), Databricks (Apache Spark-based analytics service)
  • Artificial Intelligence (AI): Machine Learning Service (develop your own ML models) and Studio (use prebuilt ML algorithms). Cognitive Services are related, and comprise Vision, Speech, Knowledge Mapping, Bing Search, and Natural Language Processing.
  • DevOps: Azure DevOps (git repos, pipelines, testing, project management) and DevTest Labs (set up environments for testing).

Azure Subscriptions

An Azure account has one or more subscriptions.

  • Resources are anything you use within Azure (e.g. VMs)
  • Resource groups are a logical group of (related) resources
  • Subscriptions are a group of user accounts and resources; limits/quotas apply
  • Management groups are groups of subscriptions which inherit access, policy and compliance rules

Subscriptions can be used to separate things like environments (e.g. Dev and Prod), departments and billing. They can also be used to deal with limits at a subscription level by adding additional subscriptions. Subscriptions provide isolation in the form of a billing boundary (e.g. bill by department) and an access control boundary (e.g. you can only access Dev and Test environments).

A billing profile can be used to manage invoicing across subscriptions. A billing account can have multiple billing profiles (each of which is an invoice), each of which can have multiple invoice sections, each of which can contain multiple Azure subscriptions.

On management groups:

  • Can be nested to form a hierarchy; rules are inherited by children (other management groups, subscriptions, resource groups and resources)
  • Can give users access to multiple subscriptions via Role-Based Access (RBAC)
  • Can have up to 10,000 management groups in a single directory
  • Can have up to 6 levels of depth (excluding root and leaves (subscriptions))
  • Each can have only one parent
  • Each can have many children

On resource groups:

  • All resources must be in a resource group
  • Resource groups can’t be nested
  • Each resource can only be in one resource group
  • Serve as logical grouping of resources
  • Deleting a resource group deletes all the resources in it
  • Act as scope for RBAC permissions

The Azure Resource Manager (ARM) is a management layer that can be thought of as providing CRUD around resources. ARM also allows resources to be managed by templates – this is covered in more detail in a later learning path.

  • Azure Portal, Azure Powershell & Azure CLI (via SDKs), and REST clients talk to the Azure API
  • Azure API talks to ARM
  • ARM verifies authentication & authorisation
  • ARM can then perform actions (e.g. provision) on Azure services

Geography

A region is basically a geographical area where Azure has its datacentres.

  • A region has one or more datacentres nearby, connected by a low-latency network
  • Most resources have to be deployed to a region (which you choose)
  • Some services are only available in certain regions
  • Some services are global and don’t need a region at all
  • Deploying resources across regions gives you scalability, redundancy, data residency (when data must legally reside within a country), and allows data to be close to users

Some regions are special and not available to the general public. This includes several isolated datacentres used by the US Government, and datacentres in China which are operated by a partner.

Availability zones (AZs) are physically separate datacentres in an Azure region.

  • One or more datacentres with independent power, cooling and networking
  • Isolation boundary: it would take a major disaster for more than one AZ to fail in the same region (see also Region pairs further below)
  • AZs in the same region are connected via high-speed fiber-optic networks
  • Not all regions support AZs

Azure services supporting availability zones:

  • Are mainly VMs, managed disks, load balancers and SQL databases
  • Zonal services: pin resource to specific zone (e.g. VMs)
  • Zone-redundant services: replicates across zones

Region pairs:

  • A region is paired (directly connected) with another region in the same geography, at least 300 miles away (where available)
  • The pair is far enough that disasters shouldn’t take out both regions
  • Updates are rolled out to one region in a pair at a time
  • In case of outages to both regions, one region in a pair is prioritised to restore service quickly to at least one region in the pair
  • Data continues to reside within the same geography, which can be important for legal/compliance reasons

Azure Fundamentals Part 6 Summary

This is a summary of Azure Fundamentals part 6: Describe Azure cost management and service level agreements, both for my own exam revision and to help others get a more concise digest of the otherwise long-winded material.

Plan and Manage Costs

This part is a summary of the Plan and manage your Azure costs module.

Total Cost of Ownership

Total Cost of Ownership (TCO) refers to the total cost of running a data centre, which may include hidden costs such as software licences, electricity, network maintenance, salaries, etc.

There’s a TCO Calculator where you can compare running workloads on-premises vs on Azure (estimated). Here you:

  1. Define your workloads, i.e. select servers, databases, storage and networking.
  2. Adjust assumptions: here you can reuse existing licences via Software Assurance, decide whether you want to replicate storage across regions, and adjust other costs such as electricity, salaries and network maintenance.
  3. View report. Here you can adjust the timeframe between 1-5 years, and see a breakdown per category (e.g. compute or storage).

Azure Subscriptions

The different types of subscriptions are:

  • Free trial: 12 months of popular free services, credit against any service for 30 days, and more than 25 services always free. Services are disabled when the trial ends or the credit expires.
  • Pay as you go: pay for what you use. You can apply for volume discounts and prepaid invoicing.
  • Member offers: other subscriptions (e.g. Visual Studio) can provide discounts against Azure usage.

Purchasing Azure Services

You can purchase Azure services via:

  • Enterprise Agreement: commit to spending a certain amount over 3 years to get custom pricing.
  • Web Direct: use the Azure Portal directly; billed monthly.
  • Cloud Solution Provider: via a reseller (Microsoft partner).

Factors Affecting Cost

  • Resource type: different resources have different billing criteria
  • Usage meters: resources have billable units, e.g. VMs have CPU time and other units)
  • Resource usage: you can deallocate a VM to stop incurring charges, but you could still be charged for the disk if you leave it around
  • Subscription types (e.g. free trial)
  • Azure Marketplace: billing structures set by third party
  • Location: different regions have different prices
  • Bandwidth: network traffic in/out of Azure data centres is billed according to which geographical zone they’re in

Use the Azure Pricing calculator to estimate the cost of running a number of Azure services.

Manage and Minimise Costs

  • Understand estimated costs: use the TCO and Pricing calculators, and consult the docs about usage and billing for the relevant services
  • Monitor usage with Azure Advisor which provides suggestions (e.g. unused resources)
  • Use spending limits to restrict spending and avoid accidental overrun (there are also subscription limits and quotas from Azure)
  • Azure Reservations: prepay and get up to 72% discount over Pay as you go prices
  • Choose low-cost regions and locations (but keep in mind that you pay for network traffic between regions)
  • Research available cost-saving offers
  • Use Azure Cost Management + Billing to control spending (breakdown of costs)
  • Apply tags to identify cost owners (i.e. organise billing data, e.g. by department)
  • Resize underutilised VMs (need to turn them off, so consider downtime)
  • Deallocate VMs during off hours (e.g. dev/test running only during business hours) – can be scheduled
  • Delete unused resources
  • Migrate from IaaS to PaaS services
  • Save on licencing costs
    • Choose cost-effective OS (i.e. Linux)
    • Use Azure Hybrid Benefit to repurpose Windows Server or SQL Server software licences on Azure via Software Assurance

Service Lifecycle and Availability

This part is a summary of the Choose the right Azure services by examining SLAs and service lifecycle module.

Service Level Agreements

Service Level Agreements (SLA) define guarantees about the service provided by Azure, usually around availability/uptime.

  • Access SLAs from Service Level Agreements portal.
  • Uptime/downtime is measured as a percentage, e.g. 99.9% (or three 9’s) means the service can be down 0.1% of the time, which on a weekly basis works out to 10.1 minutes per week.
  • Downtime measurements are cumulative.
  • If the SLA is breached, you can claim a service credit which is a percentage of the fees you paid (defined in the SLA).
  • There’s generally no SLA for free services.
  • Azure status tells you when there’s an outage. From there you can also reach Azure Service Health, a personalised status view within the Azure Portal.
  • You get a composite SLA by multiplying the SLA for each service.
  • SLA is affected by customisations on resources (e.g. tier, or disk type) as well as redundancy (e.g. VMs behind a load balancer).

Azure Service Lifecycle

New services go through the following stages:

  1. Development
  2. Public preview
  3. General availability

Things in preview include:

  • Preview (new) services
  • Preview features for existing services
  • Preview features for Azure Portal

Use the Azure updates page to keep track of the lifecycle status of features/services.

Sirius Planner 0.2 Released

Today I have released the second closed alpha demo of Sirius Planner. This version builds on and consolidates the features released in the first alpha demo three weeks earlier.

This video gives a quick overview of Sirius Planner 0.2, with a recap for new users and highlighting some of the new features.

This release brings:

  • Complete tag management (CRUD) including listing and prioritising tasks within a tag
  • Rich text descriptions
  • Coloured tags even while editing in task detail view
  • Several usability enhancements
  • Several bugfixes, many of which were reported by people using version 0.1

If you’d like to try Sirius Planner, please fill in the signup form and I’ll send you some credentials.

"You don't learn to walk by following rules. You learn by doing, and by falling over." — Richard Branson