Tag: Microsoft Fabric

Azure, Fabric Administrator, Fabric Capacity, Logic Apps, Microsoft Fabric, Power BI, Power BI Premium Capacity, Power BI Service

Microsoft Fabric: Capacity Cost Management Part 2, Automate Pause/Resume Capacity with Azure Logic Apps

Automate Pause Resume Suspend Fabric Capacity with Azure Logic Apps

In the previous blog post, I explained Microsoft Fabric capacities, shedding light on diverse capacity options and how they influence data projects. We delved into Capacity Units (CUs), pricing nuances, and practical cost control methods, including manually scaling and pausing Fabric capacity. Now, we’re taking the next step in our Microsoft Fabric journey by exploring the possibility of automating the pause and resume process. In this blog post, we’ll unlock the secrets to seamlessly managing your Fabric Capacity with automation that helps us save time and resources while optimising the usage of data and analytics workloads.

Right off the bat, this is a rather long blog, so I added a bonus section at the end for those who are reading from the beginning to the end. With that, let’s dive in!

The Problem

As we have learned in the previous blog post, one way to manage our Fabric capacity costs is to pause the capacity while not in use and resume it again when needed. While this can help with cost management, as it is a manual process, it is prone to human error, which makes it impractical in the long run.

The Solution

A more practical solution is to automate a daily process to pause and resume our Fabric capacity automatically. This can be done by running Azure Management APIs. Depending on our expertise, there are several ways to achieve the goal, such as running APIs on running the APIs via PowerShell (scheduling the runs separately), running the APIs via CloudShell, creating a flow in Power Automate, or creating the workflow in Azure Logic Apps. I prefer the latter, so it is the method that this blog post explains.

Automating Pause and Resume Fabric Capacity with Azure Logic Apps

Here is the scenario: we are going to create an Azure Logic Apps workflow that automatically does the following:

  • Check the time of the day
  • If it is between 8 am to 4 pm:
  • Check the status of the Fabric capacity
  • If the capacity is paused, then resume it, otherwise do nothing
  • If it is after 4 pm and before 8 am:
  • Check the status of the Fabric capacity
  • If the capacity is resumed, then pause it, otherwise do nothing

Follow these steps to implement the scenario in Azure Logic Apps:

  1. Login to Azure Portal and search for “Logic App“
  2. Click the Logic App service
Finding Logic Apps on Azure Portal

This navigates us to the Logic App service. If you currently have existing Logic Apps workflows, they will appear here.

Continue reading “Microsoft Fabric: Capacity Cost Management Part 2, Automate Pause/Resume Capacity with Azure Logic Apps” →
Azure, Fabric Administrator, Fabric Capacity, Microsoft Fabric, OneLake, Power BI, Power BI Premium Capacity, Power BI Service

Microsoft Fabric: Capacity Options and Cost Management, Part 1; The Basics

Microsoft Fabric: Capacity Options and Cost Management, Part 1

Microsoft Fabric is a SaaS platform that allows users to get, create, share, and visualise data using a wide set of tools. It provides a unified solution for all our data and analytics workloads, from data ingestion and transformation to data engineering, data science, data warehouse, real-time analytics, and data visualisation. In a previous blog post, I explained the basics of the Microsoft Fabric data platform. In a separate blog post, I explained some Microsoft Fabric terminologies and personas where I explained what Tenant and Capacities are.

In this blog post, we will explore the different types of Fabric capacities, how they affect the performance and cost of our Fabric projects, and how you can control the capacity costs by pausing the capacity in Azure when it is not in use.

Fabric capacity types

Fabric capacities are the compute resources that power all the experiences in Fabric. They are available in different sizes and prices, depending on our needs and budget. We can currently obtain Fabric capacities in one of the following options:

If we want to purchase Microsoft Fabric capacities on Azure, they come in SKUs (Stock Keeping Units) sized from F2 – F2048, representing 2 – 2048 CU (Capacity Units). A CU is a unit of measure representing the resource power available for a Fabric capacity. The higher the CU, the more resources we get on our Fabric projects. For example, an F8 capacity has 8 CUs, which means it is four times more powerful than an F2 capacity, which has 2 CUs.

When purchasing Azure SKUs with a pay-as-you-go subscription, we are billed for compute power (which is the size of the capacity we choose) and for OneLake storage, which is charged for the data stored in OneLake per gigabyte per month (approximately $0.043 (New Zealand Dollar) per GB). OneLake is the unified storage layer for all the Fabric workloads. It allows users to store and access our data in a secure, scalable and cost-effective way.

Azure Fabric capacities are priced uniquely across regions. The pay-as-you-go pricing for a Fabric capacity at Australia East region is $0.3605 (NZD) per CU per hour, which translates to a monthly price of $526.217 (NZD) for an F2 ($0.3605 * 2 * 730 hours).

Microsoft Fabric pricing overview
Microsoft Fabric pricing overview

It is important to note that billing is per second with a one-minute minimum. Therefore, we will be billed for when the capacity is not in use. Here is a full list of prices available at the Azure portal by selecting our Fabric capacity region.

Now that we have an indication of the costs of owning Microsoft Fabric capacities let’s explore the methods to control the cost.

Nuances of Fabric’s Cost of Ownership

It is important to note that all the math we have gone through in the previous section is just about the capacity itself. But are there any other costs that may apply? The answer is it depends. If we obtain any SKUs lower than F64, we must buy Power BI Pro licenses per user on top of the capacity costs. For the tiers above F64, we get unlimited free users but, BUT, we still have to purchase Power BI Pro licenses for all developers on top of the cost of the capacity itself.

Another gotcha is that the Fabric experiences are unavailable to either Power BI Premium (PPU) users or the Power BI Embedded capacities. Just be mindful of that.

The good news for organisations owning Power BI Premium capacities is that you do not need to do anything to leverage Fabric capabilities. As a matter of fact, you already own a Fabric capacity, you just need to enable it on your tenant.

Continue reading “Microsoft Fabric: Capacity Options and Cost Management, Part 1; The Basics” →
Azure DevOps, Data Visualisation, DAX, Microsoft Fabric, Power BI, Power BI Desktop, Power BI Embedded Capacity, Power BI Premium Capacity, Power BI Service, Self-Service BI, Source Control

Integrating Power BI with Azure DevOps (Git), part 2: Local Machine Integration

Integrating Power BI with Azure DevOps (Git), part 2: Local Machine Integration

This is the second part of the series of blog posts showing how to integrate Power BI with Azure DevOps, a cloud platform for software development. The previous post gave a brief history of source control systems, which help developers manage code changes. It also explained what Git is, a fast and flexible distributed source control system, and why it is useful. It introduced the initial configurations required in Azure DevOps and explained how to integrate Power BI (Fabric) Service with Azure DevOps.

This blog post explains how to synchronise an Azure DevOps repository with your local machine to integrate your Power BI Projects with Azure DevOps. Before we start, we need to know what a Power BI Project is and how we can create it.

What is Power BI Project (Developer Mode)

Power BI Project (*.PBIP) is a new file format for Power BI Desktop that was announced in May 2023 and made available for public preview in June 2023. It allows us to save our work as a project, which consists of a folder structure containing individual text files that define the report and dataset artefacts. This enables us to use source control systems, such as Git, to track changes, compare revisions, resolve conflicts, and review changes. It also enables us to use text editors, such as Visual Studio Code, to edit the artefact definitions more productively and programmatically. Additionally, it supports CI/CD (continuous integration and continuous delivery), where we submit changes to a series of quality gates before applying them to the production system.

PBIP files differ from the regular Power BI Desktop files (PBIX), which store the report and dataset artefacts as a single binary file. This made integrating with source control systems, text editors, and CI/CD systems difficult. PBIP aims to overcome these limitations and provide a more developer-friendly experience for Power BI Desktop users.

Since this feature is still in public preview when writing this blog post, we have to enable it from the Power BI Desktop Options and Settings.

Enable Power BI Project (Developer Mode) (Currently in Preview)

As mentioned, we first need to enable the Power BI Project (Developer Mode) feature, introduced for public preview in the June 2023 release of Power BI Desktop. Power BI Project files allow us to save our Power BI files as *.PBIP files deconstruct the legacy Power BI report files (*.PBIX) into well-organised folders and files.
With this feature, we can:

  • Edit individual components of our Power BI file, such as data sources, queries, data model, visuals, etc.
  • Use any text editor or IDE to edit our Power BI file
  • Compare and merge changes
  • Collaborate with other developers on the same Power BI file

To enable Power BI Project (Developer Mode), follow these steps in Power BI Desktop:

Continue reading “Integrating Power BI with Azure DevOps (Git), part 2: Local Machine Integration” →
Data Activator, Data Lake, DirectLake, Fabric Administrator, Fabric Capacity, Lakehouse, Microsoft Fabric, Notebook, OneLake, Power BI, Real-time Analytics, Synapse Data Engineering, Synapse Data Science, Synapse Data Warehouse

Microsoft Fabric: Terminologies and Personas Explained

In this blog post, I will explain some of the key concepts, personas, and terminologies related to Microsoft Fabric, a SaaS analytics platform for the era of AI. If you are not familiar with the basic concepts of SaaS analytics platforms and how Microsoft Fabric fits in, I recommend you read my previous blog post, where I explain them in detail.

Microsoft Fabric is an experience-based platform, meaning users can interact with it depending on their roles and personas. For example, a data engineer can use the Data Engineering experience to perform large-scale data transformation through the lakehouse. A data scientist can use Data Science experience to develop AI models on a single foundation without data movement. A business analyst can use the Power BI experience to create and consume interactive reports and dashboards. And a data steward can use the Data Activator experience to govern and secure data across the organisation.
The Data Activator experience is in private preview and is not available for public use yet!

Microsoft Fabric Terminologies

To understand how Microsoft Fabric works, it is crucial to know some of the terminologies that are used in the platform. Some of them are existing terms that are also used in Power BI or Azure services, while some of them are new and specific to Microsoft Fabric. Here are some of the key terms that you should know:

  • Tenant: A tenant is a dedicated instance of Microsoft Fabric that is provisioned for an organisation or a department within an organisation. A tenant has its own set of users, groups, permissions, capacities, workspaces, items, and experiences. A Fabric tenant is associated with an Azure Active Directory (AAD) tenant, which is a directory service that the organisations own when they sign up for a Microsoft cloud service such as Azure, Microsoft 365, Power BI, etc. AAD provides identity and access management for cloud applications. A tenant in Microsoft Fabric can only be accessed by users who belong to the same AAD tenant.
  • Capacity: Capacity is a term that refers to the amount of resources available to support a computing service. In the context of SaaS applications, capacity refers to the ability of the system to handle a certain amount of load or demand based on the required resources and infrastructure such as compute power (CPU, RAM, etc.), storage, network bandwidth and whatnot. As explained in my previous post, Microsoft Fabric is a SaaS platform. So, from a Microsoft Fabric perspective, capacities are sets of resources that are allocated to a tenant to run analytics workloads. The capacities sit in a tenant, and the available resources can be shared by multiple workspaces or dedicated to a single workspace for better performance and isolation. Microsoft Fabric capacities are available in various F SKUs that offer different levels of resources and features. For more information about capacities and SKUs, see Microsoft Fabric Capacity and SKUs.
  • Workspace: A workspace is a logical container that holds a collection of items and artefacts. A workspace can have one or more owners who can manage its settings and permissions and one or more members who can access its items. A workspace can also be assigned to a capacity to run its analytics workloads. In Microsoft Fabric, workspaces are based on Power BI workspaces.

The above terms also apply to Power BI, so they have been used within the community for a long time. The hierarchy starts with an organisation acquiring their potential Tenants, and then the purchased Capacities are available to tenants and the Workspaces that are assigned to capacities.

Continue reading “Microsoft Fabric: Terminologies and Personas Explained” →