Microsoft Fabric: Resolving Capacity Admin Permission Issues in Automate Capacity Scaling with Azure LogicApps

Resolving Capacity Admin Permission Issues in Automate Capacity Scaling with Azure LogicApps

A while back, I published a blogpost explaining how to use Azure LogicApps to automate scaling Microsoft Fabric F capacities under the PAYG (Pay-As-You-Go) licensing option. Some of my followers reported an issue with their Capacity Admin settings when running the LogicApp solution. The issue was that their capacity admins disappeared after they had run the LogicApps to upscale or downscale the capacity. After some investigation, I found out what the problem was. At the same time, some of my other followers suggested a fix which involved hardcoding the admins into the solution. While this would work in some cases, it is not a practical solution in the long run, as the admin settings may evolve over time. This makes the solution hard to maintain and unreliable. Back then, I suggested using the APIs and an HTTP action in a new LogicApps solution. This blog is the continuation of the previous blog and a quick and easy fix that ensures the automation runs smoothly with minimal to no manual work or maintenance afterwards. I have also published a tutorial video on YouTube explaining the process from the beginning (which was already covered in my previous blog, so I do not explain it here again) which you can watch here:

A Reminder of the Previous Solution

I suggest you read my blog about automating Fabric capacity scaling with Azure LogicApps as it provides a step-by-step guide to implement the solution. But if you have already implemented, or you are just after the fix, jump to the next section. The following image shows how the original solution works:

Automate Scaling Microsoft Fabric F Capacities with Azure LogicApps
Automate Scaling Microsoft Fabric F Capacities with Azure LogicApps

Here is a quick explanation of how it works:

  1. The Trigger runs the workflow automatically every hour.
  2. The Read a Resource, which is an Azure Resource Manager operation, reveals information about a resource that, in our implementation, is a Microsoft Fabric Capacity.
  3. A condition to check the Status of the capacity. If the capacity is Paused (the condition is true), then do nothing. This is needed as this method only works when the capacity is Active.
  4. If the capacity is Active (the condition is false), then check the current time to see if it is between 2pm and 4pm. This is the timeframe for which we want to upscale the capacity.
  5. If the condition is True, then upscale the capacity to F8 using another ARM operation: Create or update a resource.
  6. If the condition is False, then set the capacity SKU to F2.

The solution works fine if you do not have any Capacity Admin settings either on Azure Portal or on Admin Portal on Microsoft Fabric. But in many cases, we indeed have capacity admins. Let’s see what the issue is.

The Problem

The issue arises when we add some capacity administrators; that are wiped after running the above solution in its current implementation. The following image shows the Capacity Admin settings on both portals:

Fabric Capacity Admins on Azure Portal and Fabric Admin Portal
Fabric Capacity Admins on Azure Portal and Fabric Admin Portal

The reason if that the Create or update a resource also updates the properties of the resource with the ones we define in LogicApps. Therefore, if we do not add any capacity admins, we literally empty the existing capacity admins. Let’s run the solution again to understand why this is happening. The following image shows my capacity admins are wiped out after running my LogicApp workflow:

Capacity Admins deleted after running LogicApps
Capacity Admins deleted after running LogicApps

The issue is also demonstrated in the tutorial video on YouTube:

Continue reading “Microsoft Fabric: Resolving Capacity Admin Permission Issues in Automate Capacity Scaling with Azure LogicApps”

Microsoft Fabric: Overcome Reaching the Maximum Number of Fabric Trial Capacities

Microsoft Fabric Overcome Reaching the Maximum Number of Fabric Trial Capacities

If you are evaluating Microsoft Fabric and do not currently own a Premium Capacity, chances are you’re using Microsoft Fabric Trial Capacities. All Power BI users within an organisation or specific security groups given the rights can opt into Fabric Trial Capacities. Therefore, you may already have several Trial Fabric Capacities in your tenant. Your Fabric Administrators can specifically control who can opt into the Fabric Trial capacities within the Fabric Admin Portal, on the Help and support settings section, and enabling the Users can try Microsoft Fabric paid features setting as shown in the following image:

Enable Users can try Microsoft Fabric paid features for specific security groups via Fabric Admin Portal
Enable Users can try Microsoft Fabric paid features for specific security groups via Fabric Admin Portal

The authorised users can then opt into Fabric Trial by following this process:

  1. Click the Account Manager on the top right corner of the page
  2. Click the Start trial button
  3. Click the Start trial button again
  4. Provide the required details
  5. Click the Extend my free trial button

The following image shows the preceding steps:

Start Fabric Free Trial
Start Fabric Free Trial

As you see, opting into Fabric Trial is simple, unless it isn’t!

There are cases where authorised users cannot start their Fabric Trial because their tenant has already exceeded the limit of available trial capacities. In that case, the users get the following message:

Continue reading “Microsoft Fabric: Overcome Reaching the Maximum Number of Fabric Trial Capacities”

Incremental Refresh in Power BI, Part 2; Best Practice; Do NOT Publish Data Model Changes from Power BI Desktop

Incremental Refresh Best Practice, Do NOT Publish Changes from Power BI Desktop

In a previous post, I shared a comprehensive guide on implementing Incremental Data Refresh in Power BI Desktop. We covered essential concepts such as truncation and load versus incremental load, understanding historical and incremental ranges, and the significant benefits of adopting incremental refresh for large tables. If you missed that post, I highly recommend giving it a read to get a solid foundation on the topic.

Now, let’s dive into Part 2 of this series where we will explore tips and tricks for implementing Incremental Data Refresh in more complex scenarios. This blog follows up on the insights provided in the first part, offering a deeper understanding of how Incremental Data Refresh works in Power BI. Whether you’re a seasoned Power BI user or just getting started, this post will provide valuable information on optimising your data refresh strategies. So, let’s begin.

When we publish a Power BI solution from Power BI Desktop to Fabric Service, we upload the data model, queries, reports, and the loaded data into the data model to the cloud. In essence, the Power Query queries, the data model and the loaded data will turn to the Semantic Model and the report will be a new report connected to the semantic model with Connect Live storage mode to the semantic model. If you are not sure what Connect Live means, then check out this post where I explain the differences between Connect Live and Direct Query storage modes.

The Publish process in Power BI Desktop makes absolute sense in the majority of Power BI developments. While Power BI Desktop is the predominant development tool to implement Power BI solutions, the publishing process is still not quite up to the task, especially on more complex scenarios such as having Incremental Data Refresh configured on one or more tables. Here is why.

As explained in this post, publishing the solution into the service for the first time does not create the partitions required for the incremental refresh. The partitions will be created after the first time we refresh the semantic model from the Fabric Service. Imagine the case where we successfully refreshed the semantic model, but we need to modify the solution in Power BI Desktop and republish the changes to the service. That’s where things get more complex than expected. Whenever we republish the new version from Power BI Desktop to Fabric Service, we get a warning that the semantic model exists in the target workspace and that we want to Overwrite it with the new one. In other words, Power BI Desktop currently does not offer to apply the semantic model changes without overwriting the entire model. This means that if we move forward, as the warning message suggests, we replace the existing semantic model and the created partitions with the new one without any partitions. So the new semantic model is now in its very first stage and the partitions of the table(s) with incremental refresh are gone. Of course, the partitions will be created during the next refresh, but this is not efficient and realistically totally unacceptable in production environments. That’s why we MUST NOT use Power BI Desktop for republishing an already published semantic model to avoid overriding the already created tables’ partitions. Now that Power BI Desktop does not support more advanced publishing scenarios such as detecting the existing partitions created by the incremental refresh process, let’s discuss our other options.

Alternatives to Power BI Desktop to Publish Changes to Fabric Service

While we should not publish the changes from Power BI Desktop to the Service, we can still use it as our development tool and publish the changes using third-party tools, thanks to the External Tools support feature. The following subsections explain using two tools that I believe are the best.

Continue reading “Incremental Refresh in Power BI, Part 2; Best Practice; Do NOT Publish Data Model Changes from Power BI Desktop”

Microsoft Fabric: Use Copilot to Generate Data Model Synonyms

Microsoft Fabric: Use Copilot to Generate Data Model Synonyms

One of my older posts explains how to enable Copilot on Fabric and how to use Copilot to generate Power BI reports. In this post, I aim to explain yet another use case for Copilot that can help us to make a better and more useful semantic model in Power BI using synonyms. In an old post published in May 2016, I explained how to use Power BI synonyms to take our Power BI Q&A experience to another level. In that post, I explained how we could use synonyms to translate data model objects in different languages so the end-user could ask questions in their native language and get the results in Power BI. That was such a cool use case for synonyms, I suppose, wasn’t it? Fast track to December 2023, I believe the Q&A is still one of the coolest Power BI features that stands out when demoing the solutions to the customers; therefore, it makes absolute sense to use synonyms to improve the Q&A‘s efficiency and accuracy. This blog post explores the possibility of using Copilot to define synonyms in Power BI Desktop.

Prerequisites

As explained here, we first need to enable Copilot on Fabric Service. Please note that the technique explained in this post requires either Power BI Premium Capacity or at least F64 Fabric capacity and won’t work on PPU, Embedded capacities, Fabric capacities smaller than F64 or Fabric Trial (FT) capacities.

We also need to have the latest version of Power BI Desktop installed on our machine. With that, let’s begin.

Using Power BI Copilot to generate synonyms

While defining synonyms for the semantic model objects significantly helps with the Q&A experience, it is still a cumbersome process if done manually. So, if we meet the prerequisites, we can summon Copilot to the rescue. Follow these steps after opening a Power BI file in Power BI Desktop:

  1. Ensure you’re signed into Fabric service with your account
  2. Click the Insert tab
  3. Select the Q&A visual
  4. On the Q&A visual, click the Q&A Setup button shown with a gear icon
  5. On the Q&A Setup window, you must see a message offering to “Improve Q&A with synonyms from Copilot” on top of the window; click the Add synonyms button

The following image shows the preceding steps:

Improve Q&A with synonyms from Power BI Copilot in Microsoft Fabric
Improve Q&A with synonyms from Copilot
Continue reading “Microsoft Fabric: Use Copilot to Generate Data Model Synonyms”