It’s 2022, it’s been almost a year since the last (and first) post. A lot has happened, including the beginning of the end of the pandemic, and a paradigm shift of telemetry and instrumentation for the software company I work at. I have finally started the transition to introduce Grafana at work - and with that transition, came the upfront cost of expensive trials and tribulations. How do I make this SSL? How do I add Azure AD to Grafana? How do I reverse proxy Grafana with IIS?

My environment

Before anyone yells at me for not using Linux. We’re a .NET shop and a lot, if not all, our applications actively run on Windows-based hosts. This might change in the future. We’re not locked to Windows, except for our WPF apps, but we’re certainly comfy with Windows and I’m more comfortable in a Windows environment than Linux environment. So, here is the environment I’m working with:

  • Windows Server 2019 (Standard)
  • IIS 10
  • Grafana (self hosted) 8.3.6
  • AzureAD
  • Certify The Web 5.6.5

This assumes a base, vanilla, installation of Grafana situated on localhost:3000. If you have changed your port, adapt these instructions.

Adding an IIS Reverse Proxy

Outcome: Adding bindings for HTTP(S) traffic on Default Web Site for IIS.

  1. Launch Certify The Web, create a new managed certifficate for the target Website
    • Before requesting a certificate, under Tasks add a new task to Export Certificate. Under Task Parameters:
      • Keep Authentication as Local
      • Change destination file path to a file path of your choice, in my case C:\SSL\cert.pem
      • Change Export As to PEM - Primary Certificate
    • Once you have added this task, repeat the process, this time adding an export certificate task for the PEM - Private Key

Your deployment tasks should look like this:

image

  1. Request the certificate and verify the bindings have been allocated to the website on IIS
  2. Install the URL Rewrite IIS module
  3. Install the Application Request Routing IIS module
  4. Open your target website and select the URL rewrite module, clicking Add Rule(s)... on the right hand panel

image

image

  1. With the new rule dialog open, in the Inbound Rules, enter https://localhost:3000.

Modifying Grafana config for SSL

When we are going to add AzureAD OAuth, we need a https redirect - unfortunately, we cannot tell Grafana to blindly run on https, it requires a valid certificate to do so. Leaving it empty will cause Grafana to (gracefully?) crash at startup.

Go to the Grafana configuration ini, for my Windows Server installation, that’s at C:\Program Files\GrafanaLabs\grafana\conf. Ensure you edit sample.ini or any renamed variant of this. I have called my custom.ini. Do not edit the defaults.ini file if you can help it.

  1. Stop Grafana service, if it’s running
  2. Open the ini configuration file
  3. Find the [server] section
  4. Uncomment and change protocol to protocol = https
  5. Uncomment and change domain to your desired domain, i.e. domain = grafana.contoso.com
  6. Uncomment and change root_url to root_url = %(protocol)s://%(domain)s/, the important note here is we’re removing the port
  7. Uncomment and change cert_file to cert_file = C:\SSL\cert.pem or the location of your certificate file, as exported in Certify The Web’s manager
  8. Uncomment and change cert_key to cert_key = C:\SSL\key.pem or the location of your certificate’s key file, as export in Certify The Web’s manager
  9. Start Grafana service and verify it is reachable via your domain

Because IIS will be running on 443. We cannot run Grafana on 443 by default, it will continue to run on 3000. By changing the root_url configuration, we’re forcing Grafana to act as though it’s directly hosted on 443, rather than behind a reverse proxy.

The disadvantage to this is that you will no longer be able to access Grafana via localhost, if that’s a requirement, unless you are willing to accept the ““untrusted”” certificate. I say ““untrusted”” because it was generated by us, for the target domain.

Adding Azure AD to Grafana

This is mostly a documented process on the official Grafana documentation, which is great. Assuming you have followed these steps, modified the config further to include AzureAD and restarted your Grafana service, you will now see a Log in with Microsoft button, using your AzureAD tenant.

The redirect URL should work, and your reverse proxy is none the wiser.

This process was mostly trial and error. It is not perfect by any means and I will be revisit it once we have to migrate servers/versions, but it does get AzureAD working in Grafana on self deployed environments. At least in Windows.

Want help?

Comment here and I will do my best to respond.