Rotate the Grafana secret key

The fix/etc/grafana/grafana.ini

[security]
secret_key = $__env{GF_SECURITY_SECRET_KEY}

Read the safety section before changing this. Rotating the key without re-encrypting first makes every existing data source stop working.

The default is public

From Grafana’s defaults.ini, unchanged for years:

secret_key = SW2YcwTIb9zpOOhoPsMm

Twenty characters, identical in every Grafana installation in the world, sitting in a public source repository. It is not a placeholder that Grafana replaces at install time — it is the operative value until somebody changes it, and most never do.

What it protects

secret_key encrypts the sensitive fields Grafana stores in its database: data source passwords, API tokens, cloud credentials, TLS client keys and similar secrets. The scheme is AES-256 with PBKDF2 key derivation.

The cryptography is fine. The key is the problem — encryption with a publicly known key is encoding, not encryption. Anyone who obtains the Grafana database can decrypt every credential in it without attacking anything, and open-source tools exist that do exactly this in one command.

So the real question is how often a database escapes, and the answer is: more often than credentials should.

• Backups. grafana.db is a SQLite file that backup jobs copy wholesale, frequently to object storage with looser access control than the Grafana host.
• Container volumes and images. A Docker volume, a snapshot, or an image someone built with data baked in.
• A file-read vulnerability. This is not hypothetical — CVE-2021-43798 let unauthenticated attackers read arbitrary files from Grafana 8.x hosts. grafana.db and grafana.ini were both readable.
• A database server. Grafana can use MySQL or Postgres instead of SQLite, which means a DBA, a replica, or a compromised database host also has it.

In each case, changing this one value is the difference between “an attacker has an encrypted blob” and “an attacker has your production database passwords.”

Why this outranks the admin password

The admin password protects the front door. The secret key protects the contents — and the contents outlive the door.

Rotating an exposed admin password closes the access. Rotating a secret key does not un-decrypt credentials already taken with the old one, which is why the follow-up to any exposure is rotating the credentials themselves at the backends, not just at Grafana.

Generating one

openssl rand -base64 32

Keep it out of grafana.ini for the same reason as the admin password — use $__env{} or, in containers, GF_SECURITY_SECRET_KEY from a secret store.

Grafana also supports envelope encryption with an external KMS (encryption_provider), where a cloud key management service holds the root key and Grafana never sees it. That is the stronger arrangement if you already run a KMS, and it removes this file from the blast radius entirely.

Verify it worked

Check whether you are still on the shipped default. This is the whole check:

grep -E '^\s*secret_key' /etc/grafana/grafana.ini
docker inspect grafana --format '{{range .Config.Env}}{{println .}}{{end}}' | grep GF_SECURITY_SECRET_KEY

No output from either means you are using the default, because an unset value falls through to defaults.ini. Absence is the finding here — which is what makes it easy to miss.

Confirm what the default would decrypt, to size the exposure:

sudo sqlite3 /var/lib/grafana/grafana.db \
  'SELECT name, type FROM data_source WHERE secure_json_data IS NOT NULL;'

Every row is a credential that the public key protects.

Then check who can read the database and its backups:

ls -l /var/lib/grafana/grafana.db
sudo find / -name 'grafana.db*' -not -path '/var/lib/grafana/*' 2>/dev/null

Stray copies outside /var/lib/grafana — in a backup staging directory, a home directory, a container volume — are the realistic exposure path.

Before you apply this

This is the most breakable change on this checklist. Grafana does not transparently re-encrypt when the key changes. Edit secret_key, restart, and every stored secret becomes undecryptable — every data source fails to authenticate, and every dashboard built on them goes blank. It looks like total data loss.

Grafana provides the migration path, and it must run before the key changes:

# 1. Back up first — non-negotiable
sudo systemctl stop grafana-server
sudo cp /var/lib/grafana/grafana.db /var/lib/grafana/grafana.db.bak
sudo systemctl start grafana-server

# 2. Re-encrypt secrets while the OLD key is still in effect
sudo grafana-cli admin secrets-migration re-encrypt-data-keys
sudo grafana-cli admin secrets-migration re-encrypt

# 3. Now change secret_key, then restart
sudo systemctl restart grafana-server

Command names vary across versions — confirm against your own binary before relying on the above:

grafana-cli admin secrets-migration --help

If you have no viable migration path, the honest alternative is to accept a short outage and re-enter the credentials: change the key, restart, then set each data source password again through the UI or provisioning. On an instance with a handful of data sources this is often faster and less risky than the migration.

Do this on a maintenance window with a tested restore. Verify your backup actually opens before you begin:

sudo sqlite3 /var/lib/grafana/grafana.db.bak 'SELECT count(*) FROM data_source;'

And if you are rotating because you believe the database leaked, rotating the key is not sufficient — rotate the credentials at each backend too. The old key already decrypted them.

Related

• Data source credential theft — the chain this default completes.
• Restrict data source permissions — limiting what those credentials can do.