↓ Skip to main content
Oh My Ops | DevOps Guides, stacks & tools for engineers

Secrets Management for Small Teams: Stop Storing Passwords in Slack

·1439 words·7 mins· loading · loading ·
Author
Maksim P.
DevOps Engineer / SRE
Table of Contents

TL;DR
#

  • Never store secrets in git — not in .env files, not commented out, not in “private” repos.
  • Use your platform’s secrets store for CI/CD credentials. Don’t invent your own.
  • For runtime secrets, use a managed secrets manager. Not a shared .env file on a server.
  • Least privilege + rotation are the two levers that limit blast radius when something leaks.

Who this guide is for
#

This is for small teams (3–10 engineers) that are shipping a real product and have secrets spread across too many places: a .env.production file someone has, credentials in Slack DMs, API keys hardcoded in a config file from two years ago that “we’ll clean up later.”

This guide won’t make you SOC2-ready overnight. It will give you a sane, maintainable baseline that catches 90% of the real risks without turning secret management into a full-time job.

What counts as a secret
#

A secret is any value that grants access or that you wouldn’t want in a public leak:

  • Database passwords and connection strings.
  • API keys for third-party services (payment providers, email, SMS).
  • Private keys (TLS, SSH, signing keys).
  • OAuth client secrets.
  • Cloud provider credentials (AWS access keys, GCP service account keys).
  • Internal service-to-service tokens.
  • Encryption keys.

If leaking it would cause you to spend a Saturday rotating things and writing apology emails, it’s a secret.

The wrong patterns (and why they persist)
#

.env files committed to git
#

This is the most common leak vector. It usually starts as .env.example in the repo, then someone commits .env by accident, then it’s in git history forever even after deletion.

Fix: add .env* to .gitignore and enforce it with a pre-commit hook.

Secrets in CI pipeline files
# 

# Don't do this
- run: deploy.sh --db-password=hunter2

Pipeline files are checked into git. Anyone with repo access sees the value. Anyone who forks the repo sees it.

Fix: use your CI platform’s secrets store and reference the variable by name.

Shared credentials everyone knows
#

One database password for the whole team. Everyone connects with it. Nobody knows who used it last. Rotating it breaks three things nobody documented.

Fix: per-service credentials at minimum. Per-person short-lived credentials when possible.

Secrets in application logs
# 

# Don't do this
logger.info(f"Connecting to DB with: {db_url}")  # db_url contains the password

Fix: structured logging, never log full connection strings or credential values.

Secrets sent over Slack / email
#

Someone needs the prod database password urgently. It goes over Slack. Now it’s in Slack’s search history, Slack’s servers, and probably someone’s laptop notifications.

Fix: define a clear process for emergency access. It shouldn’t involve Slack DMs.

Layer 1: CI/CD secrets
#

Every major CI platform has a built-in secrets store. Use it.

  • GitHub Actions: repository secrets, organization secrets, environment secrets.
  • GitLab CI: CI/CD variables (masked and protected).
  • Bitbucket Pipelines: repository variables and deployment variables.

Basic rules:

  • Use environment-scoped secrets where your platform supports it. Staging secrets should not be accessible in prod pipelines, and vice versa.
  • Mask secrets in logs (most platforms do this automatically for stored secrets, but check).
  • Never echo or print secrets in pipeline steps.
  • Give the CI identity the minimum permissions it needs — deploy access to one cluster, write access to one registry.

One practical detail with GitHub Actions: use environment protection rules to require a manual approval before secrets for production are accessible. This adds a gate without much overhead.

Layer 2: Runtime secrets (what your app uses)
#

The question is: how does your application get the secrets it needs to run?

Option A: Managed secrets manager (recommended default) #

Most cloud providers offer a managed secrets service:

  • AWS Secrets Manager — stores secrets with rotation support, costs ~$0.40/secret/month.
  • AWS Systems Manager Parameter Store — cheaper (free tier available), fewer features, good enough for most.
  • GCP Secret Manager — similar to AWS Secrets Manager.
  • Azure Key Vault — equivalent for Azure.
  • HashiCorp Vault — powerful, self-hosted or managed (HCP Vault), more complex to operate.

Your app fetches secrets at startup (or just-in-time) via the provider’s SDK. No secrets in environment variables in plaintext. No secrets on disk.

For most small teams: start with your cloud provider’s native secrets manager. It’s boring, it works, it integrates with IAM, and you don’t have to operate it.

Option B: Environment variables injected at runtime
#

A step up from .env files in the repo — secrets are stored somewhere else and injected as environment variables when your container/process starts.

In Kubernetes: use Kubernetes Secrets (imperfect but standard), or better, an operator like External Secrets Operator that syncs from your secrets manager into Kubernetes Secrets.

In plain Docker/containers: your orchestrator (ECS, Cloud Run, etc.) pulls from Secrets Manager and injects as env vars. The secret value never touches your source code.

What to avoid
#

  • Baking secrets into container images.
  • Storing secrets in Kubernetes ConfigMaps (they’re not secret — ConfigMaps are not encrypted at rest by default).
  • Secrets in Helm values.yaml committed to git.

Layer 3: Access control and least privilege
#

A secret is only as secure as the identities that can access it.

Least privilege means each identity (a CI job, an application, a developer) can only access the secrets it actually needs — not a master key that unlocks everything.

In practice:

  • Your staging app has read access to staging secrets only.
  • Your CI pipeline for frontend deploys doesn’t have access to database credentials.
  • Individual developers don’t have direct read access to production secrets for day-to-day work.
  • Production admin access is restricted, logged, and requires justification.

Use IAM roles (AWS), service accounts (GCP), managed identities (Azure), or Vault policies — not shared access keys passed around.

Layer 4: Rotation
#

Rotation means periodically replacing a secret with a new value. It limits damage if a credential was quietly compromised.

Minimum rotation policy for small teams:

  • Rotate all credentials when a team member leaves (same day, not next sprint).
  • Rotate third-party API keys quarterly or after any security incident.
  • Rotate database passwords at least annually, or use short-lived credentials if your setup supports it.

AWS Secrets Manager and HashiCorp Vault support automatic rotation for common credential types (RDS passwords, IAM keys). This is worth setting up for your most critical secrets.

A rotation drill: once a year, pick one critical secret, rotate it, and verify nothing broke. This validates both your rotation process and your deployment pipeline’s ability to pick up new values without downtime.

Layer 5: Audit and detection
#

You should know when secrets are accessed and by whom.

At minimum:

  • Enable CloudTrail (AWS) or equivalent audit logging on your secrets manager.
  • Set an alert if a secret is accessed from an unexpected source (a developer’s IP in production at 3am is suspicious).
  • Use git-secrets, truffleHog, or gitleaks as a pre-commit hook or CI check to catch accidental commits of secrets.

A 15-minute CI job that scans for committed secrets is worth more than a policy document nobody reads.

Vault: when does it make sense?
#

HashiCorp Vault is powerful. It supports dynamic secrets (credentials generated on demand and auto-expired), fine-grained policies, multiple auth backends, and secret leasing.

For most small teams, Vault is overkill early on. The managed cloud options are simpler to start with.

Consider Vault (or HCP Vault) when:

  • You have multiple cloud providers or hybrid infrastructure.
  • You need dynamic database credentials (connections that auto-expire).
  • You have compliance requirements that demand a single secrets audit trail across all systems.
  • You’re already comfortable operating stateful services.

If nobody on your team has operated Vault before, don’t make it your first priority. Get the basics right first.

Secrets management checklist
#

Foundations
#

  • No secrets committed to git (ever, including history).
  • .env* in .gitignore and enforced.
  • Secret scanning runs in CI.

CI/CD
#

  • All CI credentials stored in the platform’s secrets store.
  • Secrets scoped to environments (staging vs prod).
  • CI identities have least-privilege permissions.

Runtime
#

  • No secrets baked into container images.
  • App secrets fetched from a managed secrets manager (or injected by the orchestrator).
  • No plaintext credentials in Kubernetes ConfigMaps.

Access control
#

  • Per-service credentials, not one shared master password.
  • Production admin access is restricted and logged.
  • Developer access to production secrets requires justification.

Rotation
#

  • Credentials rotated when team members leave.
  • Rotation process is documented and tested.
  • Automatic rotation configured for critical credentials.

Detection
#

  • Audit logging enabled on secrets manager.
  • Secret scanning in CI (truffleHog, gitleaks, or equivalent).

If you can tick these boxes, you’re in better shape than most teams your size.

Related reads #

Reply by Email