Skip to Content
PracticesOperations

Operations

Incident Mangement

  • We Monitor important functions with alerts to chat and/or email
  • Email or chat is monitored by SREs 24/7
  • The process has a dedicated owner
  • We store process documentation in an accessible location
  • All applications should have a playbook for most common troubleshooting, triaging and process of development
  • The process has an escalation path for further analysis
  • We train employees responsible for incident detection in this process
  • We have a checklist of potential attacks to simplify incident detection
  • We have an agreed upon incident classification
  • The process considers Root Case Analysis for high severity incidents
  • Employees responsible for incident response are trained in this process
  • Forensic analysis tooling is available
  • We perform reviews at least annually
  • We update the checklist of potential attacks with external and internal data

Environment Management

  • We use least privilege for applications where applicable by using managed identity (azure)
  • We use WAFs with at least inspect and check the logs where applicable for externally published web apps
  • We segment the access between systems based on vNets (prefereable) or external IP’s or service tags where applicable
    • When service tags are used, we try to add a second “factor” to the firewall rule (for example auth header, or front door id)
  • Auditing is configured and sent to a central log analytics workspace and with alerts for unusual activity
  • We use PIM for admin elevation
  • We use MFA for all accounts
  • All important applications shall have availability tests
  • Applications/functions should be classified (NetworkLayer: Internal, External) (Importance: Critical, Important, NotImportant)
  • We take backups of all applicable data
  • We ship backups to a second location

Infrastructure as code

  • We deploy our infrastructure templates with the respective CD pipeline if there’s any changes to the template
  • We deploy all main software “areas” through templates that are first deployed and tested in the development environment then production.
  • Preferred templates are Bicep for azure but others are allowed too.
  • We strive towards using azure integrations that don’t require a user/password (see openidconnect  integrations)

Operational Management

  • We maintain a data catalog, including types, sensitivity levels, and processing and storage locations
  • We know which data elements are subject to specific regulation
  • We have retention requirements for data
  • We regularly evaluate the lifecycle state and support status of every software asset and underlying infrastructure component, and estimate their end of life
  • We use a scheduled what-if deployment to keep track of configuration drift.
  • All applications shall log telemetry to APM solution (app insights in azure)
  • Telemetry shall not contain PII data
  • All applications should, if applicable, expose a health endpoint for availability monitoring
  • We periodically run disaster recovery scenarios
Last updated on