Implementation
Secure Build
The build process
We always strive to have automated build process in all our project (without human interactions).
- Reproducible builds - Every project shall have an embedded/local definition with the project in source control, this should also be executed by the local runner for every developer. This shall be included as a step in the central Continues Integration System (CI) and be hardened as per best practice of the vendor.
- Verification - We should always strive to output verification (checksums) of the artifacts with the build system. This can be used for further verifications regarding tampering of produced artifacts. Builds should fail if the application doesn’t meet a predefined security baseline where we have a maximum accepted severity for vulnerabilities and we always log warnings and failures in CI.
- Secrets - All secrets that the build system needs access to, should be setup with the principal of least privilege and stored securely in CI as per best practice of the vendor.
See our Tools section for what kind of build system that fit your explicit project.
Software dependencies
We keep a record of all dependencies used throughout the target production environment. This is sometimes referred to as a Software Bill of Materials (SBOM). The list can be found under Dependencies
Information on dependencies cover the following
- Why it is used
- Where it is used
- Version we use
- What license it has
- Source information (link to repository, author’s name, etc.)
- Support and maintenance status of the dependency
Note: We should quickly find out which applications are affected by a particular CVE
We analyzed, addressed, and documented findings from dependencies at least during pull requests.
Dependencies vetting
Once a month the engineering manager reviews the Security insights in GitHub. Based on findings and severity tasks are then prioritized for the team to mitigate.
Secure Deployment Process
All our deployments goes through the automated pipeline in (CI/CD), where we explicit use tags on the main branch for releases. Only security officer can alter the configuration of the deployments. Only explicit users can trigger deployments.
For system in use for deployment, see our Tools documentation.
Secrets management
Developers should not have access to secrets or credentials for production environments. Secrets we use are always different between different application instances and environments. We always store sensitive credentials and secrets for production systems with encryption-at-rest at all times.
Todo: We have a Schema - internal link for secrets management and rotation established for each team.
Defects tracking
Defects are tracked in Jira as bugs. The defects are prioritize based on severity.
Metrics and feedback
We conduct quaterly retrospectives where we reflect on our processess and evaluate what we can do to improve. We track the following metrics: Lead Time, Deployment Frequency, Mean Time To Repair (MTTR), and Change Failure Rate (%)
Lead time
The time it takes for work to be implemented, tested, and delivered Shorter production delivery lead times are better since they enable faster feedback on what we are building and allow us to course correct more rapidly. Short lead times are also important when there is a defect or outage, and we need to deliver a fix rapidly and with high confidence.
Measure: When we start working on a story until it is deployed in production.
Deployment frequency
Deployability - Ability to independently deploy an application (ideally - at a component level) separate to other applications or services Testability - Ability to independently test an application (ideally - at a component level) without requiring an integrated environment
Measure: Deployments to production per week.
Mean time to repair
Reliability is measured as time between failures. However, in modern software products and services, which are rapidly changing complex systems, failure is inevitable, so the key question becomes. How quickly can service be restored?
Goal: To keep the less than 120 minutes.
Change failure rate
A key metric when making changes to systems is what percentage of changes to production (including, e.g. software releases and infrastructure configuration changes) fail. A failed release is when we need to apply a hotfix or rollback a change within 48 hours.
Goal: Keep the change failure rate below 10%.