Our Approach to External Security Testing
Atlassian is regularly asked for penetration test reports by customers seeking assurance of the processes we have in place to identify (and fix) security vulnerabilities in Atlassian Products and Cloud. Our external security testing approach is built around the concept of 'continuous assurance' – rather than a point-in-time penetration test, we have an always-on, always-testing model using a crowd-sourced bug bounty.
Our philosophy and approach
Atlassian is well known for our values, and those values genuinely influence everything we do – including our approach to security testing. In practice, our values have led us to the following philosophies and approaches:
- Bugs are an unavoidable part of the development process - the question is not whether we have bugs, the question is how effectively and quickly we find them and address them. This doesn’t mean we like bugs or aren’t constantly innovating ways to reduce their frequency and severity, but when it comes to software bugs, denial is not an effective approach.
- We aim to increase the cost of finding and exploiting vulnerabilities in our products and services - through rapidly identifying and resolving issues we aim to increase the economic cost of finding security bugs. By increasing the cost of exploiting a vulnerability – by making it take longer, require more knowledge and more resources from the bad guys – we reduce their return on investment. If we can reduce it far enough, it becomes prohibitive or not attractive to them.
- We support and use industry standards - standardization in our terminology and approach helps us ensure we’re covering everything, and helps customers understand how to understand what we do. For example, common ratings of vulnerabilities using the Common Vulnerability Scoring System (CVSS) ensures clarity for severity of a particular vulnerability between us and our customers. We also follow the vulnerability management processes outlined in ISO 27001 and the Cloud Security Alliance (CSA).
- External security researchers are a valuable extension of our team - if an Atlassian product has a vulnerability in it, it is in everyone’s interest – both ours, and our customers’ – to find it and fix it as quickly as possible. Atlassian incentivizes external researchers to identify vulnerabilities in our products via cash awards from our bug bounty program. With the help of external security researchers, we are able to scale our team far beyond traditional approaches!
- We are open and transparent about our security testing program - our bug bounty program provides statistics about bugs found, we are open about the speed at which we try to fix security bugs, and we make available summary test reports, when possible.
Ongoing security assurance
We do use specialist security consulting firms to complete penetration tests on high risk products and infrastructure. This may be a new infrastructure set up for us (e.g. our Cloud environment), a new product (e.g. Trello) or a fundamental re-architecture (e.g. the extensive use of micro-services).
Our approach to penetration testing in these cases is highly targeted and focused. Such tests will generally be:
- White box - The testers will be provided with design documentation and briefings from our product engineers to support their testing
- Code assisted - The testers will have full access to the relevant code base to help diagnose any unexpected system behaviour during testing and to identify potential targets
- Threat based - Testing will focus on a particular threat scenario, such as assuming a compromised instance exists, and testing lateral movement from that starting point
We post Letters of Assessments (LoA) from our Penetration Testing partners available for external consumption at the bottom of this page. Due to the extensive internal information made available to the testers in conducting these assessments, we do not provide full reports. The majority of these systems and products will subsequently be included in our public bug bounty program, providing the on-going external assurance that our customers seek. Any findings from these assessments will be triaged and remediated according to our Public Security Vulnerability SLO.
Our bug bounty program is hosted by Bugcrowd. The goal of this program is to ensure that our products are being constantly tested for security vulnerabilities. This is the centerpiece of our external security testing program and is the result of significant research, analysis and comparison between testing models.
We believe the crowd of independent security researchers participating in our bug bounty provides the most effective external security testing process because:
- A bug bounty is always on. Penetration tests are generally time-boxed to a few weeks. In a truly agile development environment with frequent releases, continuous testing is a necessity.
- A bug bounty has 60,000+ potential testers. Penetration tests generally have 1 or 2 individuals. Regardless of how good those 1 or 2 individuals are, they will never surpass the aggregate capability of the team of bug bounty participants.
- Bug bounty researchers develop specialised tooling and process vertically (specific bug types) and horizontally (specific bounties). This specialization provides the greatest chance of identifying obscure - but significant - vulnerabilities.
We continue to use penetration testing and specialist security consultants for internal support, but for our broad-coverage external program, a bug bounty is a better fit. We believe the combination of approaches maximizes our chances of finding vulnerabilities.
More than 25 of our products or environments – ranging across our server products, mobile apps and Cloud products – are in-scope for our bug bounty program. Details of the number of vulnerabilities reported, our average response time, and average payout, are all included on the Bugcrowd site, with more than 800 testers having registered specifically for our program.
Vulnerabilities we seek to be identified through our bug bounty program include common types captured in the Open Web Application Security Project (OWASP) and Web Application Security Consortium (WASC) threat lists. They include:
- Cross Instance Data Leakage/Access
- Remote Code Execution (RCE)
- Server-Side Request Forgery (SSRF)
- Cross-site Scripting (XSS)
- Cross-site Request Forgery (CSRF)
- SQL Injection (SQLi)
- XML External Entity Attacks (XXE)
- Access Control Vulnerabilities (Insecure Direct Object Reference issues, etc)
- Path/Directory Traversal Issues
As part of our initiative to be open and transparent, we invite anyone to visit our bug bounty program page, sign up for the program, and test us.
Marketplace Apps - Marketplace Apps are excluded from the main Atlassian Bug Bounty program. However, Marketplace Apps are covered in separate bug bounty programs hosted by Atlassian, such as the Vulnerability Disclosure Program and the Atlassian Built Apps Bug Bounty Program.
While we believe our bug bounty programs provide a more efficient and economical approach for assessing security of our products and services, we understand that you might want to test the security on your own. We allow for security assessments (pen tests, vulnerability assessments) to be performed by customers, we just ask that you follow a few rules to keep all of us safe.
If you do find an issue that you would like to report to Atlassian, please refer to our instructions on how to report a vulnerability.
At Atlassian, one of our values is Open Company, No Bullshit, and we believe that vulnerability disclosure is a part of that value. We aim to fix security vulnerabilities within the relevant service level objectives (SLOs). We accept disclosure requests made through our bug bounty programs after an issue has been fixed and released in production. However, if the report contains any customer data the request will be rejected. If you are planning to disclose outside of our bug bounty programs, we ask that you give us reasonable notice and wait until the associated SLO has passed.
Exclusions from our security testing program
Just as we are open and clear about the testing we do, we are also open and clear about the testing we don’t do ourselves or don’t currently support. The following are excluded from our security testing program:
Certain low-risk vulnerability types - Our products are developed to unleash the potential in every team, and this requires collaboration. Vulnerabilities related to enumeration and information gathering are generally not considered significant risks.
Measuring the right things
We have a security bug fix policy that specifies Service Level Objective (SLO) timeframes for remediating vulnerabilities based on severity and product. When assessing vulnerabilities, we use the Common Vulnerability Scoring System, which helps communicate the severity of vulnerabilities to our customers.
Further, our goal is to increase the cost of finding and exploiting vulnerabilities in our products. We use our bug bounty programs to quantify that cost. As our security posture improves, the number of bugs identified through our bug bounty programs should decrease, and as that happens we will need to increase the amount we are willing to pay for such bugs if we want quality reports to continue coming in. For example, if the amount of effort required to find a vulnerability with a $3000 reward eventually makes it not worthwhile (since it takes more than $3000 worth of effort) we will have increased the cost of finding that vulnerability.
In other words, you should expect to see our bounty payments increase over time.
Atlassian has a mature, open and transparent external security testing program built around our bug bounty.
Want to dig deeper?
We’ve referred to quite a few other documents and resources in this brief paper, and we encourage you to dig into them to understand more about our approach to security testing.
- Atlassian Trust Center
- Atlassian Security Practices
- Atlassian’s CSA STAR
- Atlassian’s Bug-fix Policy
- Atlassian’s Vulnerability Reporting Page
- Atlassian Security Incident Responsibilities
- Atlassian Bug Bounty Home Page
- Atlassian’s Bug Bounty Programs
Download current bug bounty reports
Any security vulnerabilities identified in the reports below are tracked in our internal Jira as they come through the Bug Bounty intake process and any findings from the Bug Bounty will be triaged and remediated according to our Public Security Vulnerability SLO.
- Download the latest Atlassian bug bounty report (2022-07)
- Download the latest Halp bug bounty report (2022-07)
- Download the latest Jira Align bug bounty report (2022-07)
- Download the latest Opsgenie bug bounty report (2022-07)
- Download the latest Statuspage bug bounty report (2022-07)
- Download the latest Trello bug bounty report (2022-07)
Download Letters of Assessment (LoA)
Any security vulnerabilities identified in the reports below are tracked in our internal Jira as they come through the penetration test report process and any findings from these assessments will be triaged and remediated according to our Public Security Vulnerability SLO.
- Letter of Assessment for Micros images and KITT images (2021-05)
- Letter of Assessment for Compass (2021-06)
- Letter of Assessment for JSD Cloud Gmail OAuth (2021-06)
- Letter of Assessment for k8s provider and KubeCompute platform (2021-07)
- Letter of Assessment for Jira Cloud Mobile App (2021-07)
- Letter of Assessment for Trello Black Box Assessment (2021-07)
- Letter of Assessment for Polaris (2021-08)
- Letter of Assessment for Jira Align (2022-04)
- Letter of Assessment for Bitbucket Pipelines (2022-05)