At Slack, we’re committed to security that goes beyond the ordinary. We continuously strive to earn and maintain user trust by safeguarding critical components integral to every user’s experience. From passwords to session cookies, and tokens to webhooks, we prioritize protecting everything essential to how users log into the platform and remain authenticated. Through proactive measures and innovative automations that leverage cutting-edge threat intelligence, we’re dedicated to shielding users from potential breaches, cookie hijacking malware, and inadvertent exposure of sensitive information and secrets.
Secrets should remain secret
Slack’s strategy has always been to anticipate and mitigate threats before they can impact our users. Since 20161, we have worked to proactively invalidate credentials exposed on the internet based on regular expressions2 tailored to the specifics of our tokens and webhooks. Oftentimes these secrets get inadvertently exposed when they get hard-coded into development code and then published somewhere like GitHub. Since these secrets provide varying levels of access to a user’s workspace, our tooling automatically and immediately invalidates tokens and webhooks upon discovery and notifies their respective owners.
Following this, we aimed to extend the same level of protection and automation to Slack passwords and session cookies. Password reuse across multiple platforms poses a significant risk to user security. A 2023 study on account takeovers found that 70% of victims reported that they reused the same password across multiple sites and services, leading to 53% of them having had multiple accounts taken over.3 Put in numbers, 29% of American adults experienced an account takeover by 2023, equating to roughly 77.5 million victims according to government population figures.4 At the same time, passwords and session cookies are also susceptible to malware that is built to steal it from a user’s browser, something we’ll get into below.
Exposed password detection and rotation
This led us to collaborate with strategic threat intelligence partners who collect data from diverse sources such as breaches, dark web forums, botnets, and malware. These partnerships provide us with high-fidelity, actionable data in real-time that lets us stay ahead of threat actors, while also making Slack a less appealing target by rendering credentials stolen by those threat actors invalid and useless.
We continuously ingest this threat intelligence via our partners’ APIs and proactively find matches between the credentials of our users and those appearing in threat actor datastreams. When a match is found, that credential is immediately reset and blocked from being reused by the associated user in the future now that it’s compromised. Oftentimes we can catch these vulnerable passwords so quickly that we’re able to reset them before a threat actor is able to use them to gain unauthorized access to an account.
This process of comparing passwords provided by our threat intelligence partners with entries in our database isn’t straightforward, however. While credential breach data is available in plaintext, allowing bad actors to use it, Slack passwords are securely stored as salted hashes, making a direct comparison impossible. To solve this, we created a data pipeline that automatically ingests candidate credentials from our threat intelligence data sources and then salts and hashes each password so that a comparison to Slack’s database can be made before the datapoint is purposefully—and permanently—discarded.
Although the process of salting and hashing each candidate password is deliberately time consuming, we’re able to process millions of credentials5 in a reasonable amount of time within the secure confines of Slack’s backend. We accomplish this by dividing them into smaller batches and processing them in parallel in a job queue. When our backend process identifies a match, the associated user’s password is immediately reset and the user is notified via an email explaining this activity, alongside some follow-up actions they can take to improve the security of their account going forward. At this point or if no match is found, the pipeline discards the datapoint so threat intelligence data is never accumulated or stored by Slack.
Invalidating hijacked cookies
All cookies of any app or service, including the Slack session cookie each Slack user possesses, are locally stored on a user’s device. This local storage affords benefits like speed, efficiency, scalability, and offline functionality, but it also produces a security risk. If a threat actor is able to compromise that user’s device, they could also gain access to the cookies on that device and use the Slack cookies to gain access to the user’s workspaces.
To proactively guard against this possibility, apart from monitoring for signs of cookie misuse within Slack, we also constantly monitor threat intelligence datastreams and invalidate exfiltrated Slack cookies in a timely manner, balancing security with user experience. This extends the initial system we implemented that discovers and invalidates user session tokens on a Slack user’s behalf so the cookies are also protected from malicious use.
Oftentimes, our threat intelligence partners uncover hijacked Slack cookies so quickly6 that we’re able to not only invalidate each before a bad actor could have the chance to misuse them, but in a way that’s tailored to each respective user’s geography and timezone. When a Slack cookie is invalidated, its associated session gets marked for termination, which once complete logs the user out of their workspace. That’s a good thing, in terms of protecting the user’s accounts from unauthorized access, but we also know no one wants to lose access to Slack during a critical conversation or in the middle of presenting in a huddle.
During runtime, our automation reviews each compromised cookie to evaluate whether the associated user’s geography means it’s during their typical weekday working hours. If so, the invalidation of that specific cookie is scheduled to occur in a time window outside of that range, while cookies belonging to users who are not currently within their weekday working hours are invalidated immediately. This lets us provide a positive user experience that considers each user’s timezone while calculating the most efficient and timely invalidation for the exfiltrated cookie.
As with exposed passwords, when a cookie gets invalidated we notify the impacted user via email. Additionally, if that user’s workspace is on an Enterprise plan supporting Slack Audit Logs, we also add an event for the cookie invalidation into their audit logs for transparency.
Conclusion
Our dedication to security at Slack extends beyond conventional measures by leveraging cutting edge threat intelligence with innovative automations for discovering and invalidating vulnerable user credentials at scale. We firmly believe that proactively safeguarding against existing and emerging threats is not only a necessity for fostering a secure platform, but critical for sustaining user trust in our brand. We also pride ourselves on designing approaches that emphasize a seamless and transparent user experience, all while simultaneously implementing robust security protocols to thwart unauthorized access attempts by threat actors.
Interested in helping us protect Slack users?
Apply now
Footnotes
1 https://medium.com/slack-developer-blog/slack-tokens-what-they-are-how-theyre-used-and-how-to-keep-your-data-safe-ed1b00d2715a
2 A regular expression (or ‘regex’) is a code pattern used to find a specific string(s) of text within larger bodies of text. They’re made up of symbols and characters that inform a computer what text you’re searching for, what parts of text can be ignored, or even what groups of text need to appear together for a match to be valid etc. Although every Slack token contains a unique secret string inside of it, the rest of the token follows the same formatting pattern every Slack token uses. This consistency allows us to create regular expressions that can efficiently and accurately locate tokens within code bodies without us needing to specify the secret credential component.
3 https://www.security.org/digital-safety/account-takeover-annual-report/
4 https://wonder.cdc.gov/population-projections-2014-2060.html
5 It’s important to note we prevent threat actors from doing the same by enforcing strict rate limits around how many user login attempts can be made in any given period of time. This way, a threat actor can’t rapidly cycle through a list of credentials in hopes of finding one that matches the credential a Slack user has chosen without Slack blocking their login attempts.
6 Our threat intelligence partners gain access to raw data-streams from sources like a malware’s system logger and extract Slack-related intel so we can take preventative action before that same data is processed and put up for sale on the dark web. Imagine going to a bakery expecting to pick up fresh warm chocolate chip cookies, only to discover someone has already picked out all the chocolate chips from the batch while it was still in the backroom and now stale leftovers are all that’s available for purchase.
