Engineering at Slack
type: channel
description: Hear directly from Slack's engineers about what we build, why and how we build it, and how you can use it.
date: 2026-03-19T16:21:18Z
update period: hourly
update frequency: 1
update base: 2000-01-01T12:00+00:00
generator agent: https://wordpress.org/
items: How Slack Rebuilt Notifications 📣
items: Streamlining Security Investigations with Agents
items: Android VPAT journey
items: Build better software to build software better
items: Advancing Our Chef Infrastructure: Safety Without Disruption
items: Deploy Safety: Reducing customer impact from change
items: Building Slack’s Anomaly Event Response
items: Optimizing Our E2E Pipeline

How Slack Rebuilt Notifications 📣
type: item
creator: Frances Coronel
date: 2026-03-19T19:00:54Z
subject: Uncategorized
subject: android
subject: backend
subject: cross-platform
subject: design
subject: frontend
subject: ios
subject: product-launch
subject: push-notification
description: Introduction 🔔 At Slack, notifications are how teams stay in the loop, but they can also become overwhelming when not designed with intention. Our goal was to make staying informed feel effortless. We set out to rebuild one of Slack’s most complicated systems from the ground up by bringing calm, consistency, and clarity to the…
encoded: Introduction At Slack, notifications are how teams stay in the loop, but they can also become overwhelming when not designed with intention. Our goal was to make staying informed feel effortless. We set out to rebuild one of Slack’s most complicated systems from the ground up by bringing calm, consistency, and clarity to the experience. Diagnosing the Noise Problem We knew perception of noise in Slack was a universal challenge affecting teams everywhere. Across workspaces, notification overload consistently ranks among the most common frustrations. Research showed that the more channels a person joins, the more likely they are to feel overwhelmed and confused about notification behavior. Internally, the data told a clear story. Issues with notifications are one of the top three drivers of Customer Experience tickets, with users often unsure how to control or understand their settings. The noise problem wasn’t just about volume—it was baked into the architecture itself. Our legacy notification system evolved over years, accumulating complexity that made it nearly impossible for users to understand or control. Four conflicting mental models: Desktop and mobile each had their own preference systems, with different options and behaviors. A “nothing” setting on mobile meant something entirely different from “Off” on desktop. Users couldn’t predict what would happen when they changed a setting. Hidden coupling between preferences: What users were notified about was tightly coupled with how they received notifications. Wanting fewer push notifications meant sacrificing in-app awareness entirely—there was no way to separate the two. Inconsistent state across clients: Settings didn’t reliably sync. Users would configure notifications on desktop only to find mobile behaving completely differently, leading to confusion and duplicate configuration work. Power users left behind: Advanced controls were scattered across multiple menus with no clear hierarchy. Features like “badge …

Streamlining Security Investigations with Agents
type: item
creator: Dominic Marks
date: 2025-12-01T16:00:42Z
subject: Uncategorized
subject: development
subject: security
subject: software-engineering
description: Slack’s Security Engineering team is responsible for protecting Slack’s core infrastructure and services. Our security event ingestion pipeline handles billions of events per day from a diverse array of data sources. Reviewing alerts produced by our security detection system is our primary responsibility during on-call shifts. We’re going to show you how we’re using AI…
encoded: Slack’s Security Engineering team is responsible for protecting Slack’s core infrastructure and services. Our security event ingestion pipeline handles billions of events per day from a diverse array of data sources. Reviewing alerts produced by our security detection system is our primary responsibility during on-call shifts. We’re going to show you how we’re using AI agents to optimize our working efficiency and strengthen Slack’s security defenses. This post is the first in a series that will unpack some of the design choices we’ve made and the many things we’ve learnt along the way. The Development Process The Prototype At the end of May 2025 we had a rudimentary prototype of what would grow into our service. Initially, the service was not much more than a 300 word prompt. The prompt consisted of five sections: Orientation: “You are a security analyst that investigates security alerts […]” Manifest: “You have access to the following data sources: […]” Methodology: “Your investigation should follow these steps: […] ” Formatting: “Produce a markdown report of the investigation: […]” Classification: “Choose a response classification from: […]” We implemented a simple “stdio” mode MCP server to safely expose a subset of our data sources through the tool call interface. We repurposed a coding agent CLI as an execution environment for our prototype. The performance of our prototype implementation was highly variable: sometimes it would produce excellent, insightful results with an impressive ability to cross-reference evidence across different data sources. However, sometimes it would quickly jump to a convenient or spurious conclusion without adequately questioning its own methods. For the tool to be useful, we needed consistent performance. We needed greater control over the investigation process. We spent some time trying to refine our prompt, stressing the need to question assumptions, to verify data from multiple sources, and to make use of the complete set of da…

Android VPAT journey
type: item
creator: Hye Jung Choi
date: 2025-11-19T18:00:39Z
subject: Uncategorized
subject: a11y
subject: accessibility
subject: android
subject: empathy
subject: ios
subject: ui
description: Background A Voluntary Product Accessibility Template (VPAT) is a document that outlines how well a product aligns with accessibility (a11y) standards. Its primary purpose is to inform customers about a product’s a11y features, enabling them to make informed decisions before purchasing software. At Slack, we conducted a VPAT by a third party a11y vendor in…
encoded: Background A Voluntary Product Accessibility Template (VPAT) is a document that outlines how well a product aligns with accessibility (a11y) standards. Its primary purpose is to inform customers about a product’s a11y features, enabling them to make informed decisions before purchasing software. At Slack, we conducted a VPAT by a third party a11y vendor in 2024 following our significant UI change (IA4). During this VPAT process, we identified a11y issues across Android, iOS, and desktop. Of the issues identified on Android, we immediately assigned the obvious shovel-ready issues, such as those with color contrast and missing image labels, to the responsible teams for resolution. To address remaining Android issues, we conducted a comprehensive triage process. In this document, we take a look at the major recurring themes of Android accessibility issues and share resolution strategies and lessons learned in the hopes of helping others make similar improvements. Identified themes [Note: for resolved and for future work.] P1: Error messages are inaccessible P2: Headings are not identified P2: Missing a11y label on EditField P2: Incorrect number of items in list P2: Drag and drop in the workspace switcher is inaccessible P2: Strikethrough information not conveyed to screen reader users P3: Errors indicated by color alone P3: Keyboard Navigation and Focus Keyboard navigability can help customers with motor and visual impairments use Slack on Android. It’s more commonly used on larger form factors (e.g. tablets). We received numerous VPAT tickets related to the keyboard navigation, however, Slack on Android currently does not yet support the large form factor. As a result, we have decided to explore other solutions that align with recommended practices and standards. Issue resolution P1: Error messages are inaccessible. Report If the user enters an invalid input to an edit field and presses “Next” (or Submit), an error message is displayed, but the error is not communicat…

Build better software to build software better
type: item
creator: David Reed (he/his)
date: 2025-11-06T16:00:47Z
subject: Uncategorized
subject: build-performance
subject: caching
subject: ci-cd
subject: developer-experience
subject: devops
subject: python
description: We manage the build pipeline that delivers Quip and Slack Canvas’s backend. A year ago, we were chasing exciting ideas to help engineers ship better code, faster. But we had one huge problem: builds took 60 minutes. With a build that slow, the whole pipeline gets less agile, and feedback doesn’t come to engineers until…
encoded: We manage the build pipeline that delivers Quip and Slack Canvas’s backend. A year ago, we were chasing exciting ideas to help engineers ship better code, faster. But we had one huge problem: builds took 60 minutes. With a build that slow, the whole pipeline gets less agile, and feedback doesn’t come to engineers until far too late. We fixed this problem by combining modern, high-performance build tooling (Bazel) with classic software engineering principles. Here’s how we did it. Thinking About Build (and Code) Performance Imagine a simple application. It has a backend server that provides an API and data storage, and a frontend that presents the user interface. Like many modern applications, the frontend and backend are decoupled; they can be developed and delivered independently. The graph of this build looks like this: <img loading="lazy" decoding="async" width="1642" height="430" class="alignnone size-full wp-image-17605" src="https://slack.engineering/wp-content/uploads/sites/7/2025/10/DevXP-Council-Presentation6-e1759869480617.png" alt="An example service build graph. It shows a set of Python files building into a backend artifact and a set of TypeScript files building into a frontend artifact." srcset="https://slack.engineering/wp-content/uploads/sites/7/2025/10/DevXP-Council-Presentation6-e1759869480617.png 1642w, https://slack.engineering/wp-content/uploads/sites/7/2025/10/DevXP-Council-Presentation6-e1759869480617.png?resize=640,168 640w, https://slack.engineering/wp-content/uploads/sites/7/2025/10/DevXP-Council-Presentation6-e1759869480617.png?resize=768,201 768w, https://slack.engineering/wp-content/uploads/sites/7/2025/10/DevXP-Council-Presentation6-e1759869480617.png?resize=1280,335 1280w, https://slack.engineering/wp-content/uploads/sites/7/2025/10/DevXP-Council-Presentation6-e1759869480617.png?resize=1536,402 1536w, https://slack.engineering/wp-content/uploads/sites/7/2025/10/DevXP-Council-Presentation6-e1759869480617.png?resize=380,100 380w, https:/…

Advancing Our Chef Infrastructure: Safety Without Disruption
type: item
creator: Archie Gunasekara
date: 2025-10-23T18:17:56Z
subject: Uncategorized
subject: aws
description: Last year, I wrote a blog post titled Advancing Our Chef Infrastructure, where we explored the evolution of our Chef infrastructure over the years. We talked about the shift from a single Chef stack to a multi-stack model, and the challenges that came with it – from updating how we handle cookbook uploads to navigating…
encoded: Last year, I wrote a blog post titled Advancing Our Chef Infrastructure, where we explored the evolution of our Chef infrastructure over the years. We talked about the shift from a single Chef stack to a multi-stack model, and the challenges that came with it – from updating how we handle cookbook uploads to navigating the limitations around Chef searches. If you haven’t had a chance to read that post yet, I highly recommend checking it out first to get the full context for this post. At Slack, keeping our service reliable is always the top priority. In my last post, I talked about the first phase of our work to make Chef and EC2 provisioning safer. With that behind us, we started looking at what else we could do to make deploys even safer and more reliable. One idea we explored was moving to Chef Policyfiles. That would have meant replacing roles and environments and asking dozens of teams to change their cookbooks. In the long run, it might have made things safer, but in the short term it would have been a huge effort and added more risk than it solved. So instead, this post is about the path we chose: improving our existing EC2 framework in a way that doesn’t disrupt cookbooks or roles, while still giving us more safety in our Chef deployments. Splitting Chef Environments Previously, each instance had a cron job that triggered a Chef run every few hours on a set schedule. These scheduled runs were primarily for compliance purposes — to ensure our fleet remained in a consistent and defined configuration state. To reduce risk, the timing of these cron jobs was staggered across availability zones, helping us avoid running Chef on all nodes simultaneously. This strategy gave us a buffer: if a bad change was introduced, it would only impact a subset of nodes initially, giving us a chance to catch and fix the issue before it spread. However, this approach had a critical limitation. With a single shared production environment, even if Chef wasn’t running everywhere at o…

Deploy Safety: Reducing customer impact from change
type: item
creator: Sam Bailey
date: 2025-10-07T16:33:12Z
subject: Uncategorized
subject: automation
subject: ci-cd
subject: deployment
subject: engineering
subject: incident-response
subject: infrastructure
subject: observability
description: It’s mid 2023 and we’ve identified some opportunities to improve our reliability. Fast forward to January 2025. Customer impact hours are reduced from the peak by 90% and continuing to trend downward. We’re a year and half into the Deploy Safety Program at Slack, improving the way we deploy, uplifting our safety culture and continuing…
encoded: It’s mid 2023 and we’ve identified some opportunities to improve our reliability. Fast forward to January 2025. Customer impact hours are reduced from the peak by 90% and continuing to trend downward. We’re a year and half into the Deploy Safety Program at Slack, improving the way we deploy, uplifting our safety culture and continuing our rate of change to meet business needs. Defining the problem System requirements change as businesses evolve whether that be due to changes in customer expectations, load & scale or response to business needs. In this case analysis showed two critical trends: Slack has become more mission critical for our customers, increasing expectations of reliability for the product as a whole and for specific features each company relies on. The increasing majority (73%) of customer facing incidents were triggered by Slack-induced change, particularly code deploys. We use our incident management process heavily and frequently at Slack to ensure we have a strong and rapid response to issues of all shapes and sizes based on impact and severity. A portion of those incidents are customer impacting, many of which showed variable impact depending on which features they used in the product. Change-triggered incidents were occurring in a wide variety of systems and deployment processes. Finally, we also received customer feedback that interruptions became more disruptive after about 10 minutes – something they would treat as a “blip” – and this would continue to reduce with the introduction of Agentforce in 2025. All this was occurring in a software engineering environment with 100s of internal services and many different deployment systems & practices. In the past, our approach to reliability often zeroed in on individual deploy systems or services. This led to manual change processes that bogged down our pace of innovation. The increased time and effort required for changes not only dampened engineering morale but also hindered our ability to swiftly…

Building Slack’s Anomaly Event Response
type: item
creator: Nathan Lehotsky
date: 2025-09-04T10:00:02Z
subject: Uncategorized
subject: incident-response
subject: security
description: As cyberattacks evolve to unprecedented levels of sophistication and speed, the time gap between breach detection and response has never been more critical. Traditional security approaches often operate reactively, identifying compromises only after damage has occurred. This delay grants attackers a tactical advantage, forcing security teams to focus on damage assessment and remediation rather than…
encoded: As cyberattacks evolve to unprecedented levels of sophistication and speed, the time gap between breach detection and response has never been more critical. Traditional security approaches often operate reactively, identifying compromises only after damage has occurred. This delay grants attackers a tactical advantage, forcing security teams to focus on damage assessment and remediation rather than proactive threat detection and prevention. Organizations urgently need solutions that dramatically compress the detection-to-response window to regain a defensive advantage. To tackle this challenge, we’ve developed Anomaly Event Response (AER) – a new proactive defence mechanism inside Slack. By combining real-time monitoring with advanced analytics, AER autonomously identifies high-confidence threat actor behaviours as they emerge on our platform. When suspicious activity is detected, the system automatically terminates the associated user sessions, reducing the security detection and response gap from potential days/hours to mere minutes. The result? A powerful native security capability that disrupts attack chains before they can be fully executed, preventing data exfiltration and system compromise without requiring additional security tools, integration, or human capital. In this post, we’ll explore the technical architecture behind Anomaly Event Response, examining our motivation, detection mechanisms, response capabilities, and the insights we gained along the way. The Shared Responsibility of Securing Slack Slack is where work happens. We’re a central hub for workplace communication and collaboration for tens of millions of users on a weekly basis, generating billions of interactions daily. To ensure our customers’ security, we provide comprehensive audit logs to Enterprise customers that record when entities take an action on the platform. Within the full list includes hundreds of actions that we support, we also provide customers with anomaly audit logs which ta…

Optimizing Our E2E Pipeline
type: item
creator: Dan Carton
date: 2025-04-14T09:00:30Z
subject: Uncategorized
subject: ci-cd
subject: developer-experience
subject: developer-productivity
subject: devops
subject: frontend
subject: testing
description: In the world of DevOps and Developer Experience (DevXP), speed and efficiency can make a big difference on an engineer’s day-to-day tasks. Today, we’ll dive into how Slack’s DevXP team took some existing tools and used them to optimize an end-to-end (E2E) testing pipeline. This lowered build times and reduced redundant processes, saving both time…
encoded: In the world of DevOps and Developer Experience (DevXP), speed and efficiency can make a big difference on an engineer’s day-to-day tasks. Today, we’ll dive into how Slack’s DevXP team took some existing tools and used them to optimize an end-to-end (E2E) testing pipeline. This lowered build times and reduced redundant processes, saving both time and resources for engineers at Slack. The Problem: Unnecessary Frontend Builds For one of our largest code repositories (a monolithic repository, or monorepo), Slack has a CI/CD pipeline that runs E2E tests before merging code into the main branch. This is critical for ensuring that changes are validated across the entire stack for the Slack application: frontend, backend, database, and the handful of services in between. However, we noticed a bottleneck: building the frontend code was taking longer than expected and occurred too frequently, even when there were no frontend-related changes. Here’s the breakdown: Developer Workflow: A developer makes changes and pushes to a branch. Build Process: The frontend code is built (~5 minutes). Deployment: The build is deployed to a QA environment. Testing: We run over 200 E2E tests, taking another 5 minutes. This entire process took about 10 minutes per run. Half of that time, around 5 minutes, was consumed by frontend builds, even when no frontend changes were involved. <img loading="lazy" decoding="async" width="1661" height="780" class="alignnone size-medium wp-image-17430" src="https://slack.engineering/wp-content/uploads/sites/7/2025/04/image4.png?w=640" alt="Stages of the end to end testing pipeline." srcset="https://slack.engineering/wp-content/uploads/sites/7/2025/04/image4.png 1661w, https://slack.engineering/wp-content/uploads/sites/7/2025/04/image4.png?resize=640,301 640w, https://slack.engineering/wp-content/uploads/sites/7/2025/04/image4.png?resize=768,361 768w, https://slack.engineering/wp-content/uploads/sites/7/2025/04/image4.png?resize=1280,601 1280w, https://slack…