Our Hybrid and Collaborative Summer @ Slack’s Data Eng

An internship at Slack is an exciting opportunity to learn new skills, meet other engineers, and build cool stuff. This was the reality for three interns on the Data Engineering team this summer. Throughout our time in this flex-work environment, we got to experience both the wide reach of the virtual environment and the benefits of in-person work. As data engineering interns, we got to contribute to Slack’s mission through our project which tackles Data Lineage at Slack.

Onboarding

While we are all on the same pillar, Data Engineering, we were each on different teams within the pillar, with separate mentors and managers. Thus, we had slightly different experiences during onboarding. Nonetheless, the Data Lineage project that we tackled this summer was collaborative and spanned all three teams that we were on. As such, it was a collaborative project between us, our mentors, and our managers. In order to ramp up to this project, we had to attend Onboarding at Slack meetings, which allowed us to meet new hires, current engineers, and other interns. While most of the onboarding sessions were done virtually, the organizers created an environment of collaboration, where we felt supported and empowered. We were also encouraged to join scrum meetings with our respective teams, where we were virtually welcomed with open arms, even if we weren’t familiar with all the new terms and tools mentioned. We also joined a virtual icebreaker session, where we got to chat with members of our extended Data Engineering pillar. We found that while many of us were joining remotely from vastly different geographic locations, we could still find commonalities and come together as a team. This made us interns feel welcome and excited to be at Slack.

The project

Midway through our second week, we began to familiarize ourselves with our project. The project itself focuses on data lineage. If you’re wondering what data lineage is, we were asking ourselves the same thing only a few months ago. Here is an example scenario to explain the concept:

Imagine you are an engineer who uses dashboards when communicating with other Slack employees (e.g. to show how your team has made an impact on a metric). You like to use dashboards because they allow your team to track metrics accurately and visually. 

One day, you realize that the dataset you were pulling from has a bug in it. That’s because the dataset that your dataset was derived from also has a bug in it. You don’t know how to find that upstream dataset without data lineage.

Essentially, it’s a form of annotating data to describe where it’s coming from and where it’s going. The data lineage system predated our time at Slack (Data Lineage at Slack), however, there were some shortcomings of this existing system that we addressed:

• A SQL parser was used to find data lineage information. The team tasked with maintaining it had more urgent priorities.
  • Shifting the ownership from a single team could allow more teams to get the most coverage with minimal effort and also makes it more maintainable. 
• The parser would get lineage only from certain types of specialized queries.
  • Shifting to manually annotating lineage could increase coverage.
• There was no visual component of the data lineage system, other than a notification from an internal tool that notifies downstream users of a table via Slack.
  • This notification tool also only extended one level downstream to a table.

After reading relevant resources and meeting with the team, we promptly got to work! Luckily for us, our team had supplied us with documents that would outline the objectives of the frontend and backend portions of our intern project. Starting with our first sprint, we were tasked with familiarizing ourselves with these documents. More importantly, however, this is the sprint where the interns started working together.

Execution

The way we managed our project was through the agile development cycle, where we’d set up tasks for ourselves for a two-week sprint. During the sprint, the three of us each had the chance to be scrum master. We were guided by the mentors and managers, who helped us determine the time our tasks would take to complete and how to split up our Jira tickets. By working with the larger team whilst practicing leadership through being the scrum master, we could track our progress and milestones effectively. Another part of the agile cycle was the retrospective meetings, where the team would vote on topics to talk about and celebrate the wins while recognizing where there’s room for improvement.

In the beginning of the internship, we each met with our mentors individually more than we worked together, and felt that we were too isolated as a result. Therefore, we started scheduling regular work blocks together with all of the mentors. In each work block, we would work together on a chosen task, leveraging this time to make progress. We also allotted time to update each other on our progress and ensure that everyone was on the same page.

Another unique portion of our work as interns was presenting our project to other Slack folks during several stages of our internship. Data Engineering Lightning Talks are a monthly meeting where anyone within the DE group can present what they’re working on, and we had the privilege of presenting at both the beginning and the end of our internships. Showing how our project progressed throughout time was fascinating and harbored some pride in us. We also got to present our project around the midpoint of our internships to the larger CoreDev org for Slack, roughly 200 people. Through these, we were able to garner a better collective understanding of our project while sharpening our public-speaking skills. All in all, we gained better collaboration, presentation, and technical skills during our time on DE at Slack!

Miscellaneous

One thing that was not written in the project requirements was taking the time to have fun with the team. While we thought working together was fun, our team took it another step further. On top of our regular meetings, our managers put together a game session where we got to meet the members of our extended teams and answer questions from a prompt. It sounds quite simple, but we got to go deep — especially upon meeting for the first time. At this point, we felt like we knew the people we were working with on a deeper level. It was great to know that not only would we be working with smart and talented people, but also people who we get along with as well. Other than our regular meetings and virtual get-togethers, we actually got to meet the team when we went to team lunch in Fisherman’s Wharf. We shared some dishes, chatted about how weird it is not to be seeing each other through a Zoom screen, and took some pictures.

Apart from the team events, we also had events organized by  Futureforce, Salesforce’s university recruiting program. We went to Spark social, volunteered to work at McLaren’s park in San Francisco, interacted with leaders at Slack and Salesforce via Speaker Series, and played ping pong at Spin SF. This gave us the opportunity to bond with a huge group of interns outside of Slack. Making friends was easy this summer, thanks to some of these Salesforce events. After years of being stuck inside, it was refreshing to make these friends so easily.

Final thoughts

Hannah’s thoughts – I’ve never had an internship where I worked with other interns closely on the same project, so I was curious to see if I would like it or not. To my surprise, I enjoyed it so much! Having others on my team in the same spot as me meant that we would advocate for each other and work together. Through a combination of experiences, from being the scrum lead to working together on backend tickets, I got to be a leader and learn from others at the same time. Outside of the project, I’ve met people who have made an incredible positive impact on me, who I will remember for the rest of my career.

Fenil’s thoughts – This particular setting of a cross-functional team where we had mentors belonging to different teams worked out well for me. It meant that I was never blocked, because some mentor would always pop in to unstuck me. It also helped me to learn a variety of things, which is the most important takeaway of mine from the internship. I got to acquire knowledge about frontend, backend, data engineering, containerization and deployment, agile methodology, and version control! This was only possible because our project spanned all these different areas, and we had amazing mentors — experts in the field who provided their unparalleled support and helped us grow.

Apart from all the work, I had an amazing time working and collaborating with the interns and our project. We had so much fun together. I also got to meet some amazing Armenian early-stage startup founders who were invited to the Salesforce tower to get to know more about the company.

All these experiences gave me tremendous joy and I couldn’t have asked for any better way to spend my summer.

Derek’s thoughts – Given that all of my prior internships were fully remote, having a hybrid one was an amazing experience. I was able to meet so many cool interns, engineers, managers, and execs, as well as make meaningful connections with them! Also, being able to live in San Francisco was an incredible time, and much more enjoyable than being home all summer. (I would definitely recommend taking walks around the city and exploring as much as possible to future interns!) Lastly, being able to work on a collaborative project with Hannah and Fenil was much more fun than I was expecting. Typically, group projects cause a lot of anxiety surrounding work load balances, but getting to work with other dedicated interns has been amazing. Having access to each other’s mentors has allowed the project to progress much more smoothly than I anticipated.

Reflection and takeaways

Some key takeaways:

• While remote work has its benefits, we really appreciated the chance to meet the team in person and work together.
• As cliche as it sounds, having a team to count on is so rewarding. We win together and lose together, but we always have each others’ backs.
• Having fun and connecting with your team is just as important as the actual project itself. It changes the entire experience and day-to-day life.

Working at Slack and on the Data Engineering team has afforded us a wonderful experience of getting back into the office in person while also having the flexibility of remote work. From meeting each other, our mentors and team members, we’ve thoroughly experienced the benefits of working together, whether it’s from home or in the same room. Having the hybrid option has afforded us the ability to grow professionally in any work setting we may encounter. We appreciate the opportunity to be part of Data Engineering at Slack this summer!

Special thanks

Huge thanks to our managers Suzanna Khatchatrian, Lakshmi Mohan, and Johnny Cao for overseeing our work and supporting us all the way. An extra special thanks to our mentors Joey Thaidigsman, Tom Liao, Shreya Shetty, and Shrushti Patel for consistently helping with the project and making sure we also had fun!