XM Knowledge Base
Software Engineering Careers
With millions of responses per day and a goal to grow their data platform 10x over the next year, the Qualtrics engineers are tackling interesting challenges across the stack. We talked to software engineers Kira Booth, Amir Abdullah, and Greg Burnham, along with principal engineer Scott Pead, about the benefits and challenges of moving away from a monolithic infrastructure and how Qualtrics remains democratic and transparent amid rapid growth. Interested in joining the team? Check out open roles to learn more. Want to learn more about the engineering organization? See here to learn about our engineering culture and the challenges we are solving.
Let’s start off with the basics. What does Qualtrics do?
Greg: Qualtrics enables three core activities: collecting data, analyzing it, and then acting on it. In the beginning, we focused on collection; today we are equally interested in empowering our customers to connect the dots and make key business decisions based on their data.
Scott: Our goal is to help companies understand how potential customers, current customers, users, and employees perceive them — and how that impacts their success. As Qualtrics has grown, our focus has shifted from primarily market research to now addressing customer experiences for enterprise companies. We’re growing in both the scale and the breadth of data we need to represent.
Tell us about some of the interesting challenges you are solving. Maybe we can start on the backend?
Kira: All four of us are involved with the data platform, which means we work with the data once it’s been collected but before we start seriously analyzing it. We currently help customers process millions of responses per day.
Scott: We currently have over 9,000 customers and we’re growing astronomically. To support that, our goal is to grow our data platform so it can handle ten times more data in the next year. And then do that again the following year. That kind of growth is audacious, but it’s what’s needed to support growth as fast as ours. A particular challenge we’re facing amidst that growth is implementing standardizations around different types of data. We are working out how to blend numerical data — about companies’ subscriptions, or sales, or content engagement — with experience data collected from surveys. To do that, we utilize technologies like Kafka and Spark and really fast data stores to create a real-time engine to transform data into actionable observational reports. To me, that’s really interesting, to be doing the kind of heavy-lifting power crunching you see at Facebook or Google.
Amir: I’m on a team that’s building a text analytics engine from scratch. It’s challenging because text is much more open-ended than numbers. We’re doing search, sentiment analysis, and modeling across different languages — English, German, French, Spanish. With sentiment analysis, we want to identify whether people are happy based on their responses, but we have thousands of customers in different industries, and one word doesn’t always mean the same thing. If someone says a book is “mysterious,” that’s a compliment, but if someone says a website is “mysterious,” that’s not good. Figuring out how to parse subtle differences like that is an interesting challenge.
Greg: We face a lot of interesting scaling challenges on the Survey Platform team, but we’re also tackling 100% uptime requirements, largely by updating our survey engine to a hub-and-spoke architecture. The data gets recorded and processed in the hub but the surveys are available out on the spokes. We are building in redundant availability so, if one spoke goes down, we can process that survey on another spoke. There are also latency concerns. If we’re sending a survey to a humanitarian organization in rural Africa with a spotty internet connection, how do we make sure the survey performs properly? Is that via an application that doesn’t require an internet connection? Is it via extremely lightweight surveys? Is it via servers located near that geographic region? There are a lot of different ways to answer that question.
Kira: I currently help run two teams. One is Data Processing, where we clean up data so it can be meaningfully analyzed. The other is the Exports and Imports team. On the imports side, we’re enabling customers to import and analyze millions of outside responses within the Qualtrics platform. Exports pose a scaling challenge, because we have so many types and lengths of responses. It also requires a certain amount of transformation of the data from internal unique IDs to words and numbers recognized and defined by the client.
How about the frontend. What are you working on there?
Greg: For a long time our work has been informed by the slogan, “Simple enough for an intern, sophisticated enough for a PhD.” Collecting millions of different types of responses in real time — and presenting that data in public dashboards for people who don’t use our software — is a complex challenge. We want a fresh grad to be able to do the same research as someone who’s been in the industry for years.
Amir: The backend might have all this complex stuff going on, but we want the user to find it intuitive and simple.
Kira: For instance, how do you represent n-dimensions on a screen in a way that makes sense to someone without a PhD in computer science? How do we make that intuitive? How do we make it flexible, so it shows what the customer wants?
On the Data Processing team, we want to give our customers ultimate flexibility in how they use their data — whether that’s being able to tag it with outside information, group it together, or derive data based on certain fields. We also want them to get a gut feeling for whether their data matches their expectations. Histograms and reports don’t always give that to you, so we provide different depths of analysis. Before you dive into individual responses, we show you filtered data, for example.
How would you describe the moment the Qualtrics engineering team is in?
Greg: We’re in the process of shifting from a monolithic architecture, where everything is coupled together, to a microservice architecture, where each piece of the platform talks over the wire to another piece.
Kira: That gives teams a lot of power. For instance, each team has the freedom to use whatever technology will best solve the problem at hand. I’ve owned services written in at least eight different languages using three different database technologies.
Greg: I feel like the Engineering team morale is particularly high right now. The team is enabled to do cool things without too much legacy stuff holding them back. We can run so much faster. People coming in now will be able to have a huge impact on where the company is 12 months from now.
Why did you decide to join Qualtrics?
Scott: Well, there was a vulture.
A vulture?
Scott: Yep. While I was comparing job offers, I was in the garden behind the office and someone walked up carrying a wounded vulture that had landed back there. I’m a student of history, and, according to the ancient Romans, vultures are second only to eagles in auspiciousness — so here I am.
Kira: That’s a hard answer to follow. There were no vultures for me. Meritocracy is one of the reasons I came — and a big reason I stay. I got my degree in computer science with an emphasis in video game development, but I didn’t go into that industry because I had heard so many horror stories. Larger video game companies can be hyper-competitive and they basically work newbies 100 hour weeks until they burn out and quit.
I really wanted to be at a company where I felt valued. I joined Qualtrics three years ago, right out of school, and I routinely sit in meetings with people who’ve been in the industry for decades. And my good ideas are taken just as seriously as theirs. A good idea is a good idea here, no matter who it comes from. That’s what keeps me here.
What makes Qualtrics different from other places you’ve worked?
Scott: My last job was fairly corporate, but I was in startups for 11 years before that. Qualtrics has the startup feel I was looking for — I’ve been particularly impressed at how open and democratic it is here. You can bring an idea directly to the Vice President of Engineering. I’ve had experiences at other companies where I’ve written up an initiative and sent it up the chain only to be yelled at because that’s somebody else’s territory. There’s nothing like that here. It’s obvious that this culture was intentionally designed to remain very meritocratic.
Amir: Transparency is a big thing here, like at a lot of companies, but I appreciate there’s space for confidentiality too. By confidentiality, I don’t mean secrecy, but sometimes you might need to have a challenging conversation in private. There’s space at Qualtrics to talk to your manager or supervisor and say, “Hey, nothing personal, but this is something I’m not happy with.”
Greg: We’ve done trainings over the years that have taught us how to approach difficult conversations and how to give meaningful feedback.
Transparency can be tough to maintain at bigger companies. How do you keep it going?
Scott: For one thing, our business is employee engagement and experience management, so we use our own tools internally to gather feedback and to reflect on direction. When we find something that’s an issue, we take action.
Greg: We have problems, like any company, but, unlike most companies, we never ignore them. On the contrary, we actively seek them out so that we can address them.
What’s the hardest thing about your work?
Greg: It’s interesting because the same independent, democratic, dynamic, make-your-own-decision culture we appreciate as engineers has also introduced a lot of pain for the Engineering team. The positives far outweigh the negatives, but it would be dishonest to pretend we don’t have problems. I think Engineering is ahead of our skis on a lot of things, for instance. The company is growing so quickly, no matter how good our team is, we have to be scrappy and make decisions we know might not be ideal in the long-run. We know that’s necessary sometimes — the alternative is stagnation — we just have to be sure we’re always building two-way doors so we can go back to it later.
Kira: The fact that every team is free to choose its own technology presents challenges also, including features needing to be duplicated in different languages. That’s why it’s important that we have so many smart, experienced colleagues to help us think through the risks and advantages of any decision. That helps us make quick decisions about taking calculated risks, one of the key components of innovation.
What have you learned here?
Greg: I’ve stayed here for five years because of the opportunity to learn things beyond a typical engineering skill set. I think a lot of engineers here, maybe without even knowing it, are observing things that will change their careers in the future. For instance, you learn so much from observing the shift from monolith to service-oriented architecture. Lots of companies never make a move like that. Or they start out service-oriented and you don’t learn the benefits of a monolithic structure. Or maybe you see the way that our Sales organization is growing, and that helps provide context if you want to become a CTO someday. It’s really unusual to get to observe how a successful company works before it becomes too corporate. Like, I worked at Apple before. It’s impossible to know how Apple really works. But at Qualtrics, it’s still possible to understand every piece of the product and company.
Kira: Despite being only a few years out of school, I’ve been given the opportunity to lead two separate teams at Qualtrics. That means I’ve had to learn how to manage people who are much smarter and more experienced than me.
Greg: Ryan Smith, our CEO, told us we’ll all hire people who are smarter than us. We have to if we’re going to succeed.
Kira: Exactly. At first that was super intimidating. The people on my team are brilliant when it comes to technical engineering. But then I realized I have non-technical strengths that are necessary for my team’s success. Having a culture of openness and candid feedback has also helped. As I’ve read my team members’ code and they’ve read mine, I’ve been able to step up technically and contribute to more advanced conversations.
Have you come up with any solutions you’re particularly proud of?
Amir: One thing we’re working on across the company is encryption. Data needs to be encrypted for security reasons, but then you have these big, encrypted blobs of binary data — and text analytics is all about searching for specific words. I’ve been working on making it searchable; incorporating the encryption context into lemmatization, so you can generate encrypted lemmas. It’s been a fun challenge. We’ve also been investigating an interesting technique involving domain adaptation. When we are presented with huge amounts of unlabeled data, we’re working on how to automatically detect your domain and intelligently adapt the algorithm to it.
Kira: I’m personally most proud of how the Import-Export team has supported our legacy exporters. Our oldest exporter wasn’t built for the millions of responses we handle now. We’re currently finishing our new exporter, and it’s been an interesting dance. Our data storage is asynchronous, so responses aren’t stored in the order we export them, which created a technical challenge in ensuring correctness of the ordering of data.
How often do you see the impact of your work on the end user?
Kira: Because the company is so transparent and we’ve developed relationships with people outside of Engineering, it’s fairly common for us to hear feedback from customers. As a small example, the other night, somebody from the Text Analytics team hopped onto my Slack channel to notify me that a customer gave us a low NPS score and mentioned SPSS exports in the comment. That the information was available, someone was paying attention, and they could easily ping me to help solve the problem says a lot about how things work here.
Do you collaborate with other teams frequently? What is that like?
Greg: Absolutely. We have so many opportunities to interact with each other that when a cross-departmental task comes up, we feel super comfortable working together. I’ll run into people from other teams during lunch, or during our Thursday company-wide meetings, or read their funny posts on our internal social network.
Amir: And if you don’t already have a relationship with someone from another team, managers are always happy to make that connection. Or, it’s easy to go straight to someone yourself. I interned at Google, which was cool, but it was also huge. It’s nice to get to know people here, even on different teams, and not be lost in a huge crowd.
Scott: Last week I was in a blues bar in Seattle with the entire Engineering staff. It’s pretty novel to see the VP throwing down with the Director of Tech Ops, and co-workers playing imaginary pianos. We’re very open here. Very human.