DataCamp's engineering team grew from 4 to 24 people to serve over half a million students. They built a microservices architecture with Docker and Kubernetes, plus internal tools like the TEACH editor for course creation. The platform uses Redis for personalized recommendations and a mixed backend stack of Node, R, Python, and Spark.
Chapters
Hi, my name is Jonathan. I'm Vincent.
I'm Lin.
I'm Filip.
Hi, I'm Dieter. I'm the CTO at datacamp. I co founded a company with Jonathan and Martin three years ago. Datacamp allows everyone from different demographics, different backgrounds. They have high quality, affordable data science education.
We have over half a million students.
Now taking data science courses on datacamp and in the last year our team.
Has grown from 4 to 24 people.
I work in a teach engineering team. My job is to make the process of creating courses easier for everyone. Basically, I make internal tools like the teacher and the TEACH editor and they will help course creators.
I work for the sales and marketing engineering team. We take care of internal tools like Messenger, Community and the API. We created a new application, the messenger, and this is going to communicate with Redis profiles that are being populated by the main app API. Maybe you have completed one course but not completed another one or. But you said that you were interested in a special one, so we're going to let you know. Okay, here's this course that you would like.
I work on a learning engineering team and I'm actually responsible with my team for everything related to learning. So we do everything we can to give people the best learning experience in the application. So that's about the interface you get when you're learning, but also everything that happens behind the backend so that your session runs fine.
Datacamp uses a microservices architecture. All components run inside of a Docker container that are handled by container orchestration platform called Kubernetes. The reason behind using Microservices is it allows us to upgrade all components from old technologies to newer ones. We have Angular, React, Node and so on.
In the learning engineering team, we use a bunch of different programming languages because our team is actually inherently multidisciplinary. We start with a JavaScript application on the front end, but in the back end we also have a node framework that then spins up Docker containers, that feature R, Python, but also SQL, Hadoop, Spark, all these things. So we have to know about a bunch of different programming tools.
You're working hard, but it doesn't feel like that. So you do a lot of work, but it feels like you're sitting down with a bunch of talented people and doing some great stuff.
