The video follows a machine learning engineer (Speaker B) through a typical research Monday at Max Kelsom’s ML team. The day begins with a casual coffee break and a walk through a tunnel, then shifts to a focused session on reinforcement learning (RL). The team reads a Medium article about state‑of‑the‑art Montezuma gameplay, discusses a recent under‑performing OpenAI competition entry, and decides to target the Pommerman competition—a multi‑player Bomberman‑style game—using a custom RL agent. Speaker B outlines a step‑by‑step learning plan: reviewing the Pommerman GitHub repo, studying OpenAI Gym examples, and deep‑diving into the Proximal Policy Optimization (PPO) algorithm by reading the paper and running baseline code. Throughout, informal banter about coffee, motorbikes, lunch preferences, and word‑choice (busy vs. productive) reveals the team’s collaborative culture, emphasis on continuous learning, and personal productivity mindset. The day ends with a brief exchange about a new TAFE screen‑and‑media course and a sign‑off emphasizing the “keep learning” mantra.
Chapters
Morning!
Today's research day at Max Kelsom. I'm excited. We've dedicated— well, the ML team have dedicated Mondays to reading papers and looking up new ways of doing things. Give me a second, I need to open this gate. Keep the parthies out. I'm not entirely sure what we'll be exploring. I think it'll be some something to do with maybe reinforcement learning or PyTorch. Been really wanting to start learning PyTorch lately, but whatever it is, whatever we end up diving into, as long as the other tasks that we have to do are taken care of, like writing reports and working on client projects, we'll get into some research.
Thanks, right?
You know, after years, my whole life living in Brisbane, I never knew this bridge existed. Well, I'd seen it, but I'd never been through it. Look how cool it is. It's just like a tunnel over the street. All these new places you can find when you just go exploring a little. I'm gonna go get some coffee and do some riding. How are you? Excellent, how are you?
Good, good.
Can I see inside?
Yeah, yeah. How cool is this chair I'm in?
Actually, how cool is this cafe I'm in? Check this out. Motorbike, motorbike, motorbike. Epic artwork. About half a dozen motorbikes down there. I don't yet know how to ride a motorbike, but I will learn one day. Sorry, Mum. Now I just finished writing. I took a bit longer than I thought. I got to get to work.
Lunchtime!
It's been a real productive morning. Actually, it's about 2 PM now, so lunch is a little bit late, but that's all right. Been having fun. That's a word I've been trying to swap out actually, is— everything's falling over, this table has ridges in it— is swapping out busy for productive, right? Because I think I'm not the biggest fan of the word busy. Busy to me sounds like you've got so many things going on, but productive Simple little word change and makes all the difference. Do you want to eat lunch with me? Nah, that'll be a bit boring. Let me tell you the good stuff. I'm going to eat this because I'm starving and then we'll go for a walk of what I've been up to on research day. It's been real fun. Peanut butter is well and truly my favorite food. Only the ones that are 100% peanuts though. Actually, almond butter is my favorite food. Peanut butter is a close second. Perks of having your own jar. You can eat straight from the jar. That might offend some people, but enough eating.
Let's go for a walk.
So we started off the morning reading a Medium article which discussed the state-of-the-art play for Montezuma, and Montezuma is like a 2D game. And the state-of-the-art play was DeepMind and OpenAI releasing algorithms which can essentially solve the, the game, play it better than any human, getting tens of thousands of points. Why were we doing that? Because the theme for research over the next few Mondays is to explore reinforcement learning. Why? Well, a couple weeks ago we entered a competition for OpenAI. Uh, we didn't do too very well. Our algorithm sort of finished up with Sonic just jumping randomly up and down on the screen. So in light of that, we'd like to do a little bit better on the Pommelman competition. Now what's that? And all this will be linked in the description. The Pommelman competition is a different version of Bomberman, with essentially with more than one player. It's just an 11 by 11 screen, you got characters and they move around and they drop bombs and whatnot. And so our goal is to enter that competition with a reinforcement learning algorithm of sorts, hopefully code it from scratch, and then see how we go. So, that's, that's what we're going to dedicate Mondays for over the next, over the next few weeks. And I mean, will we get anything valuable out of it? It's hard to say, right? Because that's what you got to try with research. You're going to spend a little time exploring the unknown. You never know what you might stumble across. And so, what have I been doing specifically? Took my jumper off, it's getting hot. So specifically, I've been going through the GitHub repo of the Pomerman competition and essentially just been going through the code, looking through each file, looking through some sample Python scripts and writing them out so I, so I gather an understanding. And is it the best way? Well, I'm not entirely sure, but it's working for me, right? Because what I'm trying to do is get my head around the problem, go through it in a step-by-step fashion, because this morning I really had no idea. So if we're going to implement our own version of a reinforcement learning agent, we've definitely got to have a fundamental understanding of the problem. And so for the rest of the afternoon, I think I'm going to be trying to, to get some OpenAI code running, or OpenAI— an OpenAI algorithm called PPO, which is proximal— I should really know what the acronym stands for, but we'll figure that out.
What?
Can you guess where we are? Back in that tunnel we were in this morning. Why do you think that is? Well, I've got a little metaphor on what happened for the rest of the day. So Xavier and I were going back through some OpenAI gym examples, getting some code running there because we decided to take two steps back before taking a step forward. We want to really understand the PPO, which is Proximal Proximity Optimization algorithm. I believe that's a mouthful. Before we start implementing it for Pomeranian, I think that's where our starting point is going to be. So next Monday we're gonna, we're gonna go back. I've printed out the paper for the PPO algorithm. I'm gonna go through that. I'm gonna try and get some, some code from the OpenAI. I believe they have a baseline implementation on GitHub. So I'm gonna try get that running next Monday. But for the next 2 days, I'm at a client's, I'm on site, and I don't think I can bring you along because of privacy and whatnot. But whatever I learn, Hey man, what's cracking? How was your day, little bro? Good, good.
My first day on my new course in TAFE.
What is it?
Uh, 3N, um, Screen and Media.
Screen and Media, my boy! You can help me out with my videos, make them a bit Yeah, all I did was OH&S this first lesson, so I was a little bit bored.
Safety, safety first every time. Actually, it should be safety third. Take some risks, you know. Yeah. You remember my catchphrase from last night?
Yes.
What is it?
Keep learning.
Keep learning, team. Signing off. It's been a good day. Lots of learning. I got, I got plenty more to go through. Probably gonna go home and write down what I need to cover over the next 2 days. Peace out. Later.
