Coordinating Codeforces Div. 1 Rounds, Competitive Programming in Singapore | Ashley Khoo

Welcome to another episode of the competitive programming podcast. With me today is a very special guest. He's here from Singapore, visiting my hometown of Belgrade, Serbia to participate in the finals of the Midnight Code cup. He's a two time IUI gold medalist and a legendary grandmaster. He's often in the top 10 of contributors on codeforces. It can only be Ashley Kuh or as you better know him, it's Error Gorn. Welcome.

Well, thank you. Thank you having me here.

Yeah, we usually start with the same question every time, which is, how did you even get into this world of competitive programming? How did you start programming?

Well, I think my case is a bit special because so being in Singapore, it's a really small country, there's like basically three good schools that people go to. So I just happen to be in one of these good schools. And in these three good schools there is usually a computing club or in my case, okay, my school is actually like the special one. So in my school there's like actually modules that you can take for the local Informatics Olympian. So it's through those modules that I learned of Informatic of competitive Programming of Informatics Olympian. And actually I think I started cold forces quite late because for a lot of people cold forces is like their first step into competitive programming. Whereas I got a gold in my national Olympia before I even heard of codeforces. So actually if you look at my codeforces rating, like at the start, it was like I immediately went to like do or something. And that was because I started like competitive programming like one year before I actually like joined CodeForces. So.

Yeah, that's cool. So there was, there is. I wanted to get into that as well. So Singapore has a long legacy of competitive programming, I think, and with Professor Halim and the competitive programming books, and I believe it's also core. They are, there are courses in university about competitive programming. So this is how you got into this. It's really helped you, right?

Yeah, it really helped. So even in high school I was allowed to sit in the university module on competitive programming that was taught by Professor Halim. And he was actually really nice and he just gave us like three copies of his books. I actually have the. Have it signed by, by Halim, so.

Oh, that's cool. Yeah. And did you enter that high school with the plan to try these Olympiads or was it just like, oh, I'm already here, might as well try this?

Well, so it was a funny story. I entered the school like thinking, I'll do a math Olympia because Like that's what every like child in primary school. I kind of like, I think every smart child in primary school kind of like dreams of like making it the imo. But I got kicked, like funny story, I got kicked off the IMO team in like my second year of high school. That's like grade seven. No, sorry, grade eight for like most people. So Informatics Olympia was like the other thing that I was like decent at. So I thought why not just try that? And it seems like it was less competitive because I mean, I think in many countries the Math Olympia is really, really competitive, whereas Informatics Olympiad is less competitive because like just lets people know about it. And I think at least in East Asia or like just specifically Singapore, but I think generally in East Asia a lot of people start Math Olympia from like five or six and it's really, really competitive. If you don't start from that age, like it's just over for you.

Yeah, that's, that's insane. That's like for many sports, like if you start after four, there's no chance you'll be world class and yeah, that stuff. But either way you started in high school with competitive programming and you got really good. So how was that path? Like when did you need more than the school provided to, to get better?

I think I, my school helped me get to like my national Olympia and I think like the biggest contribution my school gave me was like introducing me to cold forces and that was where you really get good. And I think being inside competitive programming program, competitive programming servers helped a lot. Especially like trying to be better than like people in the server and like getting better than them. Like you like. It's also like a very good thing to like be in this service. Like you just like make a lot of friends that you meet like in these on site competitions.

Yeah. Have this like healthy rival rivalry. Right?

Yeah. Yeah.

That's cool. And so the first two IOIs you participated were online in, organized by Singapore and the third one was in Budapest. Right. Close to here.

Not Budapest, sorry, Yomyakata in Indonesia. Oh yeah, sorry.

I mixed the two years.

It's really sad because like they're both in like my country. Neighboring country.

Yes. So you did not get the chance to travel far, but did that maybe help to for those first two years to have a more, more familiar environment?

I suppose, yes. I mean, I can't really say much. I haven't like really been like serious in like a competition that was overseas, but I think it really helped a lot like just having the contest Be in Singapore. And especially I think the COVID year made everything a lot more simpler because during the COVID year like schools are shut down. Everything was shut down. So like being like a 16 year old kid who like things like okay, competitive programming is kind of fun. Like I just like spend my entire day doing competitive programming and I think that really helped a lot with improving my competitive programming skills. Like I think I, I suppose like all my competitive programming skills is just attributed to like the training I did during COVID Yeah. I think if you look at my co forces like problem solved graph it's like you can see like when the, when the skills are shut down. Yeah.

I think, I think a lot of people actually told me that it's, it seems it was very good for the competitive programming community even with everything. But yeah. Is this the furthest you had to go for a competition?

Like I mean I've been to ocpc.

But yeah, that's a camp.

Yeah, I guess.

And it's also right next door so that's cool.

Yeah, yeah, yeah. But I guess this year IOI going to be even further the furthest one this year. Yeah. Singapore team used to travel like 40 hours from Singapore to Bolivia.

So yeah, I think it's similar from Serbia.

I thought from Europe would be much.

Closer, maybe like from Germany or something. But we first need to go to like west Europe then to. Yeah, it's, it's a long, a long way there. But yeah. Okay. So you're apart from still being a very active competitor. You only you finished high school but you're just starting university. Right. So you have many more like on site contests.

Not just finished high school. I mean I finished high school school two years ago. This is like military was in between. Yeah. So Singapore has like compulsory military and I unfortunately had to go through that. But yeah, I'm studying university this year. Yeah.

Did you get the chance to practice anytime while you were in the military?

Yeah, so I think I was, I consider myself very lucky because I was selected to be in the air Force and I think in the air Force the culture there is more like it's just a 9 to 5 job. It's not like in the army where it's like from sunrise to sunset, this is just normal 9 to 5 job. So after, after outside office hours you can just do whatever you want. And I was able to like do whatever I wanted in after those office hours. So that included some competitive ramming and I think a lot of my coordination was also done like inside the army barracks.

That's cool. That's cool. So it's important to keep that consistency, I guess because now you'll be going to icpc.

Yeah.

Will you participate this year or are you one of those that wait a bit later?

I guess I'll participate this year. I'll just see how. I mean, I think my age is already quite old. Like I know there's an age limit for icpc, so I think I'm quite old already.

Yeah, I think you have a few years for sure. But yeah. So you are still a very active competitor, but you are very active in other aspects of competitive programming. You are for instance, organizing the OCPC Mirror in Singapore. You're a coordinator on Call Forces. You were a problem setter recently at Asia Pacific Olympiad. So do you have one of those roles that are the favorite for you?

I think the coordination role is probably the favorite. I mean you get, I think you directly get to shape the meta of CF and you get to shape it towards like the problems that you like. And hopefully like other people see that, okay, this is something that, this is something that also like and they also like other people try to like drive the style of problems towards it. I think that's what the person who did at first was like Anton 3 group. Like I really respect him for what he did for competitive programming and I guess I'm just trying to like emulate what he's been doing.

How many active coordinators are there currently on Code Forces?

I'm actually not sure, but I believe it's like I'm actually not sure of the number, but I'm not sure how many of them coordinated one because I think for me I really only care about the different coordinators and right now I'm really not sure how many there are. But yeah.

And you were the one that actually asked Mike to. To become a coordinator, right?

Yeah. So it's a really funny story. Like I asked Monogon how he was how he became a co force coordinator because there was a time in the past where codeforce coordinators were only had to speak Russian because they had to translate statements. And Monocon was the first non Russian speaking coordinator. So I asked him how did it happen? And he said just like message Mike or something. So I just messaged Mike. Yeah, yeah.

And I think Code Forces is definitely for the better because of it and it can be a tough job. I think like it's one of these things where you often get blamed when something's not good. But maybe you don't get the compliments when around is really good. So I think Anton Trigup had to go a lot through that. Maybe people didn't like his philosophy, but I'm with you there. I think it's very cool. So what's like the best and your favorite part of coordinating versus the least favorite?

I mean, the least favorite part is just like, again, the least favorite part is very easy, just like the animal. Because coordinating has a lot of checks you need to do. Like, you need to make sure the statements are readable, the limits are reasonable, like the time limits and everything. And those are like just, I guess the, the boring parts of compare. I mean, of coordinating just because it's like, I guess it's a necessary evil for that. But the best parts of it are just like taking the author's problem and seeing like, what. How you can improve the. How you can improve the problems. And I think a lot of my div ones, I actually helped the authors improve their problems and made them a lot more interesting. And I feel like that's a very rewarding work to do.

Yeah, I mean, anytime. We don't have many Div 1 rounds. So when they are good, it's, it's. I think everybody loves it. And can you maybe share us?

Walk through.

Walk with us through, like what regular preparation for a round looks like from the side of a coordinator.

I mean, from the side of a coordinator, the first thing I do is that I look at their preliminary proposal. But the thing is that I think for, I think for most authors where it's the first time sitting around, usually these preliminary problems are like, not really up to my standards. So like, I reject like a lot of them. And then comes to the phase where the problem authors will like, propose to me additional problems. And this can take like months. And sometimes, like, I think there's a few cases where it took even years for the problem authors to propose a reasonable set to me. Then after that it comes to the testing phase. So. Testing phase. You just get testers from code forces, like volunteers to come and test our problems. And mainly here we are looking at whether the problem. I think mainly here we're looking at the difficulty, how hard each problem is. Because as the, the coordinator and author are only like a few people. So like, the difficulty estimate that we get is really noisy. And I think like getting testers, like the point of the thing, the best, the best use of like, the best use of testers is just to get a better estimate of like a get less noisy estimate of difficulty. And the other. Yeah, and we also like ask testers like look out for scams. So yeah, in the recent round that like we only, unfortunately we only saw like a very like, like a time limit scam like a few days before the contest. So that was very unfortunate. But yeah, the testing phase goes until like I'm happy with the round and after I'm happy the round we scheduled around. Yeah, so that's how like average round of single. But yeah, so yeah.

And so I read a lot through your blog recently and I think it's a very interesting resource for people. There is a lot of different stuff there. Sometimes you talk about learning languages, sometimes it's about food, but usually it's about something very advanced that I can't understand in competitive programming. But one thing that's very consistent is that it's very high quality. Like even some meme posts like let's say how to troll Competitive programmers. I think you probably spent more time at that than many like blogs for teaching people in competitive programming. And like your tutorials are always like very high quality. What drives you to write with such detail?

I mean, I'm not really writing for anyone. I'm honestly surprised anyone actually reads my blog. Like, I only wrote that blog because I realized that there's a lot of times where I've solved something in the past or I've thought about something in the past and I just forgot it in the future. So those blogs are mostly a reference for, for like the future me. And the other reason I started my blog was that I thought it would be a nice place to like post my university notes since I'm going to university soon. So I know I'll probably be like writing a bunch of notes. So I thought it'll be like cool to like post them into like a public, into like a public website. So yeah, the drive is mainly just, it's just for myself.

Well, I think others can find it very useful as well. My favorite one is, I think it's something like notes on practicing for informatics, Olympiads or something like this. And you went through, through a lot of different like aspects of what made you so good at ioi, from sleep to friends to. And I think everybody should read it and we'll link it in the description. But yeah, it's, it's. But through all of these. Like you obviously very much love math, which is clear to see and even learning languages, what was unique about competitive programming so that you dedicated so much time for it.

Oh, so I mean in a lot of the fields, like there's a lot of hard problems. But I think competitive programming is unique in that a lot of the problems like that you solve, you only need like really little knowledge. Like, I think the knowledge that I use to solve problems as an LGM is basically the same as exponent code forces. Like the only thing separating us is like our problem solving skills. And those are just all from first principles. And I think it's just the fact that like you can like the problems are. If you give someone the editorial of a really hard problem, they'll be able to understand it. But it's just like the frustrating thing of like them not being able to solve it even though they have all the required knowledge too. Like, they kind of all have. They kind of had the required knowledge to like in themselves. It's just that they don't have like enough problem solving skills. And I think like comparative programming is probably like the most pure form of like, of like determining how well someone is at this pure problem solving. But I guess math is too. But I didn't go to math because like.

Yeah, and you mentioned previously that you first wanted to go to imo. And do you still practice like competitive math even though you, you maybe didn't know you didn't go to imo, but did you maybe still practice it to get better at ioi?

No, I actually never practice math. So I know for the Math Olympia they split into four categories like ACG and. Right.

Yeah.

So like the reason I got kicked off the Singapore IMO team because, like, I couldn't do G. So like. And the thing is that the funny thing is that the first stage of Singapore IMO training is only G. So yeah, I just got kicked off for that. But yeah, I've done some like, math competitions. Like there's the Iranian Combinatorics and I think like the problems there are like, really, really similar to like competitive Gramming. Actually, like just looking at like the combinatorics problems in like the IM in the IMO shortlist, like some of them really can be competitive programming problems. Like we can take for example the problem 5 from last year's IMO that was like Turbo the snail. And I believe that could be like a codeforce's div1c. Like very easily could just be a codeforce's div1C. Yeah, yeah.

I think there are problems on both sides that could swap. Like my friend that's been on the channel Pavle actually competed when his problem was at the IOI fountain parks and he go problem. Well, you'll get the chance to tell him that soon. But he'll be happy to hear it. But yeah, that was originally also planned to be in IMO or some other local Mathematics Olympiad, so. But yes, it can go the other way as well.

Yeah, I know. Like the song like yo song is that S coder is this Code Forces handle Like he has actually a MO problem set called like IMO Intersection IOI. Yeah, a bunch of nice ROMs.

I should take a look at it probably.

Maybe I can send you the link. There's also like one section like the last section was like Notorious Coincidences. So it's like I believe there was like the notorious coincidence from like imo 2020 where there's like there's code forces div one round code forces problem called like Game of Paris that appeared on the IMO after the Code Forces round was released. I believe it's IMO 2020 I think cold Forces rounds. I forgot already. But it's called Game of Fairs by Jaktor. Yeah.

Yeah, I would like to check that out as well. G was always my worst. So.

It'S a C problem. But yeah, I mean there are always C problems. Yeah, always injections. Always see problems.

Yeah. That maybe I can try. But the other ones I could never deal with. They were not my cup of tea.

So.

Yeah, now you're entering also apart from university, you started the internship recently and it's kind of an AI ML research. Research internship. Am I correct?

Yeah.

So is that something you're just trying or is this something that you really want to focus on in the future? Like this?

I think just something I'm trying because I think the problem with compare programming is that you can't make it a full time job. Like it's really fun as a hobby but like you can't make money out of it. So I'm trying to see like which path I can branch out into that I can kind of like apply like half the knowledge I learned compare programming to that. That I can like that I'll like have fun doing it as a full time job. I'm not really worried about like the financial aspect of like my job. I just want it to be like something that I enjoy doing. Yeah.

Did you already start the internship?

Yeah, I've been doing it for like three weeks at this point. But I don't think I've like done it enough that I can like give you a really good like I can talk about it really well. Yeah. Yeah.

I'll ask you in a few months whether it was. It was good. So maybe you can we can use this opportunity maybe to try to transfer some tips to our maybe people that just started or maybe they're struggling, stuck at some particular level. How would you like they tap you on the shoulder, say, hey, can you give me some tips? What would you say to them?

I mean the most important tips is just like I think the most important two things that I did was just like one, have friends that you have like some like friendly rivalry between. I think that really pushed me. So sometimes like during the IOI training there'll be like contests and I always try to up solve the problem so that like I solved everything that like my friends also solved. And this especially applies, this also helps a lot if you have like a group of friends that you can like discuss hard problems with. So you can like just try to like be better than all your friends. Like I think like if you and your friends like try to be better, like obviously we'll get better each other. And the other thing is just like I think Evan Chen also mentioned this in his like his blog on like on the training for imo. It's just like solve problems is like a bit hard for you. Like don't try to solve too hard problems. Obviously don't try to solve too easy problems, just like solve problems that are like a bit hard for you. Like I remember like during COVID time I was trying to like grind for Grandmaster on code forces and the strategy I did was just like solve like 2400s every day. And yeah, I think that really helped. Like I finally broke the Grandmaster's barrier after like trying that for like a few weeks.

Yeah, a few weeks barrier. You know, I've been trying like for three years.

But no, I mean, I mean like after trying like like grinding to it 2400, like I've been trying for like I don't know how long to like break Grandmaster.

Yeah, yeah, I'm just kidding. Anyway, so, so you, you mostly participated in Olympiad style contests and even now like sometimes you blog about the recent Olympiads, whether it's API or I think was recently the Croatian Olympiad and now you're going into icpc. Are you still liking more the Olympiad styles or do you enjoy now having a team around you?

I mean, I don't really, I have not really experienced like the how the team expect of it. But like definitely like Olympian problems are more interesting than like ICPC problems. Like in icpc a lot of people complain about the problems like being classical, just implementation and I think like you don't have the same problem IOI because I don't really know what's the reason for it, but, like, yeah, a lot of people say, like, a lot of problems in ICPC are annoying. And yeah, I've did some UCUB before. Like, maybe there's like, one or two, like, really nice problems buried within the problem set. But, like, you're forced to, like, grind through, like, seven annoying problems before. Like, you can look at those, like, one or two. Nice. Yeah. So definitely, I think from my experience with ucup, I've had a lot more fun, like doing Olympia Proms. Yeah.

Yeah, that's my experience as well. I think the Olympiad problems can sometimes be maybe deeper, maybe have a few more steps than most ICPC problems. But you're already a coordinator on code forces and you were a problem setter some Olympiads. Can we maybe expect to see you in some, maybe scientific committee of APIO or IOI or something like this in the future?

I don't really think so. I'm not really sure what I want to do. And the thing is that I'm not sure whether I want to continue coordinating. I've. I don't know whether you've noticed recently, like, the number of rounds I've been cointhing has, like, kind of dropped a lot. And I guess this is because, like, like, as you said before, coining just takes a lot of time. And I think that the standards I want for my rounds, like, requires like, a lot of effort to be put in. And so I just don't have, like, the, I guess, a passion and time anymore to like, like, at this time to like, coin it rounds. Maybe, like, my decision will change in the future, but, like, right now I'm just, like, clearing the rounds I currently have. Like, I know there was a round that quanted, like, last month and that was because, like, the round has been, like, with me since, like, 2023. So it's like, just like, been with me in two years and like, I've been like. And the author just suddenly, like, awakened and, like, told me, like, okay, we actually want to finish the round one now.

Okay, so you mentioned that you have a very strict, like, criteria for accepting around, let's say. So can you maybe share when you can say for around, oh, this is good enough, or this is really good. I want to be a coordinator here.

Repeat the question again.

So what is your definition, like, of a nice round, nice problem, and stuff like this?

I mean, for my definition, I just care that the individual problems itself are good, but the community has different sense of the problem because I think one of the kind of proxies for determining how good a round was, like, how good or like how positive or how negative the downforce on the announcement block was. And I feel like that's like, optimizing for how well the round is upvoted or how well the community likes the round is actually kind of like, different from optimizing how much I like the round personally, because I like the round just because the problems are nice. But the thing is that the problems I find nice are like, the really hard problems that are like, tucked away from most of the participants and like, the general opinions around does not depend on, like, the problems that I really care about. So the other things that I. That I think generally we should care about for around, like, the one important thing that people, a lot of people care about is the difficulty curve. And that's where testers come into play because I can't really tell like, about difficulty curve when it's only made out of. So the testers need to like, virtual contest around, like, kind of give me a gauge of the difficulty curve. And I guess, like, the other important thing is like, make sure, like, there's no annoying time limits. I think we messed that up quite badly in the previous round. So, like, that. That was kind of fortunate. But yeah.

And what is a nice problem? I already asked Rico Kawasaki that this. And I'm interested to. To hear from you as well.

Oh, this is a very hard definition. I think it's like. It's like it's asking a mathematician, like, what makes a proof beautiful. Like, I don't think they can give, like, a definite answer for it, but I think it's just like a combination of like, the problem being unexpected and does not require, like, prerequisite knowledge. But yeah, it's still. There's still like a very weird definition, I guess.

Yeah, it's similar to what Riku said. I think he. He basically said that he must not have seen anything like him before, which I guess is similar to. Does not require, like, knowledge from before. But yeah, it's. It's. It seems like the coordinators and admins on ATCoder have all went into that direction, and I think it's a good direction for competitive programming to go. I think maybe it would be fun to show people more how hard. Maybe hard is not the right word, but I don't know. I've had experience of when I see people just complaining, I see that they don't know, like, what coordination even is or like, how much work goes into it. So I'm not sure if maybe we can represent that in some way.

Yeah, that's like. And like the other thing, like the other annoying part of coordination is that like I guess the ChatGPT thing but I guess like Corvals like doesn't care about that anymore. But like there was a point in time where like ChatGPT was like newly just getting really good convex and like we had to like optimize for the problems not being able to be solved by like ChatGPT. That was really annoying.

And what do you mean that codeforces doesn't care about GPT anymore?

I mean not say not care about it anymore, but there was a point in time where like we tried to, I think the Code Force team tried to like they like Khan requested us that we should prefer problems that's not solved by ChatGPT. And yeah, it was kind of hard especially for like Div2As and Div2B's there's a point in time where some Diff2A's and some Diff2B's can be solved and some can't be solved. And it's quite sad when you have a really nice div 2a but you can't use it just because chatgpt can solve it. But I guess now we've reached a point where ChatGPT can just solve all of them. So we can just not factor that into our decision when we're choosing the problems.

Yeah, that's fair enough. And there have been a lot of like recommendations on call forces from users and stuff like this. And do you have any like thoughts about where competitive programming we will need to go to deal with this, whether to just ignore it or.

Yeah, I think as an online format it's very hard for me to say because I think it's quite inevitable in the figures time where AI might be better in computer programming than a lot of us. And honestly I think it's like quite sad for like compare programmers because when I joined this sport like a few years ago like I thought okay, this seems like the final barrier of like human, of like human creativity. Like okay, maybe chess and go are like trivial to like trivial. So like that's not very creative. And I thought like okay, since this, this seems like where like all the really smart people like come to like challenge themselves. This like it really seems like the final frontier of human like. But I guess over the like recently we've just seen that compare is like getting trivialized by ChatGPT. So I guess it's not like as advanced as we thought.

And do you think like even the upper levels, like three, like 3,500 tasks will eventually be solved by AI?

Yeah, definitely. I think at this point right now I feel like it's kind of inevitable, unfortunately. Yeah.

And what do you think, like how that will change competitive programming as a whole?

Yeah, I'm not really sure about that. But I think one idea that might be really interesting is like can we get AI to like make competitive ramming contests? Because if it's better than us, like why not ask it to like make contests for us?

That's interesting. Yeah, I mean I think that's a, that's an interesting because chess has been dealing with this for like 25 years.

But I think the difference between chess is like a lot of companies that want to hire smart, let's say like training companies, they want to hire people who did go in competitive programming. Right. But we know chess. I mean chess like to most people also seem like a really smart like activity like only really smart people do. But it's not like training companies like actively hire people from chess.

Right.

But they're actively like, they're actively like sponsoring ioi, icbc, like trying to poach people from like combat programming and they don't know how like. Well like the rise of lms like delegitimized the credentials of these people. Yeah.

Especially on like before master or maybe even now master level with the current models. But maybe, maybe we'll need to have more like on site things like ocpc. But yeah, it's a difficult problem and so that's where we see like compared to programming in five years. Where do you see yourself in, in five years?

Five years? I. I don't know. It's really hard for me. I mean. Yeah, I mean like I've just started university so like. And then like all the doors are open for me right now. Like I, I plan to do like do my internships like different places so I can like see like which place best suits me. So yeah, it's really hard to like tell like where I see myself in five years.

Yeah, you're now at the position where you branch out and.

Yeah. Yeah.

You're not sure where to go. Yeah, I mean good luck with that. I'm sure you'll find something that's as interesting as competitive programming and maybe in the future you'll be able to be like full time competitive programmer. Who knows?

I mean that'll be nice.

But yeah, we'll see if that happens. Okay. We went through my questions very quickly, but I think it has been a very productive conversation. So thank you for coming and good luck on the contest tomorrow. We are participating against you, but still, I think it will be fun. So good luck with that and thanks for coming.

Thank you for having me. It's a pleasure to be here. Ok.