Adrian Reid explains his one-on-one client workflow for guiding people through an "experts guide to back testing" process. He steps clients through each stage—understanding the concept, running the step, reviewing the code, and evaluating results—and warns that improper evaluation (overfitting to best parameter sets) often causes strategies to fail in live trading.
When I work with clients one on one, my approach is to step them through the experts guide to back testing process. And I check every single stage. Now obviously with, with a group it's, it's pretty hard for me to do that. But what I like to do is make sure that the understanding is there on the concept why, what is this step about? What does it do, what's it, what's it help us with? Then how do we actually run it? And then the code, do I actually understand it? And then the analysis, am I evaluating the results correctly? And it's often the evaluation which gets people trapped or tripped up because when you first do an optimization it's like oh yeah, that one's the best one. Then you run the next parameters like oh yeah, that one's the best one. And you go yep, yep, yep, yep, yep, cool, this is easy. And then you sort start losing money in real time. It's like what the hell went wrong? And it's because it's a little more subtle than picking the best parameter set.
