@Klaus said in Hey Moonbat!:
@Moonbat said in Hey Moonbat!:
Still working on machine learning problems
Give me all the details!
My impression is that pure statistical machine learning is running into an intellectual and practical dead end.
Just today I read a fascinating article of why deep learning doesn't suffer more from overfitting problems, although those deep neural network architectures have gazillions of parameters (see here).
I actually work more with the kernel machines that article talks about, or at least their Bayesian variants - Gaussian Processes. Though inevitably we also do some deep learning. Most of my time has been spent on Bayesian optimisation in various different settings hence my familiarity with GPs as they tend to be the model of choice if you want high quality uncertainties and you relatively little data.
Deep learning seems to be getting more expensive, which perhaps is the practical dead end you speak of but i'm not sure I would mark it dead yet. I think the surprising thing for me is that the most powerful models have pretty simple architectures - e.g. the transformers and quantised auto encoders that drive things like GPT3 or Dale-E.