[ad_1]
LDA Explained with a Dog Pedigree Model
Machine learning algorithms are now so accessible that even my non-technical wife constantly asks: “Isn’t that what ChatGPT is capable of?”
The time has come for data scientists to remain vigilant on the whys and hows behind machine learning algorithms.
This 2-part blog post is an actual journey where I have attempted to explain to my wife how Latent Dirichlet Allocation (LDA, a staple in all data scientists’ arsenal for topic modelling, recommendation and more) works with the help of a dog pedigree model. By the end of the series, you should be able to answer the following:
Part 1:
- How does LDA work?
- How to explain LDA to a non-technical person?
Part 2:
- How does LDA converge?
- When to use LDA & when not to?
- What are the alternatives & variants to LDAs (excluding LLMs)?
Let’s get started.
Imagine you have the best job in the world:
Estimate the mix of pedigree of a bunch of adorable dog photos
Easy enough!
Short legs = Corgi or Dachshund;
Long body = Dachshund;
Chocolate chip muffin face = Chihuahua.
But each dog has a unique blend of traits. A dog might have a Corgi’s short legs but the face of a Chihuahua. We are not just identifying breeds but modelling a mosaic of traits into groups of breeds.
Number of Topics & Corpus
Even though we are not classifying dog photos for their breed, it is helpful to consider the physical traits we can observe from all images and roughly how…
Source link
