Under the hood of Hidden Markov Models training
Intro
This post aims to show how Hidden Markov Models (HMMs) are trained using Baum-Welch algorithm. If you want to learn more about Hidden Markov models, I suggest to read some posts:
This post assumes you are familiar with concepts like the transition, emission probabilities, hidden state, observation, forward and backward algorithm.
Goal
Baum-Welch algorithm finds values for HMM parameters that best fit the observed data. For training we need:
- sequence of observations O₁, O₂, …, Oₙ
- sequence of hidden states for these observations S₁, S₂, …, Sₙ
Algorithm tries to…
Calculating and using QR decomposition
Intro
QR decomposition (factorization) is decomposition of a matrix into orthogonal (Q) and upper triangular (R) matrices. QR factorization is used in solving linear least square problems and finding eigenvalues. This post shows how QR decomposition is computed and how to use it to solve practical problems.
Gram-Schmidt Process
QR decomposition has following formula:
A = QR, where:
- A is original matrix we want to decompose
- Q is orthogonal matrix
- R is upper triangular matrix
Main goal is rather simple, decompose matrix into matrices Q and R. To find a orthogonal matrix Q, we could used Gram-Schmidt process. This process takes input matrix…
Simple matrix decomposition
Intro
LU (lower–upper) decomposition (factorization) outputs (factors original matrix into) lower and upper triangular matrix. These matrices could be used to efficiently solve system of non-sparse linear systems or find inverse of a matrix. It might sound a bit difficult, but we’ll have an example later. Goal of this post is to show what LU decomposition is about, how it could be calculated and used solving practical tasks.
Gaussian elimination
In LU decomposition we want to decompose original into upper and lower triangular matrices, so that:
A = LU, where:
- A is original matrix we want to decompose
- L is lower triangular matrix…
Naive ways to calculate SVD
Intro
Aim of this post is to show some simple and educational examples how to calculate singular value decomposition using simple methods. If you are interested in industry strength implementations, you might find this useful.
SVD
Singular value decomposition (SVD) is a matrix factorization method that generalizes the eigendecomposition of a square matrix (n x n) to any matrix (n x m) (source).
If you don’t know what is eigendecomposition or eigenvectors/eigenvalues, you should google it or read this post. This post assumes that you are familiar with these concepts.
SVD is similar to Principal Component Analysis (PCA), but more general. PCA…
Explained by change of basis and transformation matrices.
Eigenvalues have important role in data science. I’ll try to have my try explaining them. Before let’s dive into few topics that are crucial for understanding eigendecomposition. I assume that you are familiar with terms dot product, inverse of a matrix, orthogonal matrix.
Special thanks to Hadrien Jean and his book Essential Math for Data Science which was inspiration and main source of this article.
Code shown in this article is here.
Change of basis
Change of basis means that we go from one basis system to another. Standard basis system (in two dimensions) graphs vector [2, 1] in the following way:
Injecting changes into your models
Intro
While training machine learning model you want to play with it. Use different parameters, architectures, tricks and everything else that might improve performance. How to implement those changes? You have to change source code. But that will make codebase bigger, more difficult to maintain, more error-prone. Is there a better way? Yes! We could use callbacks.
Wikipedia defines callback as a “… any executable code that is passed as an argument to other code”. To my understanding, the main idea of callbacks is to inject code into existing program (functions, classes). …
Notes based on course lessons
“This class was originally taught in-person at the University of San Francisco Data Institute in January-February 2020, for a diverse mix of working professionals from a range of backgrounds (as an evening certificate courses). There are no prerequisites for the course. This course is in no way intended to be exhaustive, but hopefully will provide useful context about how data misuse is impacting society, as well as practice in critical thinking skills and questions to ask.” (source)
Course syllabus could be found here.
This is list of lessons and my notes:
How tech companies develop modern colonialism
Lesson 6 materials are here. These are my notes and thus not complete and may contain mistakes.
Algorithmic colonialism
Tech companies are sometimes developing colonialism. One example is Facebook’s Free Basics (source). It was going to help get people online and provide free basic access to internet. Indian government was against it, which in turn provoked Marc Andreessen to tweet something that he later regretted (and removed). But it clearly shows big tech companies attitude towards people in outside US.
Going faster and higher might create problems for everybody
Lesson 5.2 materials are here. These are my notes and thus not complete and may contain mistakes.
Venture Capital, & Hypergrowth
Venture capital has some negative side-effects. It doesn’t mean that it having venture capital utterly negative thing but it isn’t a magic bullet (source):
- Venture Capital often pushes what could’ve been a successful small business to over-expand and ultimately fail; prevents companies from focusing their priorities; distracts from finding a monetization plan; causes conflict due to the misalignment of incentives between VCs and founders; and is full of far too many unethical bullies and thugs.
- Hypergrowth is nearly impossible to manage and leads…
Risto Hinno
Highly trained monkey
