theano sparse_block_dot

sparse_block_dot is a special function in Theano. 1. Function for b in rang...

Noise Contrastive Estimation

one sample: $x_i \to [y_i^0,\cdots,y_{i}^{k}]$ where $$y_i^0$$ are true l...

Perplexity Vs Cross-entropy

Perplexity: Evaluating a Language Model We have a serial of $$m$$ sentences: ...

Introduction to LSTM

It makes that short-term memory last for a long time. 1. Three Types of Ga...