Oh this is brilliant, I've spent the last month doing something just like this. As a challenge, no libraries allowed besides Python standard libs (so no numpy).

Started with Word2Vec, built an RNN, then LSTM and am halfway through building transformer architecture.