I know that Facebook's BERT model is hugely useful in various AI tasks but I didn't know that they were behind pytorch.

Damn. Pytorch kind of matters a lot in AI. Like, a lot. All of the open source AI stuff I've been tinkering with uses pytorch.