Microsoft and Nvidia partner up on speech recognition model training

Microsoft and Nvidia have announced a new collaboration focusing on the training of artificial intelligence (AI)-powered natural language processing (NLP) models, Venture Beat reports.

Specifically, the companies said they trained the Megatron-Turing Natural Language Generation (MT-NLP) system, which can perform various speech recognition-related tasks, including reading comprehension, common sense reasoning, and natural language inferences.

Building on the firms’ Turing NLG 17B and Megatron-LM models, MT-NLP reportedly contains 530 billion parameters and can achieve ‘unmatched’ accuracy levels.

Nvidia’s senior director of product management and marketing for accelerated computing, Paresh Kharya, and group program manager for the Microsoft Turing team, Ali Alvi recently wrote a blog post on the company’s website, claiming the new technology will shape the future of NLP.

“The journey is long and far from complete, but we are excited by what is possible and what lies ahead,” the technology experts wrote.

From a technological standpoint, MT-NLP was trained using a dataset with 270 billion tokens from English-language websites, most of which came from The Pile’s 850GB collection.

According to the blog post, model training was done using the Nvidia DGX SuperPOD-based Selene supercomputer powered by 560 DGX A100 servers networked with HDR InfiniBand in a full fat tree configuration. For context, each DGX A100 has, in turn, eight Nvidia A100 80GB Tensor Core GPUs, fully connected to each other by NVLink and NVSwitch.

While MT-NLP is arguably the largest and most capable AI-powered language model to date, however, Microsoft and Nvidia confirmed the system “pick[ed] up stereotypes and biases from the data on which it [was] trained.”

Biases in AI and biometric recognition are a known problem in the industry, but the companies said they are currently working to address the problem.

“We encourage continued research to help in quantifying the bias of the model,” they confirmed in the blog post.

Nvidia has been working steadily on NLP and voice biometrics in the last few years. In December 2020, the company announced a partnership with Veritone, and last August, Nvidia spoke with Biometric Update about the potential of voice biometrics for financial services applications.

Article Topics

AI  |  biometrics  |  Microsoft  |  natural language processing  |  Nvidia  |  speech recognition