Detectron Q&A: The origins, evolution, and future of our pioneering computer vision library
Meta AI’s Detectron project has been awarded the PAMI Mark Everingham Prize for contributions to the computer vision community. Detectron is a collection of state-of-the-art algorithms for tasks such as object detection and segmentation. The code has been entirely rewritten multiple times over the years, and the set of algorithms it supports has evolved enormously.
Improving Language Model Behavior by Training on a Curated Dataset
We’ve found we can improve language model behavior with respect to specific behavioral values by fine-tuning on a curated dataset of <100 examples of those values. We also found that this process becomes more effective as models get larger. While the technique is still nascent, we’re looking for OpenAI API users who would like to try it out and are excited to find ways to use these and other techniques in production use cases.
Q&A with machine translation pioneer: The future of MT is multilingual
Philipp Koehn, a Meta AI research scientist, is one of the inventors of the modern method of phrase-based MT. He talks about the latest advances in machine translation (MT) and promising directions on the path toward universal translation. Multilingual models translate multiple language pairs in a single model and are a key evolution because they generalize knowledge across many language pairs, particularly helpful for low-resource languages.
Customizing GPT-3 for Your Application
Developers can now fine-tune GPT-3 on their own data, creating a custom version tailored to their application. Customizing makes Gpt-3 reliable for a wider variety of use cases and makes running the model cheaper and faster. With fine-tuning, one API customer was able to increase correct output from 83% to 95%. By adding new data from their product each week, another reduced error rates by 50%.
Introducing Triton: Open-Source GPU Programming for Neural Networks
Triton is an open-source Python-like programming language that enables researchers with no CUDA experience to write highly efficient CUDA code. It can be used to write FP16 matrix multiplication kernels that match the performance of cuBLAS in under 25 lines of code. Researchers have already used it to produce kernels that are up to 2x more efficient than equivalent Torch implementations.
Introducing Text and Code Embeddings in the OpenAI API
New endpoint uses neural network models, which are descendants of GPT-3, to map text and code to a vector representation. Embeddings are useful for working with natural language and code, because they can be readily consumed and compared by machine learning models and algorithms like clustering or search. The new endpoint in the OpenAI API provides text and Code embeddings with a few lines of code: /embeddings.
OpenAI Codex
OpenAI Codex is the model that powers GitHub Copilot, which we built and launched in partnership with GitHub a month ago. Codex can interpret simple commands in natural language and execute them on the user’s behalf. It has a memory of 14KB for Python code, compared to GPT-3 which has only 4KB. Codex is most capable in Python, but it is also proficient in over a dozen languages including JavaScript, Go, Perl, PHP, Ruby, Swift and TypeScript.