Salesforce Research

Double Hard-Debias: Tailoring Word Embeddings for Gender Bias Mitigation

Word embeddings inherit strong gender bias in data which can be further amplified by downstream models. We propose to purify word embeddings against corpus regularities such as word frequency prior to inferring and removing the gender subspace, which significantly improves the debiasing performance.

(Re)Discovering Protein Structure and Function Through Language Modeling

In our study, we show how a language model, trained simply to predict a masked (hidden) amino acid in a protein sequence, recovers high-level structural and functional properties of proteins.

Prototypical Contrastive Learning: Pushing the Frontiers of Unsupervised Learning

Prototypical Contrastive Learning unifies clustering and contrastive self-supervised learning to push the frontiers of unsupervised learning.