Kuzman Ganchev
I was born in Sofia, Bulgaria where I lived until February 1989. My family moved to Zimbabwe and then in 1995 to New Zealand where I went to high school. I came to the US in 1999 to study at Swarthmore College. I spent the 2001-2002 academic year studying abroad in Paris. After graduating with a Bachelor of Arts in Computer Science in 2003 I worked at StreamSage Inc. in Washington DC until starting at the University of Pennsylvania in Fall 2004. During the summer of 2007 I was an intern at TrialPay in Mountain View, CA and during the summer of 2008 I was an intern at Bank of America in New York. I graduated from UPenn in 2010 and have since been working at Google Inc. in New York.
Research Areas
Authored Publications
Google Publications
Other Publications
Sort By
Conditional Generation with a Question-Answering Blueprint
Reinald Kim Amplayo
Fantine Huot
Mirella Lapata
Transactions of the Association for Computational Linguistics (2023) (to appear)
Preview abstract
The ability to convey relevant and faithful information is critical for many tasks in conditional generation and yet remains elusive for neural seq-to-seq models whose outputs often reveal hallucinations and fail to correctly cover important details. In this work, we advocate planning as a useful intermediate representation for rendering conditional generation less opaque and more grounded. Our work proposes a new conceptualization of text plans as a sequence of question-answer (QA) pairs. We enhance existing datasets (e.g., for summarization) with a QA blueprint operating as a proxy for both content selection (i.e., what to say) and planning (i.e., in what order). We obtain blueprints automatically by exploiting state-of-the-art question generation technology and convert input-output pairs into input-blueprint-output tuples. We develop Transformer-based models, each varying in how they incorporate the blueprint in the generated output (e.g., as a global plan or iteratively). Evaluation across metrics and datasets demonstrates that blueprint models are more factual than alternatives which do not resort to planning and allow tighter control of the generation output.
View details
Text-Blueprint: An Interactive Platform for Plan-based Conditional Generation
Fantine Huot
Reinald Kim Amplayo
Mirella Lapata
Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations (2023)
Preview abstract
While conditional generation models can now generate natural language well enough to create fluent text, it is still difficult to control the generation process, leading to irrelevant, repetitive, and hallucinated content. Recent work shows that planning can be a useful intermediate step to render conditional generation less opaque and more grounded. We present a web browser-based demonstration for query-focused summarization that uses a sequence of question-answer pairs, as a blueprint plan for guiding text generation (i.e., what to say and in what order). We illustrate how users may interact with the generated text and associated plan visualizations, e.g., by editing and modifying the blueprint in order to improve or control the generated output.
View details
Preview abstract
The paper presents an approach to semantic grounding of language models (LMs) that conceptualizes the LM as a conditional model generating text given a desired semantic message. It embeds the LM in an auto-encoder by feeding its output to a semantic parser whose output is in the same representation domain as the input message.
Compared to a baseline that generates text using greedy search, we demonstrate two techniques that improve the fluency and semantic accuracy of the generated text: The first technique samples multiple candidate text sequences from which the semantic parser chooses. The second trains the language model while keeping the semantic parser frozen to improve the semantic accuracy of the auto-encoder.
We carry out experiments on the English WebNLG 3.0 data set, using BLEU to measure the fluency of generated text and standard parsing metrics to measure semantic accuracy. We show that our proposed approaches significantly improve on the greedy search baseline. Human evaluation corroborates the results of the automatic evaluation experiments.
View details
Attributed Question Answering: Evaluation and Modeling for Attributed Large Language Models
Pat Verga
Jianmo Ni
arXiv (2022)
Preview abstract
Large language models (LLMs) have shown impressive results across a variety of tasks while requiring little or no direct supervision. Further, there is mounting evidence that LLMs may have potential in information-seeking scenarios. We believe the ability of an LLM to attribute the text that it generates is likely to be crucial for both system developers and users in this setting. We propose and study Attributed QA as a key first step in the development of attributed LLMs. We develop a reproducable evaluation framework for the task, using human annotations as a gold standard and a correlated automatic metric that we show is suitable for development settings. We describe and benchmark a broad set of architectures for the task. Our contributions give some concrete answers to two key questions (How to measure attribution?, and How well do current state-of-the-art methods perform on attribution?), and give some hints as to how to address a third key question (How to build LLMs with attribution?).
View details
QAmeleon: Multilingual QA with Only 5 Examples
Fantine Huot
Sebastian Ruder
Mirella Lapata
Arxiv (2022)
Preview abstract
The availability of large, high-quality datasets has been one of the main drivers of recent progress in question answering (QA). Such annotated datasets however are difficult and costly to collect, and rarely exist in languages other than English, rendering QA technology inaccessible to underrepresented languages. An alternative to building large monolingual training datasets is to leverage pre-trained language models (PLMs) under a few-shot learning setting. Our approach, QAmeleon, uses a PLM to automatically generate multilingual data upon which QA models are trained, thus avoiding costly annotation. Prompt tuning the PLM for data synthesis with only five examples per language delivers accuracy superior to translation-based baselines, bridges nearly 60% of the gap between an English-only baseline and a fully supervised upper bound trained on almost 50,000 hand labeled examples, and always leads to substantial improvements compared to fine-tuning a QA model directly on labeled examples in low resource settings. Experiments on the TyDiQA-GoldP and MLQA benchmarks show that few-shot prompt tuning for data synthesis scales across languages and is a viable alternative to large-scale annotation.
View details
Preview abstract
A wide variety of neural-network architectures have been proposed for the task of Chinese word segmentation. Surprisingly, we find that a bidirectional LSTM model, when combined with standard deep learning techniques and best practices, can achieve better accuracy on many of the popular datasets as compared to models based on more complex neuralnetwork architectures. Furthermore, our error analysis shows that out-of-vocabulary words remain challenging for neural-network models, and many of the remaining errors are unlikely to be fixed through architecture changes. Instead, more effort should be made on exploring resources for further improvement.
View details
Globally Normalized Transition-Based Neural Networks
Association for Computational Linguistics (2016)
Preview abstract
We introduce a globally normalized transition-based neural network
model that achieves state-of-the-art part-of-speech tagging,
dependency parsing and sentence compression results. Our model is a
simple feed-forward neural network that operates on a task-specific
transition system, yet achieves comparable or better accuracies than
recurrent models.
We discuss the importance of global as opposed to local normalization:
a key insight is that the label bias problem implies that
globally
normalized models can be strictly more expressive
than locally normalized models.
View details
Efficient Inference and Structured Learning for Semantic Role Labeling
Oscar Täckström
Transactions of the Association for Computational Linguistics, vol. 3 (2015), pp. 29-41
Preview abstract
We present a dynamic programming algorithm for efficient constrained inference in semantic role labeling. The algorithm tractably captures a majority of the structural constraints examined by prior work in this area, which has resorted to either approximate methods or off-the-shelf integer linear programming solvers. In addition, it allows training a globally-normalized log-linear model with respect to constrained conditional likelihood. We show that the dynamic program is several times faster than an off-the-shelf integer linear programming solver, while reaching the same solution. Furthermore, we show that our structured model results in significant improvements over its local counterpart, achieving state-of-the-art results on both PropBank- and FrameNet-annotated corpora.
View details
Semantic Role Labeling with Neural Network Factors
Oscar Täckström
Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP '15), Association for Computational Linguistics
Preview abstract
We present a new method for semantic role labeling in which arguments and semantic roles are jointly embedded in a shared vector space for a given predicate. These embeddings belong to a neural network, whose output represents the potential functions of a graphical model designed for the SRL task. We consider both local and structured learning methods and obtain strong results on standard PropBank and FrameNet corpora with a straightforward product-of-experts model. We further show how the model can learn jointly from PropBank and FrameNet annotations to obtain additional improvements on the smaller FrameNet dataset.
View details
Semantic Frame Identification with Distributed Word Representations
Karl Moritz Hermann
Jason Weston
Proceedings of the 52th Annual Meeting of the Association for Computational Linguistics (2014)
Preview abstract
We present a novel technique for semantic frame
identification using distributed representations of
predicates and their syntactic context; this technique leverages automatic syntactic parses and a generic set of word embeddings. Given labeled data annotated with frame-semantic parses, we learn a model that projects the set of word representations for the syntactic context around a predicate to a low dimensional representation. The latter is used for semantic frame identification; with a standard argument identification method inspired by prior work,
we achieve state-of-the-art results on FrameNet-style frame-semantic analysis. Additionally, we report strong results on PropBank-style semantic role labeling in comparison to prior work.
View details
Preview abstract
Entity type tagging is the task of assigning category labels to each mention of an entity in a document. While standard systems focus on a small set of types, recent work (Ling and Weld, 2012) suggests that using a large fine-grained label set can lead to dramatic improvements in downstream tasks. In the absence of labeled training data, existing fine-grained tagging systems obtain examples automatically, using resolved entities and their types extracted from a knowledge base. However, since the appropriate type often depends on context (e.g. Washington could be tagged either as city or government), this procedure can result in spurious labels, leading to poorer generalization. We propose the task of context-dependent fine type tagging, where the set of acceptable labels for a mention is restricted to only those deducible from the local context (e.g. sentence or document). We introduce new resources for this task: 11,304 mentions annotated with their context-dependent fine types, and we provide baseline experimental results on this data.
View details
Cross-Lingual Discriminative Learning of Sequence Models with Posterior Regularization
Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing
Preview abstract
We present a framework for cross-lingual transfer of sequence information from a resource-rich source language to a resource-impoverished target language that incorporates soft constraints via posterior regularization. To this end, we use automatically word aligned bitext between the source and target language pair, and learn a discriminative conditional random field model on the target side. Our posterior regularization constraints are derived from simple intuitions about the task at hand and from cross-lingual alignment information. We show improvements over strong baselines for two tasks: part-of-speech tagging and named-entity segmentation.
View details
Universal Dependency Annotation for Multilingual Parsing
Preview
Ryan McDonald
Joakim Nivre
Yoav Goldberg
Yvonne Quirmbach-Brundage
Keith Hall
Oscar Tackstrom
Claudia Bedini
Nuria Bertomeu Castello
Jungmee Lee
Association for Computational Linguistics, Association for Computational Linguistics (2013)
Using Search-Logs to Improve Query Tagging
Preview
Keith B. Hall
Ryan McDonald
Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics, Volume 2: Short Papers (ACL '12) (2012)
Posterior Sparsity in Dependency Grammar Induction
Jennifer Gillenwater
Joao Graca
Ben Taskar
Journal of Machine Learning Research, vol. 12 (2011), pp. 455-490
Preview abstract
A strong inductive bias is essential in unsupervised grammar induction. In this paper, we explore a particular sparsity bias in dependency grammars that encourages a small number of unique dependency types. We use part-of-speech (POS) tags to group dependencies by parent-child types and investigate sparsity-inducing penalties on the posterior distributions of parent-child POS tag pairs in the posterior regularization (PR) framework of Graça et al. (2007). In experiments with 12 different languages, we achieve significant gains in directed attachment accuracy over the standard expectation maximization (EM) baseline, with an average accuracy improvement of 6.5%, outperforming EM by at least 1% for 9 out of 12 languages. Furthermore, the new method outperforms models based on standard Bayesian sparsity-inducing parameter priors with an average improvement of 5% and positive gains of at least 1% for 9 out of 12 languages. On English text in particular, we show that our approach improves performance over other state-of-the-art techniques.
View details
Controlling Complexity in Part-of-Speech Induction
Joao Graca
Luisa Coheur
Ben Taskar
Journal of Artificial Intelligence Research (JAIR), vol. 41 (2011), pp. 527-551
Preview abstract
We consider the problem of fully unsupervised learning of grammatical (part-of-speech) categories from unlabeled text. The standard maximum-likelihood hidden Markov model for this task performs poorly, because of its weak inductive bias and large model capacity. We address this problem by refining the model and modifying the learning objective to control its capacity via para- metric and non-parametric constraints. Our approach enforces word-category association sparsity, adds morphological and orthographic features, and eliminates hard-to-estimate parameters for rare words. We develop an efficient learning algorithm that is not much more computationally intensive than standard training. We also provide an open-source implementation of the algorithm. Our experiments on five diverse languages (Bulgarian, Danish, English, Portuguese, Spanish) achieve significant improvements compared with previous methods for the same task.
View details
Sparsity in Dependency Grammar Induction
Preview
Jennifer Gillenwater
João Graça
Ben Taskar
48th Annual Meeting of the Association for Computational Linguistics (ACL 2010)
Posterior vs. Parameter Sparsity in Latent Variable Models
Joao Graca
Ben Taskar
Advances in Neural Information Processing Systems 22 (2009), pp. 664-672
Preview abstract
In this paper we explore the problem of biasing unsupervised models to favor sparsity. We extend the posterior regularization framework [8] to encourage the model to achieve posterior sparsity on the unlabeled training data. We apply this new method to learn first-order HMMs for unsupervised part-of-speech (POS) tagging, and show that HMMs learned this way consistently and significantly out-performs both EM-trained HMMs, and HMMs with a sparsity-inducing Dirichlet prior trained by variational EM. We evaluate these HMMs on three languages — English, Bulgarian and Portuguese — under four conditions. We find that our method always improves performance with respect to both baselines, while variational Bayes actually degrades performance in most cases. We increase accuracy with respect to EM by 2.5%-8.7% absolute and we see improvements even in a semisupervised condition where a limited dictionary is provided.
View details
Frustratingly Hard Domain Adaptation for Dependency Parsing
Preview
Mark Dredze
João V. Graça
Proceedings of the CoNLL Shared Task Session of EMNLP-CoNLL 2007, pp. 1051-1055
Posterior Regularization for Structured Latent Variable Models
Joao Graca
Jennifer Gillenwater
Ben Taskar
Journal of Machine Learning Research, vol. 11 (2010), pp. 2001-2049
Learning Tractable Word Alignment Models with Complex Constraints
Dependency Grammar Induction via Bitext Projection Constraints
Jennifer Gillenwater
Ben Taskar
47th Annual Meeting of the Association for Computational Linguistics (ACL), Association for Computational Linguistics (2009), pp. 369-377
Small Statistical Models by Random Feature Mixing
Mark Dredze
Proceedings of the ACL-2008 Workshop on Mobile Language Processing, Association for Computational Linguistics, pp. 19-20
Multi-View Learning over Structured and Non-Identical Outputs
Joao Graca
Ben Taskar
Proceedings of the Twenty-Fourth Conference on Uncertainty in Artificial Intelligence (UAI), AUAI Press (2008), pp. 204-211
Expectation Maximization and Posterior Constraints
Joao Graca
Ben Taskar
Advances in Neural Information Processing Systems 20, MIT Press, Cambridge, MA (2008), pp. 569-576
Better Alignments = Better Translations?
João Graça
Ben Taskar
Proceedings of ACL-08: HLT, Association for Computational Linguistics, Columbus, Ohio (2008), pp. 986-993
Transductive structured classification through constrained min-cuts
Proceedings of the Second Workshop on TextGraphs: Graph-Based Algorithms for Natural Language Processing, Association for Computational Linguistics (2007), pp. 37-44
Empirical Price Modeling for Sponsored Search.
Ryan Gabbard
Alex Kulesza
Qian Liu
Jinsong Tan
Michael Kearns
Third International Workshop on Internet and Network Economics (WINE), Springer (2007), pp. 541-548
Automatic Code Assignment to Medical Text
Koby Crammer
Mark Dredze
Partha Pratim Talukdar
Steven Carroll
Biological, translational, and clinical language processing, Association for Computational Linguistics (2007), pp. 129-136
Penn/UMass/CHOP Biocreative II systems
Koby Crammer
Gideon Mann
Kedar Bellare
Andrew McCallum
Steven Carroll
Yang Jin
Peter White
Proceedings of the Second BioCreative Challenge Evaluation Workshop (2007), pp. 119-124
Semi-Automated Named Entity Annotation
Mark Mandel
Steven Carroll
Peter White
Proceedings of the Linguistic Annotation Workshop, Association for Computational Linguistics (2007), pp. 53-56
Nswap: a network swapping module for Linux clusters.
Tia Newhall
Sean Finney
Michael Spiegel
Proceedings of the 13th International Conference on Parallel and Distributed Computing (Euro-Par'03), Springer (2003)