EM Training of Phrase Based Models

While the dominant method to obtain a phrase table is based on a given word alignment, we may also define a probabilistic model that explains the parallel corpus and then train this model directly with the expectation maximization (EM) algorithm.

Phrase Based Model EM is the main subject of 10 publications. 9 are discussed here.

Publications

The popular joint phrase model was proposed by Marcu and Wong (2002). The joint model may be improved by constraining it with alignment points from the intersection of IBM Model alignments (Birch et al., 2006; Birch et al., 2006b) or by not strictly requiring a unique phrase alignment (Moore and Quirk, 2007). DeNero et al. (2006) point to some problems when using EM training with conditional probabilities. Cherry and Lin (2007) show that the ITG constraint helps the joint phrase model approach, partly by enabling a faster algorithm with less search errors. The phrase alignment problem is NP-complete (DeNero and Klein, 2008). Wuebker et al. (2010) use leave-one-out to overcome the problem of over-fitting when re-aligning the training data with a model that was obtained from it. They use the obtained alignment to re-estimate translation probabilities (similar to one iteration of EM). Using only the best derivation from the forced alignment drastically reduces the size of the phrase table but hurts performance (Sanchis-Trilles et al., 2011).

Benchmarks

Discussion

Related Topics

New Publications

• Mylonakis, Markos and Sima'an, Khalil (2008): Phrase Translation Probabilities with ITG Priors and Smoothing as Learning Objective, Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing mentioned in Phrase Based Model EM and Smoothing

  DELETEIntroduction   ResearchGroups   AvailableSoftware   SpeechTranslation   SignLanguage   ComputerAidedTranslation   OtherApproaches   Deployment Data   ParallelCorpora   ComparableCorpora   Dictionaries   CorpusCleaning   SentenceAlignment   Truecasing   WordSegmentation   SpellingCorrection   SparseData   PivotLanguages   DomainAdaptation WordBasedModels   IBMModels   Symmetrization   WordAlignment     WordAlignmentBasedOnCoocurrence     WordAlignmentWithLinguisticAnnotation     WordAlignmentForSyntaxBasedModels     DiscriminativeWordAlignment     WordAlignmentCombination     WordAlignmentEvaluation     AlignmentOfSubsententialUnits   DictionariesFromComparableCorpora   LexicalChoice   MTWithoutWordAlignment PhraseBasedModels   PhraseBasedModelTraining   PhraseBasedModelEM   PhraseBasedModelRefinement   ContextFeatures   Smoothing   Paraphrasing   PruningLargeModels   SuffixArrays   ReorderingModels   PhraseBasedVsEBMT SyntaxBasedModels   InversionTransductionGrammars   HierarchicalModels   StringToTree   TreeToString   TreeToTree   BinarizingSCFG   GenerativeSyntaxModels   DependencyStructure   DeepSyntax   ParseQuality   ParallelTreebanks   SyntacticCoherence LinguisticProblems   UnknownWords   Morphology   GeneratingRichMorphology   FactoredTranslationModels   Compounds   TranslatingTenseCaseMarkers   TranslatingChunksAndClauses   DocumentLevelAspects   Transliteration     TransliterationWithFSM     TransliterationWithOtherMethods     ForwardTransliteration     TransliterationTrainingData     IntegratingTransliteration   SyntacticPrereordering   POSChunkPrereordering   SyntacticReranking   IntegratingSyntacticFeatures LanguageModels   NGramLanguageModels   TargetedLanguageModels   MorphologicalLanguageModels   VeryLargeLanguageModels Decoding   StackDecoding   ReorderingConstraints   WordBasedDecoding   FiniteStateMachines   DecodingComplexity   SyntaxModelDecoding Evaluation   EvaluationCampaigns   ManualMetrics   TaskBasedMetrics   EditRateMetrics   NGramMetrics   SyntaxBasedMetrics   CorrelationOfMetrics   TrainedMetrics   TranslationDifficulty   StatisticalSignificance MachineLearning   WordGraphs   ParameterTuning   Reranking   LargeScaleDiscriminativeTraining   MinimumBayesRisk   ConfidenceMeasures   SystemCombination   IncrementalUpdating NeuralNetworkModels   NeuralLanguageModels   AttentionModel   Training   Inference   Coverage   Vocabulary   Embeddings   MultilingualWordEmbeddings   MonolingualData   Adaptation   LinguisticAnnotation   MultilingualMultimodalMultitask   AlternativeArchitectures   AnalysisAndVisualization   NeuralComponentsInStatisticalMachineTranslation add

  

  @InProceedings{mylonakis-simaan:2008:EMNLP,
  author = {Mylonakis, Markos and Sima'an, Khalil},
  title = {Phrase Translation Probabilities with {ITG} Priors and Smoothing as Learning Objective},
  booktitle = {Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing},
  month = {October},
  address = {Honolulu, Hawaii},
  publisher = {Association for Computational Linguistics},
  pages = {630--639},
  url = {http://www.aclweb.org/anthology/D08-1066},
  year = 2008
  }
  Mylonakis and Sima'an (2008)