Moses::TranslationOptionCollection Class Reference

#include <TranslationOptionCollection.h>

Inheritance diagram for Moses::TranslationOptionCollection:

Inheritance graph
[legend]
Collaboration diagram for Moses::TranslationOptionCollection:

Collaboration graph
[legend]

List of all members.

Public Member Functions

virtual ~TranslationOptionCollection ()
const InputTypeGetSource () const
 input sentence/confusion network
const std::vector< Phrase * > & GetUnknownSources () const
 List of unknowns (OOVs).
size_t GetSize () const
 get length/size of source input
virtual void CreateTranslationOptions ()
 Create all possible translations from the phrase tables.
virtual void CreateTranslationOptionsForRange (const DecodeGraph &decodeStepList, size_t startPosition, size_t endPosition, bool adhereTableLimit, size_t graphInd)
 Create translation options that exactly cover a specific input span.
virtual bool HasXmlOptionsOverlappingRange (size_t startPosition, size_t endPosition) const
 Check if this range has XML options.
virtual void CreateXmlOptionsForRange (size_t startPosition, size_t endPosition)
 Create xml-based translation options for the specific input span.
virtual const SquareMatrixGetFutureScore () const
 returns future cost matrix for sentence
const TranslationOptionListGetTranslationOptionList (const WordsRange &coverage) const
 list of trans opt for a particular span
 TO_STRING ()

Protected Types

typedef std::vector
< std::vector
< InputLatticeNode > > 
TargetPhraseMatrix

Protected Member Functions

 TranslationOptionCollection (InputType const &src, size_t maxNoTransOptPerCoverage, float translationOptionThreshold)
void CalcFutureScore ()
void ProcessUnknownWord ()
 Force a creation of a translation option where there are none for a particular source position.
virtual void ProcessOneUnknownWord (const Word &sourceWord, size_t sourcePos, size_t length=1, const Scores *inputScores=NULL)
 special handling of ONE unknown words.
void Prune ()
 pruning: only keep the top n (m_maxNoTransOptPerCoverage) elements */
void Sort ()
 sort all trans opt in each list for cube pruning */
TranslationOptionListGetTranslationOptionList (size_t startPos, size_t endPos)
 list of trans opt for a particular span
const TranslationOptionListGetTranslationOptionList (size_t startPos, size_t endPos) const
void Add (TranslationOption *translationOption)
virtual void ProcessUnknownWord (size_t sourcePos)=0
 implemented by inherited class, called by this class
void EvaluateWithSource ()
void CacheLexReordering ()
void SetTargetPhraseFromPtMatrix ()
void SetTargetPhraseFromPtMatrix (const PhraseDictionary &phraseDictionary)
void SetTargetPhraseFromPtMatrix (const TargetPhraseCollection *phraseColl, const PhraseDictionary &phraseDictionary, const WordsRange &range)
void AddTargetPhraseFromPtMatrix (const PhraseDictionary &phraseDictionary)
const TargetPhraseCollectionGetTargetPhraseFromPtMatrix (const PhraseDictionary &phraseDictionary, const WordsRange &range) const

Protected Attributes

std::vector< std::vector
< TranslationOptionList > > 
m_collection
InputType const & m_source
SquareMatrix m_futureScore
const size_t m_maxNoTransOptPerCoverage
const float m_translationOptionThreshold
std::vector< Phrase * > m_unksrcs
std::vector< InputLatticeNode * > m_SourcePaths
std::map< const
PhraseDictionary
*, std::vector< std::vector
< const TargetPhraseCollection * > > > 
m_targetPhrasesfromPt

Friends

std::ostream & operator<< (std::ostream &out, const TranslationOptionCollection &coll)


Detailed Description

Contains all phrase translations applicable to current input type (a sentence or confusion network). A key insight into efficient decoding is that various input conditions (trelliss, factored input, normal text, xml markup) all lead to the same decoding algorithm: hypotheses are expanded by applying phrase translations, which can be precomputed.

The precomputation of a collection of instances of such TranslationOption depends on the input condition, but they all are presented to decoding algorithm in the same form, using this class.

This is a abstract class, and cannot be instantiated directly. Instantiate 1 of the inherited classes instead, for a particular input type

Definition at line 95 of file TranslationOptionCollection.h.


Member Typedef Documentation

typedef std::vector< std::vector<InputLatticeNode> > Moses::TranslationOptionCollection::TargetPhraseMatrix [protected]

Definition at line 109 of file TranslationOptionCollection.h.


Constructor & Destructor Documentation

Moses::TranslationOptionCollection::TranslationOptionCollection ( InputType const &  src,
size_t  maxNoTransOptPerCoverage,
float  translationOptionThreshold 
) [protected]

constructor; since translation options are indexed by coverage span, the corresponding data structure is initialized here This fn should be called by inherited classes

Definition at line 55 of file TranslationOptionCollection.cpp.

References Moses::StaticData::GetMaxPhraseLength(), Moses::InputType::GetSize(), Moses::StaticData::Instance(), and m_collection.

Here is the call graph for this function:

Moses::TranslationOptionCollection::~TranslationOptionCollection (  )  [virtual]

destructor, clears out data structures

Definition at line 78 of file TranslationOptionCollection.cpp.

References m_unksrcs, and Moses::RemoveAllInColl().

Here is the call graph for this function:


Member Function Documentation

void Moses::TranslationOptionCollection::Add ( TranslationOption translationOption  )  [protected]

add translation option to the list

Parameters:
translationOption translation option to be added

Definition at line 602 of file TranslationOptionCollection.cpp.

References CHECK, Moses::WordsRange::GetEndPos(), Moses::TranslationOption::GetSourceWordsRange(), Moses::WordsRange::GetStartPos(), and m_collection.

Referenced by CreateTranslationOptionsForRange(), and ProcessOneUnknownWord().

Here is the call graph for this function:

Here is the caller graph for this function:

void Moses::TranslationOptionCollection::AddTargetPhraseFromPtMatrix ( const PhraseDictionary phraseDictionary  )  [protected]

Definition at line 697 of file TranslationOptionCollection.cpp.

References Moses::InputType::GetSize(), m_source, m_targetPhrasesfromPt, and NULL.

Referenced by SetTargetPhraseFromPtMatrix().

Here is the call graph for this function:

Here is the caller graph for this function:

void Moses::TranslationOptionCollection::CacheLexReordering (  )  [protected]

void Moses::TranslationOptionCollection::CalcFutureScore (  )  [protected]

compute future score matrix in a dynamic programming fashion. This matrix used in search. Call this function once translation option collection has been filled with translation options

Definition at line 280 of file TranslationOptionCollection.cpp.

References Moses::TranslationOptionList::begin(), count, Moses::TranslationOptionList::end(), Moses::TranslationOption::GetFutureScore(), Moses::StaticData::GetMaxPhraseLength(), Moses::SquareMatrix::GetScore(), Moses::InputType::GetSize(), GetTranslationOptionList(), IFVERBOSE, Moses::StaticData::Instance(), m_futureScore, m_source, Moses::SquareMatrix::SetScore(), Moses::TranslationOptionList::size(), and TRACE_ERR.

Referenced by CreateTranslationOptions().

Here is the call graph for this function:

Here is the caller graph for this function:

void Moses::TranslationOptionCollection::CreateTranslationOptions (  )  [virtual]

Create all possible translations from the phrase tables.

Create all possible translations from the phrase tables for a particular input sentence. This implies applying all translation and generation steps. Also computes future cost matrix.

Definition at line 363 of file TranslationOptionCollection.cpp.

References CacheLexReordering(), CalcFutureScore(), CreateTranslationOptionsForRange(), EvaluateWithSource(), Moses::StaticData::GetDecodeGraphBackoff(), Moses::StaticData::GetDecodeGraphs(), Moses::StaticData::GetMaxPhraseLength(), Moses::InputType::GetSize(), Moses::StaticData::Instance(), m_collection, m_source, ProcessUnknownWord(), Prune(), Sort(), and VERBOSE.

Referenced by Moses::Manager::ProcessSentence().

Here is the call graph for this function:

Here is the caller graph for this function:

void Moses::TranslationOptionCollection::CreateTranslationOptionsForRange ( const DecodeGraph decodeGraph,
size_t  startPos,
size_t  endPos,
bool  adhereTableLimit,
size_t  graphInd 
) [virtual]

Create translation options that exactly cover a specific input span.

create translation options that exactly cover a specific input span. Called by CreateTranslationOptions() and ProcessUnknownWord()

Parameters:
decodeGraph list of decoding steps
factorCollection input sentence with all factors
startPos first position in input sentence
lastPos last position in input sentence
adhereTableLimit whether phrase & generation table limits are adhered to

Definition at line 470 of file TranslationOptionCollection.cpp.

References Add(), Moses::StaticData::AddTransOptListToCache(), Moses::DecodeGraph::begin(), CreateXmlOptionsForRange(), Moses::PartialTranslOptColl::DetachAll(), Moses::DecodeGraph::end(), Moses::StaticData::FindTransOptListInCache(), Moses::PartialTranslOptColl::GetList(), Moses::PartialTranslOptColl::GetPrunedCount(), Moses::InputType::GetSubString(), GetTranslationOptionList(), Moses::StaticData::GetUseTransOptCache(), HasXmlOptionsOverlappingRange(), Moses::StaticData::Instance(), m_source, NULL, Moses::DecodeStep::Process(), Moses::XmlExclusive, and Moses::XmlPassThrough.

Referenced by CreateTranslationOptions(), and ProcessUnknownWord().

Here is the call graph for this function:

Here is the caller graph for this function:

void Moses::TranslationOptionCollection::CreateXmlOptionsForRange ( size_t  startPosition,
size_t  endPosition 
) [virtual]

Create xml-based translation options for the specific input span.

Populates the current Collection with XML options exactly covering the range specified. Default implementation does nothing. called by CreateTranslationOptionsForRange()

Parameters:
startPos first position in input sentence
lastPos last position in input sentence

Reimplemented in Moses::TranslationOptionCollectionText.

Definition at line 592 of file TranslationOptionCollection.cpp.

Referenced by CreateTranslationOptionsForRange().

Here is the caller graph for this function:

void Moses::TranslationOptionCollection::EvaluateWithSource (  )  [protected]

virtual const SquareMatrix& Moses::TranslationOptionCollection::GetFutureScore (  )  const [inline, virtual]

returns future cost matrix for sentence

Definition at line 179 of file TranslationOptionCollection.h.

References m_futureScore.

Referenced by Moses::SearchCubePruning::CreateForwardTodos(), Moses::SearchNormal::ExpandAllHypotheses(), and Moses::SearchNormal::ExpandHypothesis().

Here is the caller graph for this function:

size_t Moses::TranslationOptionCollection::GetSize (  )  const [inline]

get length/size of source input

Definition at line 158 of file TranslationOptionCollection.h.

References Moses::InputType::GetSize(), and m_source.

Referenced by Moses::operator<<().

Here is the call graph for this function:

Here is the caller graph for this function:

const InputType& Moses::TranslationOptionCollection::GetSource (  )  const [inline]

input sentence/confusion network

Definition at line 150 of file TranslationOptionCollection.h.

References m_source.

Referenced by Moses::DecodeStepTranslation::Process().

Here is the caller graph for this function:

const TargetPhraseCollection * Moses::TranslationOptionCollection::GetTargetPhraseFromPtMatrix ( const PhraseDictionary phraseDictionary,
const WordsRange range 
) const [protected]

Definition at line 780 of file TranslationOptionCollection.cpp.

References CHECK, Moses::WordsRange::GetEndPos(), Moses::WordsRange::GetStartPos(), and m_targetPhrasesfromPt.

Referenced by SetTargetPhraseFromPtMatrix().

Here is the call graph for this function:

Here is the caller graph for this function:

const TranslationOptionList& Moses::TranslationOptionCollection::GetTranslationOptionList ( const WordsRange coverage  )  const [inline]

list of trans opt for a particular span

Definition at line 184 of file TranslationOptionCollection.h.

References Moses::WordsRange::GetEndPos(), Moses::WordsRange::GetStartPos(), and GetTranslationOptionList().

Here is the call graph for this function:

const TranslationOptionList & Moses::TranslationOptionCollection::GetTranslationOptionList ( size_t  startPos,
size_t  endPos 
) const [protected]

Definition at line 687 of file TranslationOptionCollection.cpp.

References CHECK, Moses::StaticData::GetMaxPhraseLength(), Moses::StaticData::Instance(), and m_collection.

Here is the call graph for this function:

TranslationOptionList & Moses::TranslationOptionCollection::GetTranslationOptionList ( size_t  startPos,
size_t  endPos 
) [protected]

const std::vector< Phrase * > & Moses::TranslationOptionCollection::GetUnknownSources (  )  const

List of unknowns (OOVs).

Definition at line 637 of file TranslationOptionCollection.cpp.

References m_unksrcs.

bool Moses::TranslationOptionCollection::HasXmlOptionsOverlappingRange ( size_t  startPosition,
size_t  endPosition 
) const [virtual]

Check if this range has XML options.

Check if this range overlaps with any XML options. This doesn't need to be an exact match, only an overlap. by default, we don't support XML options. subclasses need to override this function. called by CreateTranslationOptionsForRange()

Parameters:
startPos first position in input sentence
lastPos last position in input sentence
adhereTableLimit whether phrase & generation table limits are adhered to

Reimplemented in Moses::TranslationOptionCollectionText.

Definition at line 581 of file TranslationOptionCollection.cpp.

Referenced by CreateTranslationOptionsForRange().

Here is the caller graph for this function:

void Moses::TranslationOptionCollection::ProcessOneUnknownWord ( const Word sourceWord,
size_t  sourcePos,
size_t  length = 1,
const Scores inputScores = NULL 
) [protected, virtual]

special handling of ONE unknown words.

special handling of ONE unknown words. Either add temporarily add word to translation table, or drop the translation. This function should be called by the ProcessOneUnknownWord() in the inherited class At the moment, this unknown word handler is a bit of a hack, if copies over each factor from source to target word, or uses the 'UNK' factor. Ideally, this function should be in a class which can be expanded upon, for example, to create a morphologically aware handler.

Parameters:
sourceWord the unknown word
sourcePos 
length length covered by this word (may be > 1 for lattice input)
inputScores a set of scores associated with unknown word (input scores from latties/CNs)

Definition at line 203 of file TranslationOptionCollection.cpp.

References Add(), Moses::FactorCollection::AddFactor(), Moses::Phrase::AddWord(), Moses::ScoreComponentCollection::Assign(), EPSILON, Moses::TargetPhrase::Evaluate(), StringPiece::find_first_of(), Moses::FloorScore(), Moses::StaticData::GetDropUnknown(), Moses::TargetPhrase::GetScoreBreakdown(), Moses::Factor::GetString(), Moses::StaticData::GetUnknownWordPenaltyProducer(), Moses::FactorCollection::Instance(), Moses::StaticData::Instance(), m_unksrcs, NULL, Moses::TargetPhrase::SetAlignmentInfo(), Moses::TargetPhrase::SetInputScore(), Moses::TargetPhrase::SetSourcePhrase(), Moses::TransformScore(), and UNKNOWN_FACTOR.

Referenced by Moses::TranslationOptionCollectionConfusionNet::ProcessUnknownWord().

Here is the call graph for this function:

Here is the caller graph for this function:

virtual void Moses::TranslationOptionCollection::ProcessUnknownWord ( size_t  sourcePos  )  [protected, pure virtual]

implemented by inherited class, called by this class

Implemented in Moses::TranslationOptionCollectionConfusionNet, and Moses::TranslationOptionCollectionText.

void Moses::TranslationOptionCollection::ProcessUnknownWord (  )  [protected]

Force a creation of a translation option where there are none for a particular source position.

Force a creation of a translation option where there are none for a particular source position. ie. where a source word has not been translated, create a translation option by 1. not observing the table limits on phrase/generation tables 2. using the handler ProcessUnknownWord() Call this function once translation option collection has been filled with translation options

This function calls for unknown words is complicated by the fact it must handle different input types. The call stack is Base::ProcessUnknownWord() Inherited::ProcessUnknownWord(position) Base::ProcessOneUnknownWord()

Definition at line 165 of file TranslationOptionCollection.cpp.

References CreateTranslationOptionsForRange(), Moses::StaticData::GetDecodeGraphs(), Moses::InputType::GetSize(), GetTranslationOptionList(), Moses::StaticData::Instance(), Moses::StaticData::IsAlwaysCreateDirectTranslationOption(), m_source, and Moses::TranslationOptionList::size().

Referenced by CreateTranslationOptions().

Here is the call graph for this function:

Here is the caller graph for this function:

void Moses::TranslationOptionCollection::Prune (  )  [protected]

void Moses::TranslationOptionCollection::SetTargetPhraseFromPtMatrix ( const TargetPhraseCollection phraseColl,
const PhraseDictionary phraseDictionary,
const WordsRange range 
) [protected]

Definition at line 762 of file TranslationOptionCollection.cpp.

References Moses::WordsRange::GetEndPos(), Moses::WordsRange::GetStartPos(), and m_targetPhrasesfromPt.

Here is the call graph for this function:

void Moses::TranslationOptionCollection::SetTargetPhraseFromPtMatrix ( const PhraseDictionary phraseDictionary  )  [protected]

void Moses::TranslationOptionCollection::SetTargetPhraseFromPtMatrix (  )  [protected]

void Moses::TranslationOptionCollection::Sort (  )  [protected]

sort all trans opt in each list for cube pruning */

Definition at line 446 of file TranslationOptionCollection.cpp.

References Moses::TranslationOptionList::begin(), Moses::CompareTranslationOption(), Moses::TranslationOptionList::end(), Moses::StaticData::GetMaxPhraseLength(), Moses::InputType::GetSize(), GetTranslationOptionList(), Moses::StaticData::Instance(), and m_source.

Referenced by CreateTranslationOptions().

Here is the call graph for this function:

Here is the caller graph for this function:

Moses::TranslationOptionCollection::TO_STRING (  ) 


Friends And Related Function Documentation

std::ostream& operator<< ( std::ostream &  out,
const TranslationOptionCollection coll 
) [friend]

Definition at line 611 of file TranslationOptionCollection.cpp.


Member Data Documentation

Definition at line 102 of file TranslationOptionCollection.h.

Referenced by CalcFutureScore(), and GetFutureScore().

Definition at line 103 of file TranslationOptionCollection.h.

Referenced by Prune().

std::vector<InputLatticeNode*> Moses::TranslationOptionCollection::m_SourcePaths [protected]

Definition at line 107 of file TranslationOptionCollection.h.

Referenced by SetTargetPhraseFromPtMatrix().

std::map<const PhraseDictionary*, std::vector< std::vector<const TargetPhraseCollection*> > > Moses::TranslationOptionCollection::m_targetPhrasesfromPt [protected]

Definition at line 104 of file TranslationOptionCollection.h.

Referenced by Prune().


The documentation for this class was generated from the following files:

Generated on Wed Aug 27 00:31:26 2014 for Moses by  doxygen 1.5.9