Moses::TranslationOptionCollection Class Reference

#include <TranslationOptionCollection.h>

Inheritance diagram for Moses::TranslationOptionCollection:

[legend]

Collaboration diagram for Moses::TranslationOptionCollection:

[legend]

List of all members.

Public Member Functions
virtual	~TranslationOptionCollection ()
const InputType &	GetSource () const
	input sentence/confusion network
const std::vector< Phrase * > &	GetUnknownSources () const
	List of unknowns (OOVs).
size_t	GetSize () const
	get length/size of source input
virtual void	CreateTranslationOptions ()
	Create all possible translations from the phrase tables.
virtual void	CreateTranslationOptionsForRange (const DecodeGraph &decodeStepList, size_t startPosition, size_t endPosition, bool adhereTableLimit, size_t graphInd)
	Create translation options that exactly cover a specific input span.
virtual bool	HasXmlOptionsOverlappingRange (size_t startPosition, size_t endPosition) const
	Check if this range has XML options.
virtual void	CreateXmlOptionsForRange (size_t startPosition, size_t endPosition)
	Create xml-based translation options for the specific input span.
virtual const SquareMatrix &	GetFutureScore () const
	returns future cost matrix for sentence
const TranslationOptionList &	GetTranslationOptionList (const WordsRange &coverage) const
	list of trans opt for a particular span
	TO_STRING ()
Protected Member Functions
	TranslationOptionCollection (InputType const &src, size_t maxNoTransOptPerCoverage, float translationOptionThreshold)
void	CalcFutureScore ()
void	ProcessUnknownWord ()
	Force a creation of a translation option where there are none for a particular source position.
virtual void	ProcessOneUnknownWord (const Word &sourceWord, size_t sourcePos, size_t length=1, const Scores *inputScores=NULL)
	special handling of ONE unknown words.
void	Prune ()
	pruning: only keep the top n (m_maxNoTransOptPerCoverage) elements */
void	Sort ()
	sort all trans opt in each list for cube pruning */
TranslationOptionList &	GetTranslationOptionList (size_t startPos, size_t endPos)
	list of trans opt for a particular span
const TranslationOptionList &	GetTranslationOptionList (size_t startPos, size_t endPos) const
void	Add (TranslationOption *translationOption)
virtual void	ProcessUnknownWord (size_t sourcePos)=0
	implemented by inherited class, called by this class
void	CacheLexReordering ()
Protected Attributes
std::vector< std::vector < TranslationOptionList > >	m_collection
InputType const &	m_source
SquareMatrix	m_futureScore
const size_t	m_maxNoTransOptPerCoverage
const float	m_translationOptionThreshold
std::vector< Phrase * >	m_unksrcs
Friends
std::ostream &	operator<< (std::ostream &out, const TranslationOptionCollection &coll)

---

Detailed Description

Contains all phrase translations applicable to current input type (a sentence or confusion network). A key insight into efficient decoding is that various input conditions (trelliss, factored input, normal text, xml markup) all lead to the same decoding algorithm: hypotheses are expanded by applying phrase translations, which can be precomputed.

The precomputation of a collection of instances of such TranslationOption depends on the input condition, but they all are presented to decoding algorithm in the same form, using this class.

This is a abstract class, and cannot be instantiated directly. Instantiate 1 of the inherited classes instead, for a particular input type

Definition at line 59 of file TranslationOptionCollection.h.

---

Constructor & Destructor Documentation

Moses::TranslationOptionCollection::TranslationOptionCollection	(	InputType const &	src,
		size_t	maxNoTransOptPerCoverage,
		float	translationOptionThreshold
	)			[protected]

constructor; since translation options are indexed by coverage span, the corresponding data structure is initialized here This fn should be called by inherited classes

Definition at line 50 of file TranslationOptionCollection.cpp.

References Moses::StaticData::GetMaxPhraseLength(), Moses::InputType::GetSize(), Moses::StaticData::Instance(), and m_collection.

Here is the call graph for this function:

Moses::TranslationOptionCollection::~TranslationOptionCollection

(

)

[virtual]

destructor, clears out data structures

Definition at line 73 of file TranslationOptionCollection.cpp.

References m_unksrcs, and Moses::RemoveAllInColl().

Here is the call graph for this function:

---

Member Function Documentation

void Moses::TranslationOptionCollection::Add

(

TranslationOption *

translationOption

)

[protected]

add translation option to the list

Parameters:

translationOption

translation option to be added

Definition at line 578 of file TranslationOptionCollection.cpp.

References CHECK, Moses::WordsRange::GetEndPos(), Moses::TranslationOption::GetSourceWordsRange(), Moses::WordsRange::GetStartPos(), and m_collection.

Referenced by CreateTranslationOptionsForRange(), Moses::TranslationOptionCollectionText::CreateXmlOptionsForRange(), and ProcessOneUnknownWord().

Here is the call graph for this function:

Here is the caller graph for this function:

void Moses::TranslationOptionCollection::CacheLexReordering

(

)

[protected]

Definition at line 618 of file TranslationOptionCollection.cpp.

References Moses::TranslationOptionList::begin(), Moses::TranslationOption::CacheLexReorderingScores(), Moses::TranslationOptionList::end(), Moses::StaticData::GetMaxPhraseLength(), Moses::LexicalReordering::GetProb(), Moses::InputType::GetSize(), Moses::TranslationOption::GetSourcePhrase(), Moses::StatefulFeatureFunction::GetStatefulFeatureFunctions(), Moses::TranslationOption::GetTargetPhrase(), GetTranslationOptionList(), Moses::StaticData::Instance(), and m_source.

Referenced by CreateTranslationOptions().

Here is the call graph for this function:

Here is the caller graph for this function:

void Moses::TranslationOptionCollection::CalcFutureScore

(

)

[protected]

compute future score matrix in a dynamic programming fashion. This matrix used in search. Call this function once translation option collection has been filled with translation options

Definition at line 280 of file TranslationOptionCollection.cpp.

References Moses::TranslationOptionList::begin(), count, Moses::TranslationOptionList::end(), Moses::TranslationOption::GetFutureScore(), Moses::StaticData::GetMaxPhraseLength(), Moses::SquareMatrix::GetScore(), Moses::InputType::GetSize(), GetTranslationOptionList(), IFVERBOSE, Moses::StaticData::Instance(), m_futureScore, m_source, Moses::SquareMatrix::SetScore(), Moses::TranslationOptionList::size(), and TRACE_ERR.

Referenced by CreateTranslationOptions().

Here is the call graph for this function:

Here is the caller graph for this function:

void Moses::TranslationOptionCollection::CreateTranslationOptions

(

)

[virtual]

Create all possible translations from the phrase tables.

Create all possible translations from the phrase tables for a particular input sentence. This implies applying all translation and generation steps. Also computes future cost matrix.

Definition at line 363 of file TranslationOptionCollection.cpp.

References CacheLexReordering(), CalcFutureScore(), CreateTranslationOptionsForRange(), Moses::StaticData::GetDecodeGraphBackoff(), Moses::StaticData::GetDecodeGraphs(), Moses::StaticData::GetMaxPhraseLength(), Moses::InputType::GetSize(), Moses::StaticData::Instance(), m_collection, m_source, ProcessUnknownWord(), Prune(), Sort(), and VERBOSE.

Referenced by Moses::Manager::ProcessSentence().

Here is the call graph for this function:

Here is the caller graph for this function:

void Moses::TranslationOptionCollection::CreateTranslationOptionsForRange	(	const DecodeGraph &	decodeGraph,
		size_t	startPos,
		size_t	endPos,
		bool	adhereTableLimit,
		size_t	graphInd
	)			[virtual]

Create translation options that exactly cover a specific input span.

create translation options that exactly cover a specific input span. Called by CreateTranslationOptions() and ProcessUnknownWord()

Parameters:

	decodeGraph	list of decoding steps
	factorCollection	input sentence with all factors
	startPos	first position in input sentence
	lastPos	last position in input sentence
	adhereTableLimit	whether phrase & generation table limits are adhered to

Definition at line 447 of file TranslationOptionCollection.cpp.

References Add(), Moses::StaticData::AddTransOptListToCache(), Moses::DecodeGraph::begin(), CreateXmlOptionsForRange(), Moses::PartialTranslOptColl::DetachAll(), Moses::DecodeGraph::end(), Moses::StaticData::FindTransOptListInCache(), Moses::PartialTranslOptColl::GetList(), Moses::PartialTranslOptColl::GetPrunedCount(), Moses::InputType::GetSubString(), GetTranslationOptionList(), Moses::StaticData::GetUseTransOptCache(), HasXmlOptionsOverlappingRange(), Moses::StaticData::Instance(), m_source, NULL, Moses::DecodeStep::Process(), Moses::XmlExclusive, and Moses::XmlPassThrough.

Referenced by CreateTranslationOptions(), and ProcessUnknownWord().

Here is the call graph for this function:

Here is the caller graph for this function:

void Moses::TranslationOptionCollection::CreateXmlOptionsForRange	(	size_t	startPosition,
		size_t	endPosition
	)			[virtual]

Create xml-based translation options for the specific input span.

Populates the current Collection with XML options exactly covering the range specified. Default implementation does nothing. called by CreateTranslationOptionsForRange()

Parameters:

	startPos	first position in input sentence
	lastPos	last position in input sentence

Reimplemented in Moses::TranslationOptionCollectionText.

Definition at line 568 of file TranslationOptionCollection.cpp.

Referenced by CreateTranslationOptionsForRange().

Here is the caller graph for this function:

virtual const SquareMatrix& Moses::TranslationOptionCollection::GetFutureScore

(

)

const [inline, virtual]

returns future cost matrix for sentence

Definition at line 132 of file TranslationOptionCollection.h.

References m_futureScore.

Referenced by Moses::SearchCubePruning::CreateForwardTodos(), Moses::SearchNormal::ExpandAllHypotheses(), and Moses::SearchNormal::ExpandHypothesis().

Here is the caller graph for this function:

size_t Moses::TranslationOptionCollection::GetSize

(

)

const [inline]

get length/size of source input

Definition at line 111 of file TranslationOptionCollection.h.

References Moses::InputType::GetSize(), and m_source.

Referenced by Moses::operator<<().

Here is the call graph for this function:

Here is the caller graph for this function:

const InputType& Moses::TranslationOptionCollection::GetSource

(

)

const [inline]

input sentence/confusion network

Definition at line 103 of file TranslationOptionCollection.h.

References m_source.

Referenced by Moses::DecodeStepTranslation::Process().

Here is the caller graph for this function:

const TranslationOptionList& Moses::TranslationOptionCollection::GetTranslationOptionList

(

const WordsRange &

coverage

)

const [inline]

list of trans opt for a particular span

Definition at line 137 of file TranslationOptionCollection.h.

References Moses::WordsRange::GetEndPos(), Moses::WordsRange::GetStartPos(), and GetTranslationOptionList().

Here is the call graph for this function:

const TranslationOptionList & Moses::TranslationOptionCollection::GetTranslationOptionList	(	size_t	startPos,
		size_t	endPos
	)			const [protected]

Definition at line 663 of file TranslationOptionCollection.cpp.

References CHECK, Moses::StaticData::GetMaxPhraseLength(), Moses::StaticData::Instance(), and m_collection.

Here is the call graph for this function:

TranslationOptionList & Moses::TranslationOptionCollection::GetTranslationOptionList	(	size_t	startPos,
		size_t	endPos
	)			[protected]

list of trans opt for a particular span

Definition at line 654 of file TranslationOptionCollection.cpp.

References CHECK, Moses::StaticData::GetMaxPhraseLength(), Moses::StaticData::Instance(), and m_collection.

Referenced by CacheLexReordering(), CalcFutureScore(), Moses::SearchCubePruning::CreateForwardTodos(), CreateTranslationOptionsForRange(), Moses::SearchNormal::ExpandAllHypotheses(), GetTranslationOptionList(), Moses::operator<<(), Moses::SearchNormal::ProcessOneHypothesis(), ProcessUnknownWord(), Prune(), and Sort().

Here is the call graph for this function:

Here is the caller graph for this function:

const std::vector< Phrase * > & Moses::TranslationOptionCollection::GetUnknownSources

(

)

const

List of unknowns (OOVs).

Definition at line 613 of file TranslationOptionCollection.cpp.

References m_unksrcs.

bool Moses::TranslationOptionCollection::HasXmlOptionsOverlappingRange	(	size_t	startPosition,
		size_t	endPosition
	)			const [virtual]

Check if this range has XML options.

Check if this range overlaps with any XML options. This doesn't need to be an exact match, only an overlap. by default, we don't support XML options. subclasses need to override this function. called by CreateTranslationOptionsForRange()

Parameters:

	startPos	first position in input sentence
	lastPos	last position in input sentence
	adhereTableLimit	whether phrase & generation table limits are adhered to

Reimplemented in Moses::TranslationOptionCollectionText.

Definition at line 557 of file TranslationOptionCollection.cpp.

Referenced by CreateTranslationOptionsForRange().

Here is the caller graph for this function:

void Moses::TranslationOptionCollection::ProcessOneUnknownWord	(	const Word &	sourceWord,
		size_t	sourcePos,
		size_t	length = 1,
		const Scores *	inputScores = NULL
	)			[protected, virtual]

special handling of ONE unknown words.

special handling of ONE unknown words. Either add temporarily add word to translation table, or drop the translation. This function should be called by the ProcessOneUnknownWord() in the inherited class At the moment, this unknown word handler is a bit of a hack, if copies over each factor from source to target word, or uses the 'UNK' factor. Ideally, this function should be in a class which can be expanded upon, for example, to create a morphologically aware handler.

Parameters:

	sourceWord	the unknown word
	sourcePos
	length	length covered by this word (may be > 1 for lattice input)
	inputScores	a set of scores associated with unknown word (input scores from latties/CNs)

Definition at line 198 of file TranslationOptionCollection.cpp.

References Add(), Moses::FactorCollection::AddFactor(), Moses::Phrase::AddWord(), Moses::ScoreComponentCollection::Assign(), EPSILON, Moses::TargetPhrase::Evaluate(), StringPiece::find_first_of(), Moses::FloorScore(), Moses::StaticData::GetDropUnknown(), Moses::TargetPhrase::GetScoreBreakdown(), Moses::Factor::GetString(), Moses::StaticData::GetUnknownWordPenaltyProducer(), Moses::FactorCollection::Instance(), Moses::StaticData::Instance(), m_unksrcs, NULL, Moses::TargetPhrase::SetAlignmentInfo(), Moses::TargetPhrase::SetInputScore(), Moses::TargetPhrase::SetSourcePhrase(), Moses::TransformScore(), and UNKNOWN_FACTOR.

Referenced by Moses::TranslationOptionCollectionText::ProcessUnknownWord(), and Moses::TranslationOptionCollectionConfusionNet::ProcessUnknownWord().

Here is the call graph for this function:

Here is the caller graph for this function:

virtual void Moses::TranslationOptionCollection::ProcessUnknownWord

(

size_t

sourcePos

)

[protected, pure virtual]

implemented by inherited class, called by this class

Implemented in Moses::TranslationOptionCollectionConfusionNet, and Moses::TranslationOptionCollectionText.

void Moses::TranslationOptionCollection::ProcessUnknownWord

(

)

[protected]

Force a creation of a translation option where there are none for a particular source position.

Force a creation of a translation option where there are none for a particular source position. ie. where a source word has not been translated, create a translation option by 1. not observing the table limits on phrase/generation tables 2. using the handler ProcessUnknownWord() Call this function once translation option collection has been filled with translation options

This function calls for unknown words is complicated by the fact it must handle different input types. The call stack is Base::ProcessUnknownWord() Inherited::ProcessUnknownWord(position) Base::ProcessOneUnknownWord()

Definition at line 160 of file TranslationOptionCollection.cpp.

References CreateTranslationOptionsForRange(), Moses::StaticData::GetDecodeGraphs(), Moses::InputType::GetSize(), GetTranslationOptionList(), Moses::StaticData::Instance(), Moses::StaticData::IsAlwaysCreateDirectTranslationOption(), m_source, and Moses::TranslationOptionList::size().

Referenced by CreateTranslationOptions().

Here is the call graph for this function:

Here is the caller graph for this function:

void Moses::TranslationOptionCollection::Prune

(

)

[protected]

pruning: only keep the top n (m_maxNoTransOptPerCoverage) elements */

Definition at line 78 of file TranslationOptionCollection.cpp.

References Moses::TranslationOptionList::begin(), Moses::CompareTranslationOption(), Moses::TranslationOptionList::end(), Moses::TranslationOptionList::Get(), Moses::TranslationOption::GetFutureScore(), Moses::StaticData::GetMaxPhraseLength(), Moses::InputType::GetSize(), GetTranslationOptionList(), Moses::StaticData::Instance(), m_maxNoTransOptPerCoverage, m_source, m_translationOptionThreshold, Moses::TranslationOptionList::Remove(), Moses::TranslationOptionList::resize(), Moses::TranslationOptionList::size(), and VERBOSE.

Referenced by CreateTranslationOptions().

Here is the call graph for this function:

Here is the caller graph for this function:

void Moses::TranslationOptionCollection::Sort

(

)

[protected]

sort all trans opt in each list for cube pruning */

Definition at line 423 of file TranslationOptionCollection.cpp.

References Moses::TranslationOptionList::begin(), Moses::CompareTranslationOption(), Moses::TranslationOptionList::end(), Moses::StaticData::GetMaxPhraseLength(), Moses::InputType::GetSize(), GetTranslationOptionList(), Moses::StaticData::Instance(), and m_source.

Referenced by CreateTranslationOptions().

Here is the call graph for this function:

Here is the caller graph for this function:

Moses::TranslationOptionCollection::TO_STRING

(

)

---

Friends And Related Function Documentation

std::ostream& operator<<	(	std::ostream &	out,
		const TranslationOptionCollection &	coll
	)			[friend]

Definition at line 587 of file TranslationOptionCollection.cpp.

---

Member Data Documentation

std::vector< std::vector< TranslationOptionList > > Moses::TranslationOptionCollection::m_collection [protected]

Definition at line 64 of file TranslationOptionCollection.h.

Referenced by Add(), CreateTranslationOptions(), GetTranslationOptionList(), and TranslationOptionCollection().

SquareMatrix Moses::TranslationOptionCollection::m_futureScore [protected]

Definition at line 66 of file TranslationOptionCollection.h.

Referenced by CalcFutureScore(), and GetFutureScore().

const size_t Moses::TranslationOptionCollection::m_maxNoTransOptPerCoverage [protected]

Definition at line 67 of file TranslationOptionCollection.h.

Referenced by Prune().

InputType const& Moses::TranslationOptionCollection::m_source [protected]

Definition at line 65 of file TranslationOptionCollection.h.

Referenced by CacheLexReordering(), CalcFutureScore(), CreateTranslationOptions(), CreateTranslationOptionsForRange(), Moses::TranslationOptionCollectionText::CreateXmlOptionsForRange(), GetSize(), GetSource(), Moses::TranslationOptionCollectionText::HasXmlOptionsOverlappingRange(), Moses::TranslationOptionCollectionText::ProcessUnknownWord(), Moses::TranslationOptionCollectionConfusionNet::ProcessUnknownWord(), ProcessUnknownWord(), Prune(), and Sort().

const float Moses::TranslationOptionCollection::m_translationOptionThreshold [protected]

Definition at line 68 of file TranslationOptionCollection.h.

Referenced by Prune().

std::vector<Phrase*> Moses::TranslationOptionCollection::m_unksrcs [protected]

Definition at line 69 of file TranslationOptionCollection.h.

Referenced by GetUnknownSources(), ProcessOneUnknownWord(), and ~TranslationOptionCollection().

---

The documentation for this class was generated from the following files:

• mosesdecoder/moses/TranslationOptionCollection.h
• mosesdecoder/moses/TranslationOptionCollection.cpp