Moses::TranslationOptionCollection Class Reference

#include <TranslationOptionCollection.h>

Inheritance diagram for Moses::TranslationOptionCollection:

[legend]

Collaboration diagram for Moses::TranslationOptionCollection:

[legend]

List of all members.

Public Member Functions
virtual	~TranslationOptionCollection ()
const InputType &	GetSource () const
	input sentence/confusion network
size_t	GetSize () const
	get length/size of source input
virtual void	CreateTranslationOptions ()
	Create all possible translations from the phrase tables.
virtual void	CreateTranslationOptionsForRange (const DecodeGraph &decodeStepList, size_t startPosition, size_t endPosition, bool adhereTableLimit)
	Create translation options that exactly cover a specific input span.
virtual bool	HasXmlOptionsOverlappingRange (size_t startPosition, size_t endPosition) const
	Check if this range has XML options.
virtual void	CreateXmlOptionsForRange (size_t startPosition, size_t endPosition)
	Create xml-based translation options for the specific input span.
virtual const SquareMatrix &	GetFutureScore () const
	returns future cost matrix for sentence
const TranslationOptionList &	GetTranslationOptionList (const WordsRange &coverage) const
	list of trans opt for a particular span
	TO_STRING ()
Protected Member Functions
	TranslationOptionCollection (const TranslationSystem *system, InputType const &src, size_t maxNoTransOptPerCoverage, float translationOptionThreshold)
void	CalcFutureScore ()
void	ProcessUnknownWord ()
	Force a creation of a translation option where there are none for a particular source position.
virtual void	ProcessOneUnknownWord (const Word &sourceWord, size_t sourcePos, size_t length=1, const Scores *inputScores=NULL)
	special handling of ONE unknown words.
void	Prune ()
	pruning: only keep the top n (m_maxNoTransOptPerCoverage) elements */
void	Sort ()
	sort all trans opt in each list for cube pruning */
TranslationOptionList &	GetTranslationOptionList (size_t startPos, size_t endPos)
	list of trans opt for a particular span
const TranslationOptionList &	GetTranslationOptionList (size_t startPos, size_t endPos) const
void	Add (TranslationOption *translationOption)
virtual void	ProcessUnknownWord (size_t sourcePos)=0
	implemented by inherited class, called by this class
void	CacheLexReordering ()
Protected Attributes
const TranslationSystem *	m_system
std::vector< std::vector < TranslationOptionList > >	m_collection
InputType const &	m_source
SquareMatrix	m_futureScore
const size_t	m_maxNoTransOptPerCoverage
const float	m_translationOptionThreshold
std::vector< Phrase * >	m_unksrcs
Friends
std::ostream &	operator<< (std::ostream &out, const TranslationOptionCollection &coll)

---

Detailed Description

Definition at line 62 of file TranslationOptionCollection.h.

---

Constructor & Destructor Documentation

Moses::TranslationOptionCollection::TranslationOptionCollection	(	const TranslationSystem *	system,
		InputType const &	src,
		size_t	maxNoTransOptPerCoverage,
		float	translationOptionThreshold
	)			[protected]

constructor; since translation options are indexed by coverage span, the corresponding data structure is initialized here This fn should be called by inherited classes

Definition at line 50 of file TranslationOptionCollection.cpp.

References Moses::StaticData::GetMaxPhraseLength(), Moses::InputType::GetSize(), Moses::StaticData::Instance(), and m_collection.

Here is the call graph for this function:

Moses::TranslationOptionCollection::~TranslationOptionCollection

(

)

[virtual]

destructor, clears out data structures

Definition at line 74 of file TranslationOptionCollection.cpp.

References m_unksrcs, and Moses::RemoveAllInColl().

Here is the call graph for this function:

---

Member Function Documentation

void Moses::TranslationOptionCollection::Add

(

TranslationOption *

translationOption

)

[protected]

add translation option to the list

Parameters:

translationOption

translation option to be added

Definition at line 565 of file TranslationOptionCollection.cpp.

References CHECK, Moses::WordsRange::GetEndPos(), Moses::TranslationOption::GetSourceWordsRange(), Moses::WordsRange::GetStartPos(), and m_collection.

Referenced by CreateTranslationOptionsForRange(), Moses::TranslationOptionCollectionText::CreateXmlOptionsForRange(), and ProcessOneUnknownWord().

Here is the call graph for this function:

Here is the caller graph for this function:

void Moses::TranslationOptionCollection::CacheLexReordering

(

)

[protected]

Definition at line 600 of file TranslationOptionCollection.cpp.

References Moses::TranslationOptionList::begin(), Moses::TranslationOption::CacheScores(), Moses::TranslationOptionList::end(), Moses::StaticData::GetMaxPhraseLength(), Moses::LexicalReordering::GetProb(), Moses::TranslationSystem::GetReorderModels(), Moses::InputType::GetSize(), Moses::TranslationOption::GetSourcePhrase(), Moses::TranslationOption::GetTargetPhrase(), GetTranslationOptionList(), Moses::StaticData::Instance(), m_source, and m_system.

Referenced by CreateTranslationOptions().

Here is the call graph for this function:

Here is the caller graph for this function:

void Moses::TranslationOptionCollection::CalcFutureScore

(

)

[protected]

compute future score matrix in a dynamic programming fashion. This matrix used in search. Call this function once translation option collection has been filled with translation options

Definition at line 270 of file TranslationOptionCollection.cpp.

References Moses::TranslationOptionList::begin(), Moses::TranslationOptionList::end(), Moses::TranslationOption::GetFutureScore(), Moses::StaticData::GetMaxPhraseLength(), Moses::SquareMatrix::GetScore(), Moses::InputType::GetSize(), GetTranslationOptionList(), IFVERBOSE, Moses::StaticData::Instance(), m_futureScore, m_source, Moses::SquareMatrix::SetScore(), Moses::TranslationOptionList::size(), and TRACE_ERR.

Referenced by CreateTranslationOptions().

Here is the call graph for this function:

Here is the caller graph for this function:

void Moses::TranslationOptionCollection::CreateTranslationOptions

(

)

[virtual]

Create all possible translations from the phrase tables.

Create all possible translations from the phrase tables for a particular input sentence. This implies applying all translation and generation steps. Also computes future cost matrix.

Parameters:

	decodeStepList	list of decoding steps
	factorCollection	input sentence with all factors

Definition at line 355 of file TranslationOptionCollection.cpp.

References CacheLexReordering(), CalcFutureScore(), CreateTranslationOptionsForRange(), Moses::TranslationSystem::GetDecodeGraphBackoff(), Moses::TranslationSystem::GetDecodeGraphs(), Moses::StaticData::GetMaxPhraseLength(), Moses::InputType::GetSize(), Moses::StaticData::Instance(), m_collection, m_source, m_system, ProcessUnknownWord(), Prune(), Sort(), and VERBOSE.

Referenced by Moses::Manager::ProcessSentence().

Here is the call graph for this function:

Here is the caller graph for this function:

void Moses::TranslationOptionCollection::CreateTranslationOptionsForRange	(	const DecodeGraph &	decodeGraph,
		size_t	startPos,
		size_t	endPos,
		bool	adhereTableLimit
	)			[virtual]

Create translation options that exactly cover a specific input span.

create translation options that exactly cover a specific input span. Called by CreateTranslationOptions() and ProcessUnknownWord()

Parameters:

	decodeGraph	list of decoding steps
	factorCollection	input sentence with all factors
	startPos	first position in input sentence
	lastPos	last position in input sentence
	adhereTableLimit	whether phrase & generation table limits are adhered to

Definition at line 439 of file TranslationOptionCollection.cpp.

References Add(), Moses::StaticData::AddTransOptListToCache(), Moses::DecodeGraph::begin(), Moses::TranslationOption::CalcScore(), CreateXmlOptionsForRange(), Moses::PartialTranslOptColl::DetachAll(), Moses::DecodeGraph::end(), Moses::StaticData::FindTransOptListInCache(), Moses::PartialTranslOptColl::GetList(), Moses::PartialTranslOptColl::GetPrunedCount(), Moses::InputType::GetSubString(), GetTranslationOptionList(), Moses::StaticData::GetUseTransOptCache(), HasXmlOptionsOverlappingRange(), Moses::StaticData::Instance(), m_source, m_system, NULL, Moses::DecodeStep::Process(), Moses::XmlExclusive, and Moses::XmlPassThrough.

Referenced by CreateTranslationOptions(), and ProcessUnknownWord().

Here is the call graph for this function:

Here is the caller graph for this function:

void Moses::TranslationOptionCollection::CreateXmlOptionsForRange	(	size_t	startPosition,
		size_t	endPosition
	)			[virtual]

Create xml-based translation options for the specific input span.

Populates the current Collection with XML options exactly covering the range specified. Default implementation does nothing. called by CreateTranslationOptionsForRange()

Parameters:

	startPos	first position in input sentence
	lastPos	last position in input sentence

Reimplemented in Moses::TranslationOptionCollectionText.

Definition at line 555 of file TranslationOptionCollection.cpp.

Referenced by CreateTranslationOptionsForRange().

Here is the caller graph for this function:

virtual const SquareMatrix& Moses::TranslationOptionCollection::GetFutureScore

(

)

const [inline, virtual]

returns future cost matrix for sentence

Definition at line 129 of file TranslationOptionCollection.h.

References m_futureScore.

Referenced by Moses::SearchCubePruning::CreateForwardTodos(), Moses::SearchNormal::ExpandAllHypotheses(), and Moses::SearchNormal::ExpandHypothesis().

Here is the caller graph for this function:

size_t Moses::TranslationOptionCollection::GetSize

(

)

const [inline]

get length/size of source input

Definition at line 109 of file TranslationOptionCollection.h.

References Moses::InputType::GetSize(), and m_source.

Referenced by Moses::operator<<().

Here is the call graph for this function:

Here is the caller graph for this function:

const InputType& Moses::TranslationOptionCollection::GetSource

(

)

const [inline]

input sentence/confusion network

Definition at line 104 of file TranslationOptionCollection.h.

References m_source.

Referenced by Moses::DecodeStepTranslation::Process().

Here is the caller graph for this function:

const TranslationOptionList& Moses::TranslationOptionCollection::GetTranslationOptionList

(

const WordsRange &

coverage

)

const [inline]

list of trans opt for a particular span

Definition at line 134 of file TranslationOptionCollection.h.

References Moses::WordsRange::GetEndPos(), Moses::WordsRange::GetStartPos(), and GetTranslationOptionList().

Here is the call graph for this function:

const TranslationOptionList & Moses::TranslationOptionCollection::GetTranslationOptionList	(	size_t	startPos,
		size_t	endPos
	)			const [protected]

Definition at line 642 of file TranslationOptionCollection.cpp.

References CHECK, Moses::StaticData::GetMaxPhraseLength(), Moses::StaticData::Instance(), and m_collection.

Here is the call graph for this function:

TranslationOptionList & Moses::TranslationOptionCollection::GetTranslationOptionList	(	size_t	startPos,
		size_t	endPos
	)			[protected]

list of trans opt for a particular span

Definition at line 633 of file TranslationOptionCollection.cpp.

References CHECK, Moses::StaticData::GetMaxPhraseLength(), Moses::StaticData::Instance(), and m_collection.

Referenced by CacheLexReordering(), CalcFutureScore(), Moses::SearchCubePruning::CreateForwardTodos(), CreateTranslationOptionsForRange(), Moses::SearchNormal::ExpandAllHypotheses(), GetTranslationOptionList(), Moses::operator<<(), Moses::SearchNormal::ProcessOneHypothesis(), ProcessUnknownWord(), Prune(), and Sort().

Here is the call graph for this function:

Here is the caller graph for this function:

bool Moses::TranslationOptionCollection::HasXmlOptionsOverlappingRange	(	size_t	startPosition,
		size_t	endPosition
	)			const [virtual]

Check if this range has XML options.

Check if this range overlaps with any XML options. This doesn't need to be an exact match, only an overlap. by default, we don't support XML options. subclasses need to override this function. called by CreateTranslationOptionsForRange()

Parameters:

	startPos	first position in input sentence
	lastPos	last position in input sentence
	adhereTableLimit	whether phrase & generation table limits are adhered to

Reimplemented in Moses::TranslationOptionCollectionText.

Definition at line 544 of file TranslationOptionCollection.cpp.

Referenced by CreateTranslationOptionsForRange().

Here is the caller graph for this function:

void Moses::TranslationOptionCollection::ProcessOneUnknownWord	(	const Word &	sourceWord,
		size_t	sourcePos,
		size_t	length = 1,
		const Scores *	inputScores = NULL
	)			[protected, virtual]

special handling of ONE unknown words.

special handling of ONE unknown words. Either add temporarily add word to translation table, or drop the translation. This function should be called by the ProcessOneUnknownWord() in the inherited class At the moment, this unknown word handler is a bit of a hack, if copies over each factor from source to target word, or uses the 'UNK' factor. Ideally, this function should be in a class which can be expanded upon, for example, to create a morphologically aware handler.

Parameters:

	sourceWord	the unknown word
	sourcePos
	length	length covered by this word (may be > 1 for lattice input)
	inputScores	a set of scores associated with unknown word (input scores from latties/CNs)

Definition at line 201 of file TranslationOptionCollection.cpp.

References Add(), Moses::FactorCollection::AddFactor(), Moses::Phrase::AddWord(), Moses::TranslationOption::CalcScore(), EPSILON, Moses::StaticData::GetDropUnknown(), Moses::Factor::GetString(), Moses::TranslationSystem::GetUnknownWordPenaltyProducer(), Moses::StaticData::Instance(), Moses::FactorCollection::Instance(), m_source, m_system, m_unksrcs, Moses::MAX_NUM_FACTORS, NULL, Moses::Output, Moses::TargetPhrase::SetAlignmentInfo(), Moses::TargetPhrase::SetScore(), Moses::TargetPhrase::SetSourcePhrase(), and UNKNOWN_FACTOR.

Referenced by Moses::TranslationOptionCollectionText::ProcessUnknownWord(), and Moses::TranslationOptionCollectionConfusionNet::ProcessUnknownWord().

Here is the call graph for this function:

Here is the caller graph for this function:

virtual void Moses::TranslationOptionCollection::ProcessUnknownWord

(

size_t

sourcePos

)

[protected, pure virtual]

implemented by inherited class, called by this class

Implemented in Moses::TranslationOptionCollectionConfusionNet, and Moses::TranslationOptionCollectionText.

void Moses::TranslationOptionCollection::ProcessUnknownWord

(

)

[protected]

Force a creation of a translation option where there are none for a particular source position.

Force a creation of a translation option where there are none for a particular source position. ie. where a source word has not been translated, create a translation option by 1. not observing the table limits on phrase/generation tables 2. using the handler ProcessUnknownWord() Call this function once translation option collection has been filled with translation options

This function calls for unknown words is complicated by the fact it must handle different input types. The call stack is Base::ProcessUnknownWord() Inherited::ProcessUnknownWord(position) Base::ProcessOneUnknownWord()

Parameters:

	decodeStepList	list of decoding steps
	factorCollection	input sentence with all factors

Definition at line 163 of file TranslationOptionCollection.cpp.

References CreateTranslationOptionsForRange(), Moses::TranslationSystem::GetDecodeGraphs(), Moses::InputType::GetSize(), GetTranslationOptionList(), Moses::StaticData::Instance(), Moses::StaticData::IsAlwaysCreateDirectTranslationOption(), m_source, m_system, and Moses::TranslationOptionList::size().

Referenced by CreateTranslationOptions().

Here is the call graph for this function:

Here is the caller graph for this function:

void Moses::TranslationOptionCollection::Prune

(

)

[protected]

pruning: only keep the top n (m_maxNoTransOptPerCoverage) elements */

Definition at line 79 of file TranslationOptionCollection.cpp.

References Moses::TranslationOptionList::begin(), Moses::CompareTranslationOption(), Moses::TranslationOptionList::end(), Moses::TranslationOptionList::Get(), Moses::TranslationOption::GetFutureScore(), Moses::StaticData::GetMaxPhraseLength(), Moses::InputType::GetSize(), GetTranslationOptionList(), Moses::StaticData::Instance(), m_maxNoTransOptPerCoverage, m_source, m_translationOptionThreshold, Moses::TranslationOptionList::Remove(), Moses::TranslationOptionList::resize(), Moses::TranslationOptionList::size(), and VERBOSE.

Referenced by CreateTranslationOptions().

Here is the call graph for this function:

Here is the caller graph for this function:

void Moses::TranslationOptionCollection::Sort

(

)

[protected]

sort all trans opt in each list for cube pruning */

Definition at line 415 of file TranslationOptionCollection.cpp.

References Moses::TranslationOptionList::begin(), Moses::CompareTranslationOption(), Moses::TranslationOptionList::end(), Moses::StaticData::GetMaxPhraseLength(), Moses::InputType::GetSize(), GetTranslationOptionList(), Moses::StaticData::Instance(), and m_source.

Referenced by CreateTranslationOptions().

Here is the call graph for this function:

Here is the caller graph for this function:

Moses::TranslationOptionCollection::TO_STRING

(

)

---

Friends And Related Function Documentation

std::ostream& operator<<	(	std::ostream &	out,
		const TranslationOptionCollection &	coll
	)			[friend]

Definition at line 574 of file TranslationOptionCollection.cpp.

---

Member Data Documentation

std::vector< std::vector< TranslationOptionList > > Moses::TranslationOptionCollection::m_collection [protected]

Definition at line 68 of file TranslationOptionCollection.h.

Referenced by Add(), CreateTranslationOptions(), GetTranslationOptionList(), and TranslationOptionCollection().

SquareMatrix Moses::TranslationOptionCollection::m_futureScore [protected]

Definition at line 70 of file TranslationOptionCollection.h.

Referenced by CalcFutureScore(), and GetFutureScore().

const size_t Moses::TranslationOptionCollection::m_maxNoTransOptPerCoverage [protected]

Definition at line 71 of file TranslationOptionCollection.h.

Referenced by Prune().

InputType const& Moses::TranslationOptionCollection::m_source [protected]

Definition at line 69 of file TranslationOptionCollection.h.

Referenced by CacheLexReordering(), CalcFutureScore(), CreateTranslationOptions(), CreateTranslationOptionsForRange(), Moses::TranslationOptionCollectionText::CreateXmlOptionsForRange(), GetSize(), GetSource(), Moses::TranslationOptionCollectionText::HasXmlOptionsOverlappingRange(), ProcessOneUnknownWord(), Moses::TranslationOptionCollectionText::ProcessUnknownWord(), Moses::TranslationOptionCollectionConfusionNet::ProcessUnknownWord(), ProcessUnknownWord(), Prune(), and Sort().

const TranslationSystem* Moses::TranslationOptionCollection::m_system [protected]

Definition at line 67 of file TranslationOptionCollection.h.

Referenced by CacheLexReordering(), CreateTranslationOptions(), CreateTranslationOptionsForRange(), Moses::TranslationOptionCollectionText::CreateXmlOptionsForRange(), ProcessOneUnknownWord(), and ProcessUnknownWord().

const float Moses::TranslationOptionCollection::m_translationOptionThreshold [protected]

Definition at line 72 of file TranslationOptionCollection.h.

Referenced by Prune().

std::vector<Phrase*> Moses::TranslationOptionCollection::m_unksrcs [protected]

Definition at line 73 of file TranslationOptionCollection.h.

Referenced by ProcessOneUnknownWord(), and ~TranslationOptionCollection().

---

The documentation for this class was generated from the following files:

• mosesdecoder/moses/src/TranslationOptionCollection.h
• mosesdecoder/moses/src/TranslationOptionCollection.cpp