Moses::Manager Class Reference

#include <Manager.h>

Collaboration diagram for Moses::Manager:

List of all members.

Public Member Functions

Manager (size_t lineNumber, InputType const &source, SearchAlgorithm searchAlgorithm)

~Manager ()

const TranslationOptionCollection * getSntTranslationOptions ()

void ProcessSentence ()

const Hypothesis * GetBestHypothesis () const

const Hypothesis * GetActualBestHypothesis () const

void CalcNBest (size_t count, TrellisPathList &ret, bool onlyDistinct=0) const

void CalcLatticeSamples (size_t count, TrellisPathList &ret) const

void PrintAllDerivations (long translationId, std::ostream &outputStream) const

void printDivergentHypothesis (long translationId, const Hypothesis *hypo, const std::vector< const TargetPhrase * > &remainingPhrases, float remainingScore, std::ostream &outputStream) const

void printThisHypothesis (long translationId, const Hypothesis *hypo, const std::vector< const TargetPhrase * > &remainingPhrases, float remainingScore, std::ostream &outputStream) const

void GetWordGraph (long translationId, std::ostream &outputWordGraphStream) const

int GetNextHypoId ()

void OutputSearchGraph (long translationId, std::ostream &outputSearchGraphStream) const

void OutputSearchGraphAsSLF (long translationId, std::ostream &outputSearchGraphStream) const

void OutputSearchGraphAsHypergraph (long translationId, std::ostream &outputSearchGraphStream) const

void GetSearchGraph (std::vector< SearchGraphNode > &searchGraph) const

const InputType & GetSource () const

void CalcDecoderStatistics () const

void ResetSentenceStats (const InputType &source)

SentenceStats & GetSentenceStats () const

void GetForwardBackwardSearchGraph (std::map< int, bool > *pConnected, std::vector< const Hypothesis * > *pConnectedList, std::map< const Hypothesis *, std::set< const Hypothesis * > > *pOutgoingHyps, std::vector< float > *pFwdBwdScores) const

Public Attributes

InputType const & m_source

Protected Member Functions

void GetConnectedGraph (std::map< int, bool > *pConnected, std::vector< const Hypothesis * > *pConnectedList) const

void GetWinnerConnectedGraph (std::map< int, bool > *pConnected, std::vector< const Hypothesis * > *pConnectedList) const

Protected Attributes

TranslationOptionCollection * m_transOptColl

Search * m_search

HypothesisStack * actual_hypoStack

size_t interrupted_flag

std::auto_ptr< SentenceStats > m_sentenceStats

int m_hypoId

size_t m_lineNumber

Detailed Description

The Manager class implements a stack decoding algorithm for phrase-based decoding Hypotheses are organized in stacks. One stack contains all hypothesis that have the same number of foreign words translated. The data structure for hypothesis stacks is the class HypothesisStack. The data structure for a hypothesis is the class Hypothesis.

The main decoder loop in the function ProcessSentence() consists of the steps:

Create the list of possible translation options. In phrase-based decoding (and also the first mapping step in the factored model) is a phrase translation from the source to the target. Given a specific input sentence, only a limited number of phrase translation can be applied. For efficient lookup of the translation options later, these options are first collected in the function CreateTranslationOption (for more information check the class TranslationOptionCollection)
Create initial hypothesis: Hypothesis stack 0 contains only one empty hypothesis.
Going through stacks 0 ... (sentence_length-1):
- The stack is pruned to the maximum size
- Going through all hypotheses in the stack
  - Each hypothesis is expanded by ProcessOneHypothesis()
  - Expansion means applying a translation option to the hypothesis to create new hypotheses
  - What translation options may be applied depends on reordering limits and overlap with already translated words
  - With a applicable translation option and a hypothesis at hand, a new hypothesis can be created in ExpandHypothesis()
  - New hypothesis are either discarded (because they are too bad), added to the appropriate stack, or re-combined with existing hypotheses

Definition at line 94 of file Manager.h.

Constructor & Destructor Documentation

Moses::Manager::Manager	(	size_t	lineNumber,
		InputType const &	source,
		SearchAlgorithm	searchAlgorithm
	)

Definition at line 57 of file Manager.cpp.

References Moses::StaticData::InitializeForInput(), and Moses::StaticData::Instance().

Here is the call graph for this function:

Moses::Manager::~Manager ( )

Definition at line 68 of file Manager.cpp.

References Moses::StaticData::CleanUpAfterSentenceProcessing(), Moses::StaticData::Instance(), m_search, m_source, and m_transOptColl.

Here is the call graph for this function:

Member Function Documentation

void Moses::Manager::CalcDecoderStatistics ( ) const

Definition at line 406 of file Manager.cpp.

References Moses::SentenceStats::CalcFinalStats(), GetBestHypothesis(), Moses::Hypothesis::GetCurrSourceWordsRange(), Moses::Hypothesis::GetCurrTargetPhrase(), Moses::Hypothesis::GetInput(), Moses::Hypothesis::GetPrevHypo(), GetSentenceStats(), IFVERBOSE, NULL, and TRACE_ERR.

Here is the call graph for this function:

void Moses::Manager::CalcLatticeSamples	(	size_t	count,
		TrellisPathList &	ret
	)			const

Implements lattice sampling, as in Chatterjee & Cancedda, emnlp 2010

Definition at line 266 of file Manager.cpp.

References Moses::TrellisPathList::Add(), CHECK, Moses::Hypothesis::GetId(), Moses::Hypothesis::GetPrevHypo(), Moses::Hypothesis::GetScore(), GetSearchGraph(), and Moses::log_sum().

Here is the call graph for this function:

void Moses::Manager::CalcNBest	(	size_t	count,
		TrellisPathList &	ret,
		bool	onlyDistinct = `0`
	)			const

After decoding, the hypotheses in the stacks and additional arcs form a search graph that can be mined for n-best lists. The heavy lifting is done in the TrellisPath and TrellisPathCollection this function controls this for one sentence.

Parameters:

	count	the number of n-best translations to produce
	ret	holds the n-best list that was calculated

Definition at line 199 of file Manager.cpp.

References Moses::TrellisPathList::Add(), Moses::TrellisPathCollection::Add(), CHECK, Moses::TrellisPath::CreateDeviantPaths(), Moses::Search::GetHypothesisStacks(), Moses::StaticData::GetNBestFactor(), Moses::TrellisPathCollection::GetSize(), Moses::TrellisPathList::GetSize(), Moses::TrellisPath::GetSurfacePhrase(), Moses::StaticData::Instance(), m_search, NULL, Moses::TrellisPathCollection::pop(), and Moses::TrellisPathCollection::Prune().

Here is the call graph for this function:

const Hypothesis* Moses::Manager::GetActualBestHypothesis ( ) const

const Hypothesis * Moses::Manager::GetBestHypothesis ( ) const

Definition at line 1417 of file Manager.cpp.

References Moses::Search::GetBestHypothesis(), and m_search.

Referenced by CalcDecoderStatistics(), MosesCmd::doConsensusDecoding(), and MosesCmd::getLatticeMBRNBest().

Here is the call graph for this function:

Here is the caller graph for this function:

void Moses::Manager::GetConnectedGraph	(	std::map< int, bool > *	pConnected,
		std::vector< const Hypothesis * > *	pConnectedList
	)			const `[protected]`

Definition at line 1141 of file Manager.cpp.

References Moses::HypothesisStack::begin(), Moses::HypothesisStack::end(), Moses::Hypothesis::GetArcList(), Moses::Search::GetHypothesisStacks(), Moses::Hypothesis::GetId(), Moses::Hypothesis::GetPrevHypo(), m_search, and NULL.

Here is the call graph for this function:

void Moses::Manager::GetForwardBackwardSearchGraph	(	std::map< int, bool > *	pConnected,
		std::vector< const Hypothesis * > *	pConnectedList,
		std::map< const Hypothesis , std::set< const Hypothesis > > *	pOutgoingHyps,
		std::vector< float > *	pFwdBwdScores
	)			const

Referenced by MosesCmd::doConsensusDecoding(), and MosesCmd::getLatticeMBRNBest().

Here is the caller graph for this function:

int Moses::Manager::GetNextHypoId ( )

Definition at line 1422 of file Manager.cpp.

References m_hypoId.

void Moses::Manager::GetSearchGraph ( std::vector< SearchGraphNode > & searchGraph ) const

Referenced by CalcLatticeSamples(), OutputSearchGraph(), OutputSearchGraphAsHypergraph(), and OutputSearchGraphAsSLF().

Here is the caller graph for this function:

SentenceStats & Moses::Manager::GetSentenceStats ( ) const

Definition at line 1431 of file Manager.cpp.

References m_sentenceStats.

Referenced by Moses::HypothesisStackNormal::AddPrune(), Moses::HypothesisStackCubePruning::AddPrune(), CalcDecoderStatistics(), Moses::Hypothesis::CalcScore(), Moses::LanguageModelImplementation::Evaluate(), Moses::SearchNormalBatch::ExpandHypothesis(), Moses::SearchNormal::ExpandHypothesis(), Moses::Hypothesis::Hypothesis(), Moses::SearchNormalBatch::ProcessSentence(), Moses::SearchNormal::ProcessSentence(), Moses::SearchCubePruning::ProcessSentence(), ProcessSentence(), Moses::HypothesisStackNormal::PruneToSize(), and Moses::HypothesisStackCubePruning::PruneToSize().

Here is the caller graph for this function:

const TranslationOptionCollection * Moses::Manager::getSntTranslationOptions ( )

Definition at line 130 of file Manager.cpp.

References m_transOptColl.

const InputType& Moses::Manager::GetSource ( ) const [inline]

Definition at line 156 of file Manager.h.

References m_source.

Referenced by Moses::HypothesisStackCubePruning::SetBitmapAccessor().

Here is the caller graph for this function:

void Moses::Manager::GetWinnerConnectedGraph	(	std::map< int, bool > *	pConnected,
		std::vector< const Hypothesis * > *	pConnectedList
	)			const `[protected]`

Definition at line 1184 of file Manager.cpp.

References Moses::HypothesisStack::begin(), Moses::HypothesisStack::end(), Moses::Hypothesis::GetArcList(), Moses::Search::GetHypothesisStacks(), Moses::Hypothesis::GetId(), Moses::Hypothesis::GetPrevHypo(), m_search, and NULL.

Here is the call graph for this function:

void Moses::Manager::GetWordGraph	(	long	translationId,
		std::ostream &	outputWordGraphStream
	)			const

Definition at line 521 of file Manager.cpp.

References Moses::HypothesisStack::begin(), Moses::HypothesisStack::end(), Moses::Hypothesis::GetArcList(), Moses::Search::GetHypothesisStacks(), Moses::StaticData::GetParam(), Moses::StaticData::Instance(), m_search, NULL, Moses::OutputWordGraph(), and Moses::Scan< bool >().

Here is the call graph for this function:

void Moses::Manager::OutputSearchGraph	(	long	translationId,
		std::ostream &	outputSearchGraphStream
	)			const

Definition at line 1330 of file Manager.cpp.

References GetSearchGraph(), and Moses::OutputSearchNode().

Here is the call graph for this function:

void Moses::Manager::OutputSearchGraphAsHypergraph	(	long	translationId,
		std::ostream &	outputSearchGraphStream
	)			const

! Output search graph in hypergraph format of Kenneth Heafield's lazy hypergraph decoder

Definition at line 833 of file Manager.cpp.

References count, Moses::Hypothesis::GetCurrTargetPhrase(), Moses::Hypothesis::GetId(), Moses::Hypothesis::GetPrevHypo(), GetSearchGraph(), Moses::Phrase::GetSize(), Moses::Phrase::GetWord(), NULL, UTIL_THROW_IF, and VERBOSE.

Here is the call graph for this function:

void Moses::Manager::OutputSearchGraphAsSLF	(	long	translationId,
		std::ostream &	outputSearchGraphStream
	)			const

! Output search graph in HTK standard lattice format (SLF)

Definition at line 982 of file Manager.cpp.

References Moses::Hypothesis::GetCurrTargetPhrase(), Moses::Hypothesis::GetId(), Moses::Hypothesis::GetPrevHypo(), GetSearchGraph(), Moses::Phrase::GetSize(), and Moses::Phrase::GetWord().

Here is the call graph for this function:

void Moses::Manager::PrintAllDerivations	(	long	translationId,
		std::ostream &	outputStream
	)			const

void Moses::Manager::printDivergentHypothesis	(	long	translationId,
		const Hypothesis *	hypo,
		const std::vector< const TargetPhrase * > &	remainingPhrases,
		float	remainingScore,
		std::ostream &	outputStream
	)			const

void Moses::Manager::printThisHypothesis	(	long	translationId,
		const Hypothesis *	hypo,
		const std::vector< const TargetPhrase * > &	remainingPhrases,
		float	remainingScore,
		std::ostream &	outputStream
	)			const

void Moses::Manager::ProcessSentence ( )

Main decoder loop that translates a sentence by expanding hypotheses stack by stack, until the end of the sentence.

Definition at line 80 of file Manager.cpp.

References Moses::SentenceStats::AddTimeCollectOpts(), Moses::TranslationOptionCollection::CreateTranslationOptions(), Moses::Timer::get_elapsed_time(), GetSentenceStats(), IFVERBOSE, m_lineNumber, m_search, m_source, m_transOptColl, Moses::Search::ProcessSentence(), ResetSentenceStats(), Moses::Timer::start(), and VERBOSE.

Here is the call graph for this function:

void Moses::Manager::ResetSentenceStats ( const InputType & source )

Definition at line 1427 of file Manager.cpp.

References m_sentenceStats.

Referenced by ProcessSentence().

Here is the caller graph for this function:

Member Data Documentation

HypothesisStack* Moses::Manager::actual_hypoStack [protected]

Definition at line 118 of file Manager.h.

size_t Moses::Manager::interrupted_flag [protected]

actual (full expanded) stack of hypotheses

Definition at line 119 of file Manager.h.

int Moses::Manager::m_hypoId [protected]

Definition at line 121 of file Manager.h.

Referenced by GetNextHypoId().

size_t Moses::Manager::m_lineNumber [protected]

Definition at line 122 of file Manager.h.

Referenced by ProcessSentence().

Search* Moses::Manager::m_search [protected]

Definition at line 116 of file Manager.h.

Referenced by CalcNBest(), GetBestHypothesis(), GetConnectedGraph(), GetWinnerConnectedGraph(), GetWordGraph(), ProcessSentence(), and ~Manager().

std::auto_ptr<SentenceStats> Moses::Manager::m_sentenceStats [protected]

Definition at line 120 of file Manager.h.

Referenced by GetSentenceStats(), and ResetSentenceStats().

InputType const& Moses::Manager::m_source

source sentence to be translated

Definition at line 133 of file Manager.h.

Referenced by GetSource(), ProcessSentence(), and ~Manager().

TranslationOptionCollection* Moses::Manager::m_transOptColl [protected]

pre-computed list of translation options for the phrases in this sentence

Definition at line 115 of file Manager.h.

Referenced by getSntTranslationOptions(), ProcessSentence(), and ~Manager().

The documentation for this class was generated from the following files:

mosesdecoder/moses/Manager.h
mosesdecoder/moses/Manager.cpp


Public Member Functions
	Manager (size_t lineNumber, InputType const &source, SearchAlgorithm searchAlgorithm)
	~Manager ()
const TranslationOptionCollection *	getSntTranslationOptions ()
void	ProcessSentence ()
const Hypothesis *	GetBestHypothesis () const
const Hypothesis *	GetActualBestHypothesis () const
void	CalcNBest (size_t count, TrellisPathList &ret, bool onlyDistinct=0) const
void	CalcLatticeSamples (size_t count, TrellisPathList &ret) const
void	PrintAllDerivations (long translationId, std::ostream &outputStream) const
void	printDivergentHypothesis (long translationId, const Hypothesis hypo, const std::vector< const TargetPhrase > &remainingPhrases, float remainingScore, std::ostream &outputStream) const
void	printThisHypothesis (long translationId, const Hypothesis hypo, const std::vector< const TargetPhrase > &remainingPhrases, float remainingScore, std::ostream &outputStream) const
void	GetWordGraph (long translationId, std::ostream &outputWordGraphStream) const
int	GetNextHypoId ()
void	OutputSearchGraph (long translationId, std::ostream &outputSearchGraphStream) const
void	OutputSearchGraphAsSLF (long translationId, std::ostream &outputSearchGraphStream) const
void	OutputSearchGraphAsHypergraph (long translationId, std::ostream &outputSearchGraphStream) const
void	GetSearchGraph (std::vector< SearchGraphNode > &searchGraph) const
const InputType &	GetSource () const
void	CalcDecoderStatistics () const
void	ResetSentenceStats (const InputType &source)
SentenceStats &	GetSentenceStats () const
void	GetForwardBackwardSearchGraph (std::map< int, bool > pConnected, std::vector< const Hypothesis > pConnectedList, std::map< const Hypothesis , std::set< const Hypothesis * > > pOutgoingHyps, std::vector< float > pFwdBwdScores) const
Public Attributes
InputType const &	m_source
Protected Member Functions
void	GetConnectedGraph (std::map< int, bool > pConnected, std::vector< const Hypothesis > *pConnectedList) const
void	GetWinnerConnectedGraph (std::map< int, bool > pConnected, std::vector< const Hypothesis > *pConnectedList) const
Protected Attributes
TranslationOptionCollection *	m_transOptColl
Search *	m_search
HypothesisStack *	actual_hypoStack
size_t	interrupted_flag
std::auto_ptr< SentenceStats >	m_sentenceStats
int	m_hypoId
size_t	m_lineNumber