Moses::LanguageModel Class Reference

Abstract base class which represent a language model on a contiguous phrase. More...

#include <LanguageModel.h>

Inheritance diagram for Moses::LanguageModel:

[legend]

Collaboration diagram for Moses::LanguageModel:

[legend]

List of all members.

Public Types
typedef const void *	State
Public Member Functions
virtual	~LanguageModel ()
size_t	GetNumScoreComponents () const
	see ScoreProducer.h
virtual LMType	GetLMType () const =0
	Single or multi-factor.
virtual bool	Useable (const Phrase &phrase) const =0
void	CalcScore (const Phrase &phrase, float &fullScore, float &ngramScore) const
void	CalcScoreChart (const Phrase &phrase, float &beginningBitsOnly, float &ngramScore) const
virtual float	GetValue (const std::vector< const Word * > &contextFactor, State *finalState=0, unsigned int *len=0) const =0
State	GetState (const std::vector< const Word * > &contextFactor, unsigned int *len=0) const
	get State for a particular n-gram
size_t	GetNGramOrder () const
	max n-gram order of LM
const Word &	GetSentenceStartArray () const
	Contains factors which represents the beging and end words for this LM. Usually <s> and </s>.
const Word &	GetSentenceEndArray () const
float	GetWeight () const
	scoring weight. Shouldn't this now be superceded by ScoreProducer???
void	SetWeight (float weight)
virtual std::string	GetScoreProducerDescription () const =0
	returns a string description of this producer
std::string	GetScoreProducerWeightShortName () const
	returns the weight parameter name of this producer (used in n-best list)
virtual void	InitializeBeforeSentenceProcessing ()
	overrideable funtions for IRST LM to cleanup. Maybe something to do with on demand/cache loading/unloading
virtual void	CleanUpAfterSentenceProcessing ()
virtual const FFState *	EmptyHypothesisState (const InputType &input) const
	return the state associated with the empty hypothesis for a given sentence
virtual FFState *	Evaluate (const Hypothesis &cur_hypo, const FFState *prev_state, ScoreComponentCollection *accumulator) const
	This interface should be implemented. Notes: When evaluating the value of this feature function, you should avoid calling hypo.GetPrevHypo(). If you need something from the "previous" hypothesis, you should store it in an FFState object which will be passed in as prev_state. If you don't do this, you will get in trouble.
Protected Member Functions
void	ShiftOrPush (std::vector< const Word * > &contextFactor, const Word &word) const
	Usually <s> and </s>.
	LanguageModel (bool registerScore, ScoreIndexManager &scoreIndexManager)
Protected Attributes
float	m_weight
std::string	m_filePath
	scoring weight. Shouldn't this now be superceded by ScoreProducer???
size_t	m_nGramOrder
	for debugging purposes
Word	m_sentenceStartArray
	max n-gram length contained in this LM
Word	m_sentenceEndArray

---

Detailed Description

Abstract base class which represent a language model on a contiguous phrase.

---

Member Typedef Documentation

typedef const void* Moses::LanguageModel::State

---

Constructor & Destructor Documentation

Moses::LanguageModel::LanguageModel	(	bool	registerScore,
		ScoreIndexManager &	scoreIndexManager
	)			[protected]

constructor to be called by inherited class

Parameters:

registerScore

whether this LM will be directly used to score sentence. Usually true, except where LM is a component in a composite LM, eg. LanguageModelJoint

References Moses::ScoreIndexManager::AddScoreProducer().

Here is the call graph for this function:

Moses::LanguageModel::~LanguageModel

(

)

[virtual]

---

Member Function Documentation

void Moses::LanguageModel::CalcScore	(	const Phrase &	phrase,
		float &	fullScore,
		float &	ngramScore
	)			const

References GetSentenceStartArray(), Moses::Phrase::GetSize(), GetValue(), Moses::Phrase::GetWord(), Moses::Word::IsNonTerminal(), m_nGramOrder, and ShiftOrPush().

Referenced by Moses::LMList::CalcScore().

Here is the call graph for this function:

Here is the caller graph for this function:

void Moses::LanguageModel::CalcScoreChart	(	const Phrase &	phrase,
		float &	beginningBitsOnly,
		float &	ngramScore
	)			const

References GetSentenceStartArray(), Moses::Phrase::GetSize(), GetValue(), Moses::Phrase::GetWord(), Moses::Word::IsNonTerminal(), m_nGramOrder, and ShiftOrPush().

Referenced by Moses::LMList::CalcAllLMScores().

Here is the call graph for this function:

Here is the caller graph for this function:

virtual void Moses::LanguageModel::CleanUpAfterSentenceProcessing

(

)

[inline, virtual]

Reimplemented in Moses::LanguageModelIRST, and Moses::LanguageModelRandLM.

Referenced by Moses::StaticData::CleanUpAfterSentenceProcessing().

Here is the caller graph for this function:

const FFState * Moses::LanguageModel::EmptyHypothesisState

(

const InputType &

input

)

const [virtual]

return the state associated with the empty hypothesis for a given sentence

Implements Moses::StatefulFeatureFunction.

References NULL.

FFState * Moses::LanguageModel::Evaluate	(	const Hypothesis &	cur_hypo,
		const FFState *	prev_state,
		ScoreComponentCollection *	accumulator
	)			const [virtual]

This interface should be implemented. Notes: When evaluating the value of this feature function, you should avoid calling hypo.GetPrevHypo(). If you need something from the "previous" hypothesis, you should store it in an FFState object which will be passed in as prev_state. If you don't do this, you will get in trouble.

Implements Moses::StatefulFeatureFunction.

References Moses::SentenceStats::AddTimeCalcLM(), Moses::Hypothesis::GetCurrTargetLength(), Moses::Hypothesis::GetCurrTargetWordsRange(), Moses::WordsRange::GetEndPos(), Moses::Hypothesis::GetManager(), GetSentenceEndArray(), GetSentenceStartArray(), Moses::Manager::GetSentenceStats(), Moses::Hypothesis::GetSize(), Moses::WordsRange::GetStartPos(), GetState(), GetValue(), Moses::Hypothesis::GetWord(), IFVERBOSE, Moses::Hypothesis::IsSourceCompleted(), Moses::LMState::lmstate, m_nGramOrder, NULL, and Moses::ScoreComponentCollection::PlusEquals().

Here is the call graph for this function:

virtual LMType Moses::LanguageModel::GetLMType

(

)

const [pure virtual]

Single or multi-factor.

Implemented in Moses::LanguageModelMultiFactor, and Moses::LanguageModelSingleFactor.

Referenced by Moses::LanguageModelFactory::CreateLanguageModel().

Here is the caller graph for this function:

size_t Moses::LanguageModel::GetNGramOrder

(

)

const [inline]

max n-gram order of LM

References m_nGramOrder.

Referenced by Moses::LMList::Add(), Moses::LanguageModelSingleFactor::GetScoreProducerDescription(), and Moses::LanguageModelMultiFactor::GetScoreProducerDescription().

Here is the caller graph for this function:

size_t Moses::LanguageModel::GetNumScoreComponents

(

)

const [virtual]

see ScoreProducer.h

Implements Moses::ScoreProducer.

virtual std::string Moses::LanguageModel::GetScoreProducerDescription

(

)

const [pure virtual]

returns a string description of this producer

Implements Moses::ScoreProducer.

Implemented in Moses::LanguageModelMultiFactor, and Moses::LanguageModelSingleFactor.

std::string Moses::LanguageModel::GetScoreProducerWeightShortName

(

)

const [inline, virtual]

returns the weight parameter name of this producer (used in n-best list)

Implements Moses::ScoreProducer.

const Word& Moses::LanguageModel::GetSentenceEndArray

(

)

const [inline]

References m_sentenceEndArray.

Referenced by Evaluate().

Here is the caller graph for this function:

const Word& Moses::LanguageModel::GetSentenceStartArray

(

)

const [inline]

Contains factors which represents the beging and end words for this LM. Usually <s> and </s>.

References m_sentenceStartArray.

Referenced by CalcScore(), CalcScoreChart(), and Evaluate().

Here is the caller graph for this function:

LanguageModel::State Moses::LanguageModel::GetState	(	const std::vector< const Word * > &	contextFactor,
		unsigned int *	len = 0
	)			const

get State for a particular n-gram

References GetValue().

Referenced by Evaluate().

Here is the call graph for this function:

Here is the caller graph for this function:

virtual float Moses::LanguageModel::GetValue	(	const std::vector< const Word * > &	contextFactor,
		State *	finalState = 0,
		unsigned int *	len = 0
	)			const [pure virtual]

Implemented in Moses::LanguageModelInternal, Moses::LanguageModelIRST, Moses::LanguageModelJoint, Moses::LanguageModelParallelBackoff, Moses::LanguageModelRandLM, Moses::LanguageModelRemote, Moses::LanguageModelSkip, and Moses::LanguageModelSRI.

Referenced by CalcScore(), CalcScoreChart(), Evaluate(), GetState(), Moses::LanguageModelSkip::GetValue(), and Moses::LanguageModelJoint::GetValue().

Here is the caller graph for this function:

float Moses::LanguageModel::GetWeight

(

)

const [inline]

scoring weight. Shouldn't this now be superceded by ScoreProducer???

Reimplemented in Moses::LanguageModelSingleFactor.

References m_weight.

Referenced by Moses::LMList::CalcScore().

Here is the caller graph for this function:

virtual void Moses::LanguageModel::InitializeBeforeSentenceProcessing

(

)

[inline, virtual]

overrideable funtions for IRST LM to cleanup. Maybe something to do with on demand/cache loading/unloading

Reimplemented in Moses::LanguageModelIRST, and Moses::LanguageModelRandLM.

Referenced by Moses::StaticData::InitializeBeforeSentenceProcessing().

Here is the caller graph for this function:

void Moses::LanguageModel::SetWeight

(

float

weight

)

[inline]

Reimplemented in Moses::LanguageModelSingleFactor.

References m_weight.

void Moses::LanguageModel::ShiftOrPush	(	std::vector< const Word * > &	contextFactor,
		const Word &	word
	)			const [protected]

Usually <s> and </s>.

Contains factors which represents the beging and end words for this LM.

Referenced by CalcScore(), and CalcScoreChart().

Here is the caller graph for this function:

virtual bool Moses::LanguageModel::Useable

(

const Phrase &

phrase

)

const [pure virtual]

Implemented in Moses::LanguageModelMultiFactor, and Moses::LanguageModelSingleFactor.

Referenced by Moses::LMList::CalcAllLMScores(), and Moses::LMList::CalcScore().

Here is the caller graph for this function:

---

Member Data Documentation

std::string Moses::LanguageModel::m_filePath [protected]

scoring weight. Shouldn't this now be superceded by ScoreProducer???

Referenced by Moses::LanguageModelMultiFactor::GetScoreProducerDescription(), Moses::LanguageModelSRI::Load(), Moses::LanguageModelSkip::Load(), Moses::LanguageModelRandLM::Load(), Moses::LanguageModelParallelBackoff::Load(), Moses::LanguageModelJoint::Load(), Moses::LanguageModelIRST::Load(), and Moses::LanguageModelInternal::Load().

size_t Moses::LanguageModel::m_nGramOrder [protected]

for debugging purposes

Referenced by CalcScore(), CalcScoreChart(), Evaluate(), GetNGramOrder(), Moses::LanguageModelRemote::GetValue(), Moses::LanguageModelJoint::GetValue(), Moses::LanguageModelSRI::Load(), Moses::LanguageModelSkip::Load(), Moses::LanguageModelRemote::Load(), Moses::LanguageModelRandLM::Load(), Moses::LanguageModelParallelBackoff::Load(), Moses::LanguageModelJoint::Load(), Moses::LanguageModelIRST::Load(), and Moses::LanguageModelInternal::Load().

Word Moses::LanguageModel::m_sentenceEndArray [protected]

Referenced by Moses::LanguageModelSRI::CreateFactors(), Moses::LanguageModelRandLM::CreateFactors(), Moses::LanguageModelParallelBackoff::CreateFactors(), Moses::LanguageModelIRST::CreateFactors(), GetSentenceEndArray(), Moses::LanguageModelParallelBackoff::GetValue(), Moses::LanguageModelSkip::Load(), Moses::LanguageModelJoint::Load(), and Moses::LanguageModelInternal::Load().

Word Moses::LanguageModel::m_sentenceStartArray [protected]

max n-gram length contained in this LM

Referenced by Moses::LanguageModelSRI::CreateFactors(), Moses::LanguageModelRandLM::CreateFactors(), Moses::LanguageModelParallelBackoff::CreateFactors(), Moses::LanguageModelIRST::CreateFactors(), GetSentenceStartArray(), Moses::LanguageModelParallelBackoff::GetValue(), Moses::LanguageModelSkip::Load(), Moses::LanguageModelJoint::Load(), and Moses::LanguageModelInternal::Load().

float Moses::LanguageModel::m_weight [protected]

Referenced by Moses::LanguageModelSingleFactor::GetWeight(), GetWeight(), Moses::LanguageModelSRI::Load(), Moses::LanguageModelSkip::Load(), Moses::LanguageModelRemote::Load(), Moses::LanguageModelRandLM::Load(), Moses::LanguageModelParallelBackoff::Load(), Moses::LanguageModelJoint::Load(), Moses::LanguageModelIRST::Load(), Moses::LanguageModelInternal::Load(), Moses::LanguageModelSingleFactor::SetWeight(), and SetWeight().

---

The documentation for this class was generated from the following files:

• moses/src/LanguageModel.h
• moses/src/LanguageModel.cpp