[CS241] Asgn2 questions

[Lenora Huang]

Hi, Justin.
I can understand that  0 * log(0) = 0. But as I compute, there's no
p(tense)=0 case, but lots of conditional p = 0.
So if the KL order is p(tense)|| p(tense| tense in prev sent) , there
will be (non-zero)* log(0) case ?
However , this won't be a problem if we reverser the order to KL( p
(tense | tense in prev sent) || p (tense) ).
Is this correct?

Thanks,
Lenora
 
Justin Palmer wrote:
> Hi Lenora,
>
> Here's my take on your questions.  I might be wrong...
>
> For KL divergence, yes, order matters.  D(p || q) != D(q || p).
> That's why KL divergence is not a distance/metric.  So, I'm using the
> order Eugene asked for in the assignment, e.g, D( p(tense) || p(tense
> | tense in prev sent) ).  Regarding the log(0) issue, I ran into it
> too; my understanding is that for entropy calculations, we define 0 *
> log 0 = 0.
>
> To compute D( p(tense) || p(tense|tense in prev sent) ), I did:
>
>    KL += p(tense) * log (p(tense) / p(tense | tense in prev sent))
>
> for all tenses.  So we get 2 numbers saying how useful the two
> conditional probability distributions are as predictors of tense.
> Does that make sense?
>
> I understand part d to mean calculate p(tense | tense any but prev
> sent).  I'm computing:
>
>   P(past | past), P(future | past), etc.
>
> So if you're at sentence i, and it's past, and a past tense also
> occurred in any of the previous sentences other than the last one,
> that counts.  Also, if a future occurred in any sentence other than
> the previous sentence, add another count.  And so on.
>
> If you'd like to to compare numbers, please let me know.
>
> Also, is anyone else having problems with the Tree::newstory flag?
> Appears that it never gets set, but I'm probably missing something
> obvious.
>
> Thanks,
>
>   -- j
>