tag:blogger.com,1999:blog-11295132.post751712815454057762..comments2019-09-09T05:26:56.880-07:00Comments on A Neighborhood of Infinity: Expectation-Maximization with Less Arbitrarinesssigfpehttp://www.blogger.com/profile/08096190433222340957noreply@blogger.comBlogger3125tag:blogger.com,1999:blog-11295132.post-7554430494501872992016-12-22T08:08:09.667-08:002016-12-22T08:08:09.667-08:00Hi Dan... I recently came across an old article by...Hi Dan... I recently came across an old article by you (from 2005) and am responding here instead of there (thinking you might not see it if I add this to an old blog post.) Anyway I came across your article b/c I'm searching for some very specific things, which I didn't find but thought you might be able to point me to...<br /><br />I am not well-versed in logic, but I am wanting to find out if any of the formal alternative logics include a phenomenon where both A is greater than B (or, A includes B) AND, B is greater than A (or B includes A). This is something that Sir Geoffrey Vickers, in Value Systems & Social Process, describes as "chinese boxes"... and his example is that, from one perspective, science about human beings, is just one aspect of a much larger field of science.... whereas from another perspective, 'doing science' is just one part of what human beings do. <br /><br />Anyway, I've come across that kind of relationship before, and am curious whether there is any kind of formal logic that explores that...<br /><br />My second question: from the field of group facilitation... groups often get stuck in a polarity of "either A OR B", which we might depict as a line that includes A at one end and B at the other.... One way of expanding the conversation has been described as "exploring the emergent axis" which could be described as another line, that includes BOTH A and B as well as NEITHER A nor B... <br /><br />and of course when there are two lines, in Euclidian geometry that defines a whole larger plane, so it greatly expands the field of possibilities under consideration...<br /><br />just curious where I might look, for any work along these lines... thanks so much!<br /> Rosa Z.https://www.blogger.com/profile/06865766788759680875noreply@blogger.comtag:blogger.com,1999:blog-11295132.post-89324321688695729372016-10-18T05:05:36.893-07:002016-10-18T05:05:36.893-07:00Thank you for sharing this insightful observation!...Thank you for sharing this insightful observation!<br /><br />There is a small typo in the formula for the linear approximation. The derivative has to be multiplied by $(x-x_0)$, not by $x$.Ingo Blechschmidthttps://www.blogger.com/profile/10091270490109382584noreply@blogger.comtag:blogger.com,1999:blog-11295132.post-8711139297522023262016-10-17T09:41:03.709-07:002016-10-17T09:41:03.709-07:00Although I use the example of steepest ascent (or ...Although I use the example of steepest ascent (or descent) to motivate EM, there's an interesting difference pointed out to me by a work colleague.<br /><br />When using steepest ascent you're using the fact that the linear proxy function matches the original function in a small region. So when you maximise the proxy you need to perform a maximisation in a small region. This is essentially why we typically take small step sizes in the steepest ascent algorithm. This means that steepest ascent can get stuck in local minima.<br /><br />In the case of EM we similarly ensure that the proxy matches the true objective locally in a small region. However, the concavity of the log function means that the proxy is always less than or equal to the original function. As a result, we don't have to be conservative. Globally maximising the proxy is guaranteed to be safe. Because EM isn't restricted to small steps it can sometimes make big jumps from one local maximum to another. That doesn't mean it'll always find the global maximum of your likelihood. But it is a qualitative difference from steepest ascent.<br /><br />(Pure Newton-Raphson can also make big jumps. But, unmodified, it's not always a good algorithm because there are no guarantees that the quadratic proxy is always less than the true objective function.)Dan Piponihttps://www.blogger.com/profile/08096190433222340957noreply@blogger.com