Readings for next week are now posted. Although the readings are not ordered, the order listed is probably the best one. Make sure your respond with your comments by 9/25 at noon. A few " goodies " follow: a humorous cartoon about curve fitting, and an example of boosting. The following example of boosting is taken from here . I've removed the code and some of the math below to make it easier to follow. 1. Start with some data. We use the normal distribution. 2. Fit an initial model. Here we fit a line. You could view this as an intercept-only line or a line with a slope of zero. In either case, it's not hard to see that the residuals are balanced above and below the line, which is what line fitting is supposed to do. 3. Fit a model on the residuals from step 2 (step 2 residuals). This time we use a decision stump , which is a decision tree with one branch point. This is the stump we fit: As you can see, any x (the horizontal axis) less than
Quinn et al. (2010)
ReplyDeleteI found this article to be much easier to read based off of our readings regarding LSI last week. This concept seemed very similar if not the same. I find it interesting that the post analysis phase requires more time and effort. I think I am a little confused about how that phase works. The article discusses stemming and removing most punctuation and capitalization which was one of our discussion points last week. I am glad this was clarified because that seems that it would make a large difference in senate speeches. I found the figures in this article to be of particular interest/use. The clustering in figure one helped my overall understanding while part A of Figure 3 was particularly interesting to me. Although this process has a minimal startup value and more investment in the post analysis, I think overall it seems very useful over human coders, dictionaries, and supervised learning. In particular, the authors claim that this model can “track in detail the dynamics by which issues and frames are adopted by parties, absorbed into existing ideologies, or disrupt the nature of party competition (p. 226).” This seems of particular use to track parties over time. The main question I have is who uses this information and for what purpose other than just to have general knowledge?
Baroni et al. (2014)
This article was a little harder for me to understand than the previous article; although, I did enjoy the rhetoric and commentary of the authors. For example, the authors say that “our secret wish was to discover that it is all hype, and count vectors are far superior to their predictive counterparts (p.244).” This is not what the authors found. On the contrary, they found the opposite. The predict models are better. I was glad that the authors said this could not be contributed to count models that were not very good but, rather, that the count models used were good so this provided an accurate picture of comparison.
Quinn et al. (2010):
ReplyDeleteThere are many pragmatic future possibilities for this kind of research. Although this research was mostly classificatory, the next step would be predictive. This could include everything from gathering intelligence on and seeking to influence another state, to doing that internally. Not only does it provide a way to model the legislative and representational process, but it could be expanded to “evaluate the positioning of individual legislators” (p. 226). The authors perhaps hint towards its intelligence possibilities when they mention “our method travels beyond English and beyond the Congressional setting” (ibid.). This also means that foreign entities could equally model and seek to influence politics using machine learning. As the threat of cyberwarfare becomes less sci-fi and more real, this adds a new dimension. Just like modeling crime allows for manipulation (reduction) of it, this could allow for better manipulation of a state’s political mechanisms. What I’m wondering is, what is the future of predictive intelligence, and what role does machine learning play in that?
Baroni et al. (2014):
This is a comparison of count and predictive distributional semantic models. I’d be grateful if someone could explain to me in which ways this is related to (or differs from) the LSI or LSA that we were talking about the other week?
Quin et al. (2010)
ReplyDeleteI am just getting introduced meaningfully to text-analysis and think the table on p. 211 is interesting in portraying the trade-offs involved. I was also very impressed by their results. The topics and their hierarchical relationships were very interesting and seem to reflect a useful breakdown of the types of topics I imagine they speak about, as well as a useful ability to read beyond the bureaucratic double-speak like “restrictions on use of Department of Defense medical facilities” to refer to oversees abortions for military personnel (p. 222). I am curious how this model might generalize to less formal venues for discussion. That is, the congressional meetings would likely contain more formal discussion of these topics, largely delineated temporally (e.g., abortion discussion is more likely to occur as a result of previous discussions and current attention), and probably contain more rehearsed and prototypical language (potentially even shared speechwriters). Could something like this method be useful in categorizing, say, a meandering political discussion among friends? What about online forums, Twitter, or comments sections of articles?
Baroni et al. (2014)
One idea that came to mind was regarding their semantic relatedness comparisons. Given that LSA and similar features are purported to somewhat approximate the interrelatedness of words similar to their representation in the human brain (though they are clear in stating that it is not trying to mimic this, just that it is a similar idea), it would be interesting to compare some of these approaches to the more implicit connections of human representation. One example may be to compare the approaches’ performance against human performance on word-stem completion tasks, word/non-word judgments, or other forms of priming studies from the connectionist/spreading activation literature. While we may be familiar with many words and understand them well, there may be less conscious, more nuanced relationships that we aren’t able to introspectively pick up on.
That is, while someone may struggle to numerically compare garden and unicorn, there may be a more meaningful semantic relationship than a rating of 0 (e.g., garden => lawn gnome => gnome/mythical creature => unicorn). Similarly, are mother/father (likely to be rated high on similarity) quantitatively related to the same level as table/chair (also likely to be highly rated)?
As a side note, I greatly appreciate how frank their discussion was (i.e., that they intended this article to show that count vectors were superior).
Quinn et al. (2010):
ReplyDeleteReading the article on LSI last week laid a great base foundation for this particular article. I think I would have understood less if we had not discussed our topic from last week. I thought the domain was extremely interesting and I really enjoyed all the diagrams and models. I think it helped me understand the process more, seeing a visual. I am curious as to what other domains this type of modeling could attend to? Or is it specific to politics-geared analyses? It seemed to me that there is lots more that could be done with this type of processing.
Baroni et al. (2014):
So the results that were rendered from this model comparison were interesting to me. With obtaining results that were quite different than what were expected, I wonder if model comparison should be implemented more? If it would even be beneficial? the response to their findings came off as a positive thing to me. I also think it was important that they recognized the vast room for error, I think it would be helpful to further explore these in hopes to gain even more insight into their research. By the end of the article in the conclusion, it definitely appeared that there is a lot of room for future work in this particular area, which is neat.
The Quin et al., article showed a really cool and imaginative way to use LSI and the types of analyses that we talked about last week. I think one of the most interesting speculations the authors touched on was the potential ability to analyze the ways in which political parties adopt and frame certain topics. Sometimes it seems like the two parties speak in slightly different languages, I wonder if this kind of analysis could demonstrate how one word thematic to one topic is framed or means something slightly different according to each party. As a basic and not altogether accurate example, I wonder if there are significant differences between parties’ tendencies to follow the word “freedom” with either “from” or “to.” I know that’s put in a non-predictive way, but I bet you could use these analyses to derive some other predictive information about other themes based on these differences.
ReplyDeleteI really appreciated the rundown given by Baroni et al., about the state of the art in distributional semantics. I have questions about the parameters used in the models that were tested. How are the window size and dimensionality parameters related? I understand what they are, more or less, but why is a larger window and more dimensions not almost always better, especially when seeking to predict within contexts that I would guess require more than 2 words on either side of a target.
Quinn et al. (2010):
ReplyDeleteI thought this article showed a really interesting real life example of how some of these ideas can be used. I like the application to politics, and I think it could be really helpful with much of the dense literature on that topic. Since a lot of political discussion in today's world seems to be done via tweets and Facebook posts, I wonder if this method could be used in that way. And if so, would it be thrown off by typos or other errors? Would changing those typos when processing the texts alter the meaning of the texts in any way?
Baroni et al. (2014)
This article was also interesting, although harder for me to understand than the other article. I appreciated that they admitted how wrong they were in how useful they thought the method was going to be. I wonder if words that have fairly different meanings in different contexts (i.e. crane, date, etc) would have any affect on the outcome?
Quinn et al. (2010)
ReplyDeleteThis article summarized and compared different methods of analyzing the substance of political attention. Even the topic model requires users spend more time interpreting and validating
the results, it seems an attractive tool which can be applied to very large corpora. We talked about LSA last week. I am wondering whether we can use LSA to do the analysis in this paper. Which one is better?
Baroni et al. (2014)
I searched Wiki and found that LSA is one kind of distributional semantic models. Is LSA one kind of the context-counting models? The paper found that context-predicting models are better. Does this mean that we should use context-predicting models not context-counting models?
Quinn 2010:
ReplyDeleteI thought the application of LSI in a field like politics, which is completely different than anything we have talked about so far, is neat. It reminds me of the most recent presidential election, when I saw some statistics regarding the candidates responses to questions, including how many times they cut each other's sentences off. This article looks at political speeches, rather than debate answers like my example, but I think the entire concept is pretty interesting and I would be curious to see it applied to other types of speeches, too. Something like a paired test, comparing two+ speeches on the same type of material or topic, but given by two+ different political view points would be interesting to look at. I also think what John said about analyzing the particular language each political party uses would be interesting as well. I don't really know anything about text analysis, but having a predictive model seems like it could be a useful tool for researchers to have.
Baroni 2014:
I feel like the most prior exposure I've had to this type of work is count models, and having it be compared to the predictive models was helpful. I think the conclusions and caveats of this paper serve to remind us that no predictive model is actually perfect and there are always a lot of areas to improve on and questions to answer.
Quinn
ReplyDeleteI like that the authors of this paper are focusing on something as important as the Senate. I liked how exhaustive their clarifying notes and the list of issues analyzed. I like that they looked at the Iraq War and 9/11 as these were two very influential events in our history. The figures were helpful and informative particularly how they showed the words per month on the military aligned with war or terrorist attacks. This method is useful when examining a large corpora of political texts, but what other methods are there?
Baroni
The Baroni 2014 paper evaluates context predicting models of word meaning. Why do co-occurrence techniques not work as well as DSM vector approaches? They looked at synonyms, context, and analogies to evaluate their model. It will be interesting to see in the future if the two models are complementary, but I liked that the authors admitted that they were hoping to overturn the “hype” and told us about their biases.
Quinn et al 2010
ReplyDeleteThis was a very nice extensive after learning about LSA last week. I have had brief contact with topic modeling before, but never actually fully understood it. They mentioned tradeoffs for many of the methods of text analysis such as deep reading, human-coding, dictionaries an supervised learning, and I know they mentioned that topic modeling forgoes much of the extreme cost at the front end of preparation, but it incurs a cost in the postanalysis stage. It wasn’t exactly clear exactly how large that cost is compared to the front-end cost of methods like dictionaries and supervised learning. That being said 100% intercoder reliability is certainly a very nice advantage. I was also wondering what other kinds of applications would this topic modeling be good for? I would think similar to last week’s meta-analysis using LSI, topic modeling could be useful for certain types of meta-analyses.
Baroni et al 2014
I certainly like that predictive strategies seem to have won out here. Although, I still don’t quite understand how exactly they work. The count-models seem pretty straight forward, using vectors of word counts, but also even for the co-occurrence counts, I would like to know further how they’re different than LSA. All of these models obviously take context into account which seems to make them very different than LSA. Also, I wonder how these predictive models could be used to analyze documents similar to how topic modeling does it and would it be an improvement?