Wort des Tages: Begehungskeks
"Software Development" xkcd.com/2021/ #lustiges #feedly
tinysubversions Promising! I'll have to check if this includes their journals and other archives twitter.com/krenshar_posts…
tinysubversions Is there a way to like, pay monthly to get access to a ridiculous number of online journals and archives as though I were affiliated with a university? Right now my solution is "take a single class once a year and pay out of pocket for it"
perseveresther kombinatorisches Explosiönchen
Auxquelles, aber als belgisches Dorf
SdeWijs Great piece by @berndulrich in @DIEZEIT - "The Magic of populists" arguing that West's political crisis had first hit Social democrats + is now tearing apart the conservatives. Perhaps the answer is in more @enmarchefr parties uniting the Social Democratic + Conservative Center? pic.twitter.com/uJjXY4kI1V
frachtschaden Ich wäre durchaus bereit, 5 Euro Rundfunkgebühr im Monat zu bezahlen.
kathrinpassig @texttheater "The Bongo Bungler" von Janwillem van de Wetering hätte nicht "Bongo-Pfuscher" heißen sollen, sondern "Trommel-Trottel", wen interessiert denn da die Art des Perkussionsinstruments.
ojahnn Motivation chapter of my thesis twitter.com/MobyDickatSea/…
@kathrinpassig Details!
kathrinpassig Gerade ist mir die richtige und ganz einfache Lösung für ein Übersetzungsproblem eingefallen. Das Buch, in das ich die falsche geschrieben habe, ist vor 17 Jahren erschienen und seit 16 vergriffen.
acl2018 Algorithms like LSTM and RNN may work in practice. But do they work in theory?, asks M. Steedman #ACL2018 pic.twitter.com/Ua8pKFDQwS
ojahnn Kennt ihr die Geschichte von Jim Knopf, der so oft Löcher in der Hose hat, dass Frau Waas ihm einen Knopf drannäht, damit er das Loch auf- und zuknöpfen kann, statt dass sie immer die Hosen flicken muss? Sowas brauche ich für meinen Badezimmerfußboden.
oaostrik More #artdeco deets for your delectation pic.twitter.com/n2oPuzMBub
msmollyebrown Today while giving high school students a tour of the Archives one asked what the microfilm reader was. He then realized "Oh, that's what people use in horror movies to find out the scary history of the town." WHAT. A. HOT. TAKE.
drgriffis Running theme of #ACL2018: we need to start thinking about broader kinds of generalization, to testing situations that don’t mirror the training distribution. Sounds like we’ve got a good topic to think about for #ACL2019 already!
emilymbender Steedman: One component missing from standard semantics is information structure—marked by intonation in English and other things in other languages. Have worked on modeling that with CCG (point of [you LIKE] example above).
#ACL2018
emilymbender @adveisner Steedman: I find generation completely terrifying because as you say you can get away with an incomplete semantics when doing parsing, but not in generation. And we don’t know how to do that completely or properly.
#ACL2018
emilymbender @adveisner Steedman: Short answer, just as syntax & semantics are a packaged deal, I think music is too, with a very different semantics. Syntax is tiny but ambiguity is much greater.
#ACL2018
emilymbender @adveisner Steedman: Moreover, the class of grammars needed for relations between chords (with semantics of things like cadence) is exactly the same as is needed for natural language, including non-adjacent dependency.
#ACL2018
emilymbender @adveisner Steedman: Interesting discovery: This problem is big enough that it needs the same kind of statistical models used in search problem in parsing.
#ACL2018
emilymbender @adveisner Steedman: Have been parsing midi-output (pitch approximated as nearest semi-tone and duration) to work out key signature and identity of chords.
#ACL2018
emilymbender @adveisner Steedman: I believe that just as we got standard language from planning, we also got music at the same time. Have been working on this with a couple of very brilliant students.
#ACL2018
emilymbender Steedman: Even if seq2tree is psychologically real, we still face the supreme challenge of finding out what the universal semantic language looks like. SQL, GeoQuery, SPARQL are not proxies for the language of mind.
#ACL2018
emilymbender Steedman: Can they learn all the stuff in the long tail: non-constituent coordination, subject extractions, cross dependencies?
#ACL2018
emilymbender Steedman: Conclusion: LSTM and RNN work in practice, but do they work in theory?
#ACL2018
emilymbender Steedman: Skipping final section of talk: Whatever we do about that problem, the problem of semantics is harder. QA involves inference as well as semantics, and we have no idea of the semantic representation that will support it.
#ACL2018
emilymbender Steedman: But the future probably lies with hybrid systems.
#ACL2018
emilymbender Steedman: So my money is going to be on a continuing need for CCG parsers in tasks like QA where long-range dependencies matter.
#ACL2018
emilymbender Steedman: But neural SRL systems still have trouble with long-range wh dependencies. Tried English and French subject extraction (with special syntax in both) in an NMT system and it failed.
#ACL2018
emilymbender Steedman: (Knowledge graphs used to be called Semantic Nets, and didn’t work because the computers were so small.)
#ACL2018
emilymbender Steedman: DNNs are faster because we don’t have access to a universal semantic representations that would allow children to induce full CCGs for their first languages.
#ACL2018
emilymbender Steedman: But it’s very idiosyncratic (horrible knowledge graph languages), so it’s good to do with DNNs which is faster.
#ACL2018
emilymbender Steedman: So go semantic parser induction for arbitrary knowledge graphs (like Freebase) (Reddy et al 2014). But surpassed by DNNs.
#ACL2018
emilymbender Steedman: Unreliability of those parsers is one reason we’re truing to end-to-end methods. (EMB: Really? I thought it was because CS researchers like to ‘learn’ the solution to the whole thing.)
#ACL2018
emilymbender Steedman: However, limited by size of available training data. About half of the errors in wh dependencies come from lack of info in training data.
#ACL2018
emilymbender Steedman: Planning is a search problem through a labyrinth of possible states, and works with the same algorithms as parsing (CKY & simliar). So we have the infrastructure to hang language onto.
#ACL2018
emilymbender Steedman: (Examples of more and more complex animal planning, and how they relate to the combinators.)
#ACL2018
emilymbender Steedman: So good that you can do the rest of the parsing exhaustively e.g. with A* search (shout out to Mike Lewis).
#ACL2018
emilymbender Steedman: CCG also particularly well suited to parsing with supertagging front ends. Finite-state, Markovian models that probabilistically assign tags giving info about what the word expects to combine with as a precursor to real parsing.
#ACL2018
emilymbender Steedman: Ambiguity wasn’t a problem because there is so much already and we already need the statistical models.
#ACL2018
emilymbender Steedman: However, the supposedly spurious constituents also show up in coordination and intonational phrases. Any grammar with the same coverage will have the same degree of non-determinism in the parser.
#ACL2018
emilymbender Steedman: CCG initially assumed to be hopeless for parsing because of massive spurious ambiguity.
#ACL2018
emilymbender Steedman: Vijay-Shanker and Weir 1990 proved the “shared stack” claim of Ades and Steedman (1982).
#ACL2018
emilymbender Steedman: For n=8, around 80% of the permutations are non-seperable. Useful for alignment in MT!
(And 8 is not large for the elements of a sentence.) #ACL2018
emilymbender Steedman: 21 of the 22 permitted are attested in some language, but the two forbidden ones are among the unattested three (Cinque 2005, Nchare 2012) 1 in 100 chance of this happening randomly.
#ACL2018
emilymbender Steedman: It’s the B2 rules that give CCG greater than context-free power, but still less powerful than movement.
#ACL2018
emilymbender Steedman: All long-range dependencies are done by contiguous reduction of a wh- element with an adjacent non-standard constituent with category S/NP, formed by rules of function composition.
#ACL2018
emilymbender Steedman: Combinatory Projection Principle: what’s in the lexicon (like combination direction) can’t be overridden by the combinatory rules.
#ACL2018
emilymbender Steedman: CCG lexializes all bounded dependencies and all syntactic rules are combinatory. CCG locks together syntactic and semantic categories.
#ACL2018
emilymbender Steedman:
Computational linguists realized that they were spending all their time implementing exceptions to limit the search through such expressive grammars.
#ACL2018
emilymbender Steedman: And the field fragmented. Linguists abdicated responsibility for “performance” focusing on “competence”. Psychologists became agnostic about grammar (and I think they still are).
#ACL2018
emilymbender Steedman: Can model abstractive in applicative systems by making abstraction a primitive operation (lambda) or in terms of a collection of operators on strictly adjacent terms: combinators like function composition.
#ACL2018
emilymbender Steedman: But it doesn’t follow from that that we have nothing to learn from linguists! (EMB: 💯) In fact, if I knew more linguistics, I wouldn’t have made such a mess of CCG.
#ACL2018
emilymbender Steedman: Why does natural language allow discontinuity?
#ACL2018
emilymbender Steedman: Ades & Steedman (1982) suggest that the same stack can be used to characterize both long-range dependencies and recursion. This is the idea behind the first CCGs.
#ACL2018
emilymbender Steedman: Hard to see how to generalize to multiple, crossing dependencies. If the HOLD register is a stack, the ATN becomes a two-stack machine, i.e. a Turing machine.
#ACL2018
emilymbender Steedman: ATN parser reduces all unbounded dependencies to local operations on registers.
#ACL2018
emilymbender Steedman: The starting point for this development: the central problem is discontinuity, i.e. non-adjacent or non-projective dependency. Ex with coordination and relative clause.
#ACL2018
emilymbender Steedman:
Soon after, this consensus fell apart. Chomsky saw that transformation rules were too expressive to have explanatory force. Psychologists realized that their measures of processing difficulty didn’t resemble transformational derivational complexity.
#ACL2018
emilymbender Steedman:
In the 1960s, theoretical linguists, psychologists and computational linguists (and the AI community) saw oursselves as working on the same problem.
#ACL2018
emilymbender Steedman: Psychologists had always insisted that competence/performance can’t be completely divorced. No point in having grammar w/o parser & vice versa.
#ACL2018
emilymbender Steedman: Have exasperated the mathematical linguists by changing the formalism to match what I was discovering.
#ACL2018
emilymbender Steedman: Chose that title to stress that CCG has always been an empirical grammatical formalism.
#ACL2018
AlvinGrissomII During this interesting talk by Anton van den Hengel, shortcomings of DL seem to be weaknesses in ML in general. My impression is that since CNNs were such a jump for vision (much more so than for NLP), the assumption in vision is always to learn w/ DL. Not so in NLP. #acl2018
"Kartoffeln sind toll, aber irgendwie ist mir die Konsistenz nicht eklig genug." -- Erfinder der Kartoffelklöße
yoavgo Indeed! Lexical/category knowledge is very important!
Today 14:45 (plenary) @VeredShwartz will present our work showing that SOTA end-to-end neural SNLI systems fail to acquire such knowledge. twitter.com/emilymbender/s…