Melanie K.

Are LLMs capable of Artificial General Intelligence (AGI)? I came across this article in the NY Times: https://lnkd.in/geXXbi8B (for non-subscribers here's the paper it talks about https://lnkd.in/gP-m7sxA) The upshot is that researchers are finding strong evidence that language is a very distinct part of the human brain whose purpose is to communicate and that it is not intrinsically involved in reasoning. This gets at a very fundamental question about the use of LLMs as the basis for Artificial General Intelligence. Can a model based on language be used as the basis for AGI? Time will tell I'm sure, but when I've asked this question to researchers I know, they generally say yes. However, I'm not sure what the basis is for this belief. CAVEAT: I'm not in the AI field, nor have I delved deeply into the scholarly articles. I have a fairly rudimentary understanding of LLMs, but would be happy to learn from those who know more. One of the fundamental articles shaping my personal views on LLM based AI is Ted Chiang's excellent piece "ChatGPT is a blurry JPEG of the web" (https://lnkd.in/gmkRBe6A). For me, it captured the idea that LLMs are just summarizing what we know. To be sure, they are capable of really cool things, but are they really doing any thinking? More important are they even capable of thinking? Again, time will tell, but I'm increasingly skeptical. The kicker for me is that they have no common sense. One of the reasons we find LLMs so amusing is they crazy ways they go off the rails (aka hallucinations). I think that these are largely because there's no common sense 'boundary'. When a human's thoughts start to get carried away, we have ways of reigning them in by asking ourselves "is this actually a reasonable thought?" Indeed, it can be quite alarming to encounter people who communicate about ideas that we find to be outside the bounds of common reason. It feels like this 'framework of rationality' is very important but I don't think it's something you can just use as guard rails. It seems like you need some kind of other AI to be able to generate that framework. What do you think? Are LLMs capable of AGI? Or are they like our own language processing center, just a small piece of a much larger 'intelligence framework'? [Edited to use AGI instead of GenAI so as not to be confused with Generative AI] [Edit: Thanks to Tareq Alkhatib here is a direct link to the paper in question: https://lnkd.in/g9b7Q66c]

41

21 Comments

According to Eric Schmidt, in the next year or so, the union of the large context window, agents, and text-to-action, when delivered at scale, is going to have an impact on the world that no one understands yet. Listen to the fascinating discussion in the "The Age of AI, Eric Schmidt" youtube video - https://lnkd.in/dFTKgBVY

52

1 Comment

This is a very interesting article on sparse vectors, which can help overcome limitations of vector search related to domain specific keywords. And of course, the code in the example is clear and clean! Sparse vectors have a few values and many zeros. Dense vectors have limited or no zeroes. But really, what we're talking about semantic search on terms where the context is not present or harder to infer. The author presents this as specialized keyword search such as medical terms or other domain specific terminology. "These scores reflect the significance of each term in the context of the document or query, guiding the model to allocate more weight to terms that are likely to be more meaningful for retrieval purposes." Highly recommend reading. Nice work Paul Iusztin. And a nod to Qdrant and their vector search technology that supports sparse vectors. https://lnkd.in/gcBWfTg8

3

Here we go - Llama 3 is here! 🚀 - 8B and 70B models available today on all usual places - 15 trillion tokens (7x bigger training dataset compared to Llama2!) - 8k context length - state-of-the-art performance across industry benchmarks. - 400B+ model still in training to be released in the upcoming months These models are also powering the new and improved Meta.ai on the web and in WhatsApp. Also super excited about Imagine Flash (available inside Meta.ai, I believe US only for now) which lets you generate high quality images in real-time! This is a really creative product that gives you insight into how the foundational model "thinks". More about Imagine Flash here: https://lnkd.in/eqt7nJ9q And Llama3 here: https://lnkd.in/ehzdPz88 #meta #metaai #opensource #llama3

37

I ran across this post in the Changelog newsletter: https://lnkd.in/eCdjVekq - I like the name slop for unwanted AI-generated content. I love LLMs. They are part of my daily workflow, and I've been exploring them more seriously while on my sabbatical. I wouldn't misrepresent generated content without critical review as my own, but the web seems to be filled with more generated garbage (or 'slop')—content that looks fine at first but isn't accurate or helpful. LLMs (and other generative AI) should help us to make things of higher quality, not just churn out slop more quickly.

11

1 Comment