Khalid Salama

In context of debate if large context #LLM models will eventually negate the need for #RAG, an article from Databricks has some good insights - • Longer contexts boost RAG accuracy... to a point • Performance often peaks at 32k-64k tokens, then declines • Top models (GPT-4, Claude 3.5) maintain consistency at scale • Others show unique failure modes (e.g., copyright concerns, summarization instead of Q&A) • Optimal context size varies by model and task • Lack of long-context training data may explain some issues Key takeaway: Long context and RAG are synergistic, but LLMs need refinement to fully leverage extended contexts. https://lnkd.in/eehdKtCm

17

This is really awesome. I used notebookLM to summarise a recent paper that shows how o1 like model can be reproduced. After providing the pdf from Arxiv, I noticed that the generated podcast was almost 49 minutes long. Very different from the earlier 10-12 minute version. And another impressive feature was the interactive mode where you can interject the two hosts and ask a question which they very graciously respond to. I wanted to share the audio summary but YT didn’t allow me to upload the WAV file. YT expects a video mp4 file. So I found free videos from Pexels and stitched them together with ffmpeg. The videos are meditative acquarium shots of fish swimming around. And with the audio discussing reproducing o1 as an engaging podcast episode, the video keeps the distraction to the minimum. The paper itself is a great read but if you have a 30-40 minute commute, or want to watch some cool fish swimming around and learn about o1, check out

12

OpenCoder doesn't get enough love They open-sourced the entire pipeline to create QwenCoder-level code models. This includes: - Large datasets - High-quality models - Eval framework Tons of great lessons and observations in the paper 📝 Paper: arxiv.org/abs/2411.04905 #special #linkedin #Family #Pakistan #skills #islam #government #AI #ML #MachineLearning #tesla #microsoft #openai

5

If you are getting started with LLMs and want to learn from first principles, look no further and read this outstanding blog by Andreas Stöffelbauer https://lnkd.in/gcwCdfVm

7

Can LLM preference tuning degrade Multi-Agent system performance? In a recent tweet, Andrej Karpathy questioned why all LLMs sound similar, prompting Sebastian Rascha to speculate about the uniformity of alignment datasets. Aligning user preferences is a last-mile step in training large language models. Regardless of the pre-training datasets used, could the preference datasets used during alignment compromise the effectiveness of LLMs? This issue could adversely affect multi-agent systems incorporating multiple LLMs in their design.

7

1 Comment

Today's reading is Memory Layers in Large Language Models 🧠 This research shows how memory layers can take LLM performance to the next level: 🔑 Trainable key-value lookup mechanism: Adds parameters without increasing computational cost, boosting factual accuracy and task performance. 📈 Big results, small cost: Outperforms models with much larger computational budgets and even beats mixture-of-experts models! ⚙️ Scales seamlessly with up to 128 billion memory parameters The authors from Meta highlights how memory layers are a game-changing addition to future AI architectures, offering smarter, faster, and more accurate solutions. You may want to check out the paper at: https://lnkd.in/gzuqUfYe #AI #LLM #MemoryLayers #Innovation #MachineLearning #FactualAI #FutureOfAI #Meta

19

**Introducing GraphRAG by Microsoft Research!** GraphRAG, now available on GitHub, is a tool for complex data discovery. It leverages a large language model to create rich knowledge graphs from text documents, enabling precise question-answering over private datasets. This approach outperforms naive retrieval-augmented generation (RAG) by providing structured information retrieval and comprehensive response generation. Ideal for answering global questions, GraphRAG supports informed decision-making with higher comprehensiveness and diversity. 🔗 [Read more](https://lnkd.in/eEcukH_A)

56

#RAG to #GraphRAG Only yesterday we looked at #HybridRAG which was how one uses #RAG (+) #KnowledgeGraphs together 💡 And here comes this powerful message from #Microsoft GraphRAG: Unlocking LLM discovery on narrative private data Why is this powerful ⁉️ GraphRAG uses LLM-generated knowledge graphs to provide substantial improvements in question-and-answer performance when conducting document analysis of complex information. So the challenge of vectorising all data ( not feasible at times) to be used in RAG has a new way ⏩ read more👇 Share your views on how else can we use this... https://lnkd.in/g4fxjt_6

1

In recent years, adapters have emerged as a crucial technique for improving the efficiency of Large Language Models (LLMs). These methods allow for fine-tuning models in a way that’s parameter-efficient while maintaining or improving performance for specific tasks. Below, we’ll explore the ..... #generative #llm #peft #tunning #prompttuning

8

2 Comments

https://lnkd.in/ef_gzFdU Time series data is super cool. Your outputs from a future time are your labels for the present time. This particular tutorial is especially good as it teaches many aspects of software/ML engineering. However, in my opinion, there are some shortcomings with the approach outlined. Let's see how many we can spot. For the googlers in my connections, if anyone knows how we can submit code to improve upon this, would love to know how.

13

Another good resource, if you are into NLP is Aclanthology The other I've recommended before is Arxiv. Links in the comments:

2

1 Comment

"THIS IS INSANE" - the R debugger in the Positron IDE allows to go into the underlying C code of an R function - in this example for dplyr's mutate() function. 🚀 This is powered by Ark, an R kernel for Jupyter applications, created to serve as the interface between R and the Positron IDE. #PositConf2024 #rstats #DataScience

209

9 Comments

If you think of foundation models as probabilistic compilers of natural language and images into machine representations (i.e. numeric vectors) - then designing a multi-model AI system naturally implies that you will need to account for compiler differences when writing your natural language instructions. What a smaller model like phi will do with your natural language instructions is drastically different than what a larger model like llama vision can do. It’s almost equivalent to writing in Python vs. .NET. Therefore, you need to be intentional about model benchmarks, capabilities and weaknesses, and performance and throughput etc as you design compound AI systems with multiples models at play. Otherwise, a like-for-like swap would result in a world of pain downstream.

5

Props to Charles Verleyen for pinning specific versions of the Python libraries in his article about the latest dbt release. This should become standard practice for blog demos. There are a lot of good technical articles out there that don't run anymore because the authors didn't pin a specific version of the libraries (and their dependencies). In this case, he pinned the libraries with: pip install dbt-core==1.8.0 dbt-bigquery==1.8.1

5

Great post by Celebal Technologies on implementing RAFT on top of Databricks fine-tuning to out-perform RAG: https://lnkd.in/gqfjeMTV We think this is a key tool for building compound AI systems that get the best possible quality on *your* data -- mixing the best of retrieval, fine-tuning and other AI techniques to maximize application quality. RAFT paper: https://lnkd.in/g3vg5tpw

183

4 Comments

Kolmogorov–Arnold Networks seem like a paradigm shift in ML architectures. Curious to see how they evolve. "While MLPs have fixed activation functions on nodes ("neurons"), KANs have learnable activation functions on edges ("weights"). KANs have no linear weights at all -- every weight parameter is replaced by a univariate function parametrized as a spline. " https://lnkd.in/gdPk_qCQ

29