Eric Qiu

Last night I had the fortune of representing LaunchDarkly at Sourcegraph's CodeAI Summit NYC. An important theme I took from last night's event? Context. Models like specifics. Computers can't read our brains or know what we want to achieve. Technology is kinda useless without the right context. As we move forward in this increasingly AI-generated world (for better and for worse) let us not forget the importance of adding context to the equation. Whether that's providing context to a dataset through how it was collected, sharing context of your repo to better aid a code assist tool, or simply providing context behind the why of our decisions. It's an interesting (and important) place to keep in our minds. #AI #MachineLearning #DeveloperAdvocacy #DeveloperEvents

14

1 Comment

Well sequencer centralization is surely an issue. Maybe base can allow independent sequencers. Also if sequencers fail, can have a fallback mechanism to allow to submit onto Mainnet directly to remove reliance on the centralised sequencers.

1

Big news from the Cresta team: we’ve just raised $125M in Series D funding! 🌍💡 This marks a huge step forward in our journey to create a unified platform for human and virtual agents, helping contact centers deliver exceptional customer experiences while saving costs and driving revenue. Check out the blog to see what’s next for Cresta: https://hubs.la/Q02YKsjS0!

21

Today, I found a fascinating way to think about “Tokens” in LLMs. It came from a conversation between Jensen Huang and Patrick Collison at a recent Stripe event. When asked about the future of compute capacity in five years, Jensen smartly dodged that question a bit, but then gave an excellent explanation and analogy about why tokens are the new forces that will power the next decades of humanity. He explained that we're now producing something unprecedented (and at scale): floating point numbers that possess “value”, which we now call tokens. These tokens are valuable because they encapsulate “intelligence”. People are now taking these tokens and transforming them into English, French, images, videos, chemicals, proteins, robotic movements, etc. And many are working hard to expand the range of concepts and ideas we can create with these tokens. He then goes on to made a compelling comparison between tokens and electricity. In the past industrial revolution, we successfully found a way to convert “atoms” into “electrons” (by boiling water to power electricity turbines). And now, we've discovered a way to convert "electrons" into "tokens" (by using energy to power data centers that train and run LLMs). When electricity was first introduced, few people understood its value. Today, paying for kilowatts is routine. The same will happen with tokens. Right now, only early adopters and builders are paying tokens. Soon, everyone will be paying for tokens on a daily basis to supercharge productivity and power new products and services. Many new industries will be born from and built on top of tokens. When I first heard about this way of thinking, I got goosebumps. Perhaps it’s because Jensen has a good storytelling skill that helps him sell what he builds, but I have to admit this way of thinking gave me a brief surge of pride. I'm proud of humanity's collective effort and creativity, which have taken us from living under rocks to building machines that can "think." That's nothing short but a miracle.

18

5 Comments

There are dozens of people giving away their hard-won insights on building LLM-based software for free, online. Sure, there's a lot of noise out there, but there's also an amazing amount of signal too. One of my favorites is Jason Liu - If you are building in this space, and haven't stumbled on his content yet, you should check it out. His recent episode on TWIML has some absolute gem recommendations in it. I took some notes on the episode, and wrote a short post about it on my site: https://lnkd.in/gxMkVP7Q Some of the bigger take-aways are remarkably consistent: - Figure out the real problem you need to solve. Focus on that above everything else - Look at your data, it tells you things when you focus on it - Small incremental progress compounds - Test your systems the way you've always tested them before -- and avoid shiny objects What are you learning? What are you building? Where do you need help? Don't hesitate to reach out.

3

How we index timeseries data at Datadog with Rust and RocksDB https://lnkd.in/e8x_RUmz

57

3 Comments

How cool is generative AI? I asked ChatGPT to transform an article about the fictional Jamie and Alex family (which I've been posting about recently) into lyrics for a 30-second song. Lyrics: [Verse] Jamie’s family walks the line, Saving smart in every design Certainty in every plan, Guided by a steady hand. [Chorus] Safe, simple, sound, they stride, With whole life insurance by their side From youth to age, through calm and storm, They build a future safe and warm. Then, I used Udio to bring the music to life. Here are the results: Udio generated multiple versions, each about 30 seconds long. Take a listen and let me know which one you like best! https://lnkd.in/g5e2_xmi #GenerativeAI #AIinMusic

1

3 Comments

My recent thesis on Funding Rate! https://lnkd.in/gPX4xdmR Abstract This article explores the evolution and significance of the funding rate in cryptocurrency derivatives markets, with an emphasis on its origins, mechanisms, and the arbitrage strategies it inspires. It highlights the "Golden Age" of funding rate arbitrage in Spring 2021, the subsequent decline due to market corrections, and the rebirth of these opportunities in 2024-2025 with the advent of innovative stablecoins like USDe and USDX. The critical role of CME in influencing funding rates is discussed, showing how traditional finance (TradFi) intersects with decentralized finance (DeFi). Finally, the paper argues that the influence of institutional players and CME, rather than emerging stablecoins, shapes funding rate dynamics, urging a reconsideration of the ecosystem's narrative. 摘要 本文探讨了加密货币衍生品市场中资金费率的发展及其重要性，重点分析了资金费率的起源、运作机制及其激发的套利策略。文章回顾了 2021 年春季资金费率套利的“黄金时代”，以及由于市场调整导致的衰退；同时还分析了随着创新型稳定币（如 USDe 和 USDX）的出现，2024-2025 年资金费率套利机会的复兴。此外，本文探讨了芝加哥商品交易所（CME）在资金费率定价中的关键作用，揭示了传统金融（TradFi）与去中心化金融（DeFi）之间的交集。最后，文章提出，资金费率的动态更多由机构投资者和 CME 决定，而非新兴稳定币，呼吁重新审视加密生态系统的叙事逻辑。

4

This might be controversial, but one day LLMs will be seen as just a pipeline for tokenizing and embedding input to a much larger process. No one will care about LLMs for themselves, just the input they can provide to downstream models. They're just going to be an interface.

1

What better way to learn more about ML feature stores than to implement an ML feature store. I've always been interested in the solutions implemented when custom software solutions transition to a commodity infrastructure solution. There's no denying that #AI is going to be a significant part of the technology landscape for the foreseeable future. So, it should be no surprise that I have an interest in machine learning feature stores. Therefore, I've started implementing my own _simple_ #ML feature store as a way to explore the technology. I'll be writing about the challenges faced by feature stores and using my own feature store implementation to explore and illustrate solutions to these challenges. You can read my first article introducing ML feature stores and the challenges faced by feature stores here: https://lnkd.in/eZ3UMhHS.

1

I noticed a pattern while coding with Claude today. I gave the same task to Claude twice. First time: Had to push through several debugging rounds. Second time: One well-structured prompt got us there. What made the difference? The concept of Reflection. Andrew Ng talked about Reflection workflows as a game-changer for AI outputs. Instead of relying on a single attempt, Reflection teaches AI to: – Critique its own work. – Spot errors early. – Iterate step-by-step for better results. Next time, when you're coding with an LLM: – Prompt for self-review. Ask the AI to check correctness, efficiency, and style. – Build iterative loops. Feed its feedback back into itself for smarter refinements. – Use multiple agents. One model generates. Another critiques. Together, they improve. It's fascinating how LLMs think. They're like junior developers - they need the right kind of code review prompts to really shine. #prompting

83

1 Comment

Andrew Ng's AI Fund, an incubator for AI startups, is preparing to launch its second fund, AI Venture Fund II, with plans to raise up to $120 million. According to a recent SEC filing, the fund has already secured $69.75 million from 13 partners, leaving approximately $50 million still to be raised. The new fund, while smaller than the AI Fund's initial $175 million raised in 2018, represents a significant increase from the originally reported target of $50 million. ➜ Check the news at: https://lnkd.in/d6b-yGWp ➜ Vendorvise's official Instagram: https://lnkd.in/ebK2KQ2C ➜ Vendorvise's Linktree: https://lnkd.in/dQdbMdth 🤖 If you enjoy reading my posts about AI and tech news, hit the like button and share with your friends to support my content! #ai #artificialintelligence #ainews #news #blog #andrewng #aiexpert #aifund #investment #incubator #startup #generativeai #aitech #tech #technology #content #socialmedia #future #new #business #techtrends #innovation #launch #success #markets #marketing #people #industry

8

Today I'm sharing this 🔥 video from our recent Engineering onsite at Demandbase HQ in SF! https://lnkd.in/eFAFR6QG We were so fortunate to host Zhengxuan Wu and Aryaman Arora, two upcoming AI researchers from Stanford University Department of Computer Science. Their approach to fine tuning, called ReFT, targets the representations emitted by neural net layers rather than the weights of the layers themselves. This makes it very flexible when using the same base model for swapping multiple fine tunes. The memory overhead is also very low for their intervention weights. Another highlight is how powerful it can be. One POC they showed forced the LLM to speak in only emojis exhibiting ReFT's power to control output. Control is crucial when, for example, you want to prevent your bot from offering customers unauthorized refunds!! If you're already familiar with LoRA, it should be pretty simple to use with only a few extra hyperparameters -- details in the video. Here's a link to the repo. Give them a star! https://lnkd.in/eJScNbzK Thanks again for letting us host your first external talk Aryaman and Zen. It's very exciting tech!!

17

4 Comments

Behold the power of knowledge graphs

7

Meta has been focused on advancing its capabilities by utilizing synthetic data to enhance Llama 3.1 for code-related tasks. Through methods like execution feedback, back translation, and the model-as-judge approach, Meta has made significant progress. However, the 405 billion parameter model still lags behind Claude 3.5 Sonnet by a small margin in terms of performance. What stands out is Claude 3.5 Sonnet's impressive 92% accuracy on the human evaluation benchmark. This exceptional achievement raises questions about the unique strategies and innovations that propelled Anthropic to achieve such remarkable performance with the Sonnet model. Are you knowledgeable about the human eval benchmarks or can you provide insights into potential modeling approaches that might have influenced the accuracy in favor of the Sonnet model? Meta's paper https://lnkd.in/e4pS9ZQs #Meta #Llama31 #Claude35Sonnet #SyntheticData #Innovation #Technology #MachineLearning

7

Highly recommended book...

9

Aider Update + Gemini 1.5 Pro-2: The Ultimate Free AI Coding Agent with New LLMs and Commands

1

Very excited that we shared our blockchain solution, LETUS, at SIGMOD-2024. LETUS introduce a re-designed architecture with delta-encoding and version-based indexing schema, also with block tiering and state pruning proposed. LETUS is also universal for different blockchains, adaptable for Ethereum, BNB Smart Chain, etc. Evaluations shows that, LETUS improves the throughput by up to 15.8 times for the ERC-20 smart contract transfer workload. And reduces the space usage by up to 80.3%, reduces disk bandwidth usage by up to 95%. During the 2023 Asian Games in Hangzhou, AntChain (augmented by LETUS) successfully supported the digital flame ignition event, where over 100 million torchbearers from more than 100 countries lit the digital torches in real-time online. Final version paper link: https://lnkd.in/g5h736Zb PDF link: https://lnkd.in/gyNqy4r4

1

💡 The performance increase from 30 to 60 tokens/s for Llama 3.3 70B on H100 with vLLM SpecDec (recent beta release) is going to be really meaningful for more than one reason: 🐌 30 tokens/s felt a bit slow and detrimental to UX in my opinion. With 60 tokens/s it actually feels usable. It is like this speed bump made it past an important boundary. Are you building something where the user has to wait? SpecDec in vLLM is probably going to be an enabler for you. 🤡 Elon Musk has bought all NVIDIA Blackwell production for the upcoming few months. This means most of us are stuck on H100 cards for a while longer. Fantastic that a software change can double the tokens/s output! ⏳ Once SpecDec in vLLM stabilizes (and response_format: json_object works again) the open source ecosystem will seem somewhat complete for those that want to abandon OpenAI GPT-4 and the US-owned cloud. The timeline looks like this: 1. Llama 3.1 70B (Summer 2024): The first multilingual open model viable for weird languages such as Swedish was released. Not quite GPT-4o level though. 2. Llama 3.3 70B (Winter 2024): Just as good as GPT-4o. Performance on par with Llama 3.1 405B but now fits on a single H100. 3. vLLM SpecDec (Early 2025): Performance becomes "good enough" on H100 thanks to speculative decoding. 🌟 Looking forward to the open LLM ecosystem maturing even further next year. Will further software optimizations + Llama 4 make RTX 5090 32GB the "new H100 for inference" in 2025? Likely? Happy new year! 🔗 Read the full engineering blog post here: https://lnkd.in/dZEgrerZ

14

10 Comments