Key Highlights:

Robotic Vision and Machine Learning: Delve into the complexities of robotic vision, comparing classical computer vision techniques with deep learning methods, and discussing their applications in automation, field robotics, and cloud machine learning.

Design Considerations: Understand the design considerations for integrating machine learning into robotics, addressing challenges related to real-time data processing, connectivity, hardware-software ecosystem, and the evolving roles within robotic vision and sensing.

Simulation-Driven Development: Explore the importance of simulation-driven development in robotics, leveraging tools like ROS and Moose, and the role of agile development approaches in shaping the future of robotics.

Career Paths and Continuous Learning: Gain insights into career paths in robotics beyond engineering, the vital role of simulation in robotics training, and tips for continuous learning and career advancement in the field.

Project Ideas and Internship Tips: Discover project suggestions and internship tips for aspiring robotics professionals, and considerations regarding data privacy and safety in the context of consumer-direct robotics use.

Embark on this enlightening conversation with Kausthub Krishnamurthy as he unravels the intricacies of robotic vision, machine learning, and the future of robotics in the dynamic landscape of technology and innovation.

Key Highlights:

Origins of YOLOv8: Explore the evolution of YOLO models, from YOLOv3 to YOLOv8, as Jocher reveals the technical advancements and innovations driving the development of these groundbreaking object detection algorithms.

Community Contributions: Learn about the pivotal role of open-source contributions and community collaboration in the success of YOLOv8, showcasing the power of collective intelligence in pushing the boundaries of AI vision systems.

Technical Insights: Delve into the technical intricacies of YOLOv8, including architecture changes, loss functions, and the transition from anchor-based to anchor-free systems, offering a deeper understanding of the underlying mechanisms driving object detection.

Wide Applications: Discover the diverse range of applications of YOLO models, from flaw detection in manufacturing to aiding visually impaired individuals, highlighting the versatility and real-world impact of these cutting-edge AI technologies.

Future Directions: Gain insights into the future of YOLOv8 and beyond, including plans for mobile deployment, architectural improvements, convergence with NLP, and optimization strategies for custom datasets, paving the way for advancements in AI-driven object detection and computer vision.

Embark on this enlightening journey with Glenn Jocher as he unravels the intricacies of YOLOv8 and shares his vision for the future of object detection in the ever-evolving landscape of artificial intelligence.

Key Highlights:

Guest Introduction: Meet Josh Tobin, as he shares his journey from academia to entrepreneurship, highlighting his expertise in MLOps and the practical aspects of deploying ML models in production.

ML in Production: Explore the significant differences between ML in a research setting and ML in production, emphasizing the importance of integrating ML models within broader product systems.

Emerging Trends: Tobin discusses the emerging field of MLOps, the impact of foundational models like GPT-3 on ML operations, and the nuanced challenges of deploying AI systems in real-world scenarios.

Practical Considerations: Gain insights into practical aspects of ML in industry, including experiment management, feature stores, and the complexities of integrating state-of-the-art models into production systems.

Future Outlook: Tobin offers advice for practitioners and businesses navigating the AI transformation, stressing the collaborative potential between humans and AI and underlining the critical role of prompt engineering in the next generation of AI applications.

Join us in this engaging conversation with Josh Tobin, as we explore the dynamic landscape of ML research, production, and the future trends shaping the field of artificial intelligence and machine learning.

Key Highlights:

Guest Introduction: Meet Kyle Kranen, a senior deep learning algorithm engineer at Nvidia, as he shares his wealth of experience and insights into the world of graph neural networks.

Power of Graphs in Data Representation: Explore the significance of proper data structures in machine learning and delve into how graph neural networks have overcome challenges in handling complex relationships within data.

Graph Anatomy: Uncover the intricacies of graphs, examining their role as a powerful tool for data representation and understanding their ubiquitous presence in various domains.

Local Aggregation in Graphs: Kyle introduces the concept of local aggregation in graphs, shedding light on its importance and its role in enhancing the capabilities of graph neural networks.

Message Passing: Gain a deeper understanding of the importance of message passing in graph neural networks, a fundamental mechanism for information exchange and aggregation.

Graph Neural Network Architecture: Navigate the anatomy of a graph neural network, exploring its basic building blocks and the significance of learnable parameters in capturing complex relationships.

Predictive Power: Discover the predictive power of graphs, exploring graph-level, node-level, and edge-level predictions, along with insights into representing the 'blobbiness' or unstructured nature of a graph.

Edge Classification and Graph Isomorphism: Kyle delves into specific challenges such as edge classification and the graph isomorphism test problem, providing nuanced perspectives on tackling these issues.

Popular Architectures: Explore the landscape of popular architectures for graph neural networks, understanding the diversity of approaches that cater to different applications.

Production Pipelines: Gain insights into the production pipelines for graph neural networks, unraveling the practical aspects of deploying these models in real-world scenarios.

Advantages of Graph Learning: The episode concludes with an exploration of the advantages of graph learning, highlighting the transformative potential of leveraging graph neural networks in diverse domains.

Join us in this comprehensive discussion as Kyle Kranen demystifies the realm of Graph Neural Networks, offering profound insights into their applications, challenges, and the immense potential they hold in reshaping the landscape of deep learning.

Key Highlights:

Guest Introduction: Meet Abi Aryan, a self-taught computer scientist, and machine learning engineer, as she shares her extensive experience in leveraging AI for smarter software systems development.

Challenges in MLOps: The discussion kicks off with a deep dive into the challenges of MLOps, exploring computational resources, industry distribution, and nuances of data collection and labeling.

Market Landscape: Aryan provides insights into the market landscape, highlighting the transformative role of large language models (LLMs) in diverse industries such as e-commerce, insurance, media, and entertainment.

Transition to MLOps and LLMops: Explore the transition from MLOps to LLMops, understanding the unique challenges and future prospects in the development and deployment of large language models.

Q&A Session: Engage in a dynamic Q&A session where Aryan addresses audience questions, covering topics such as challenges in LLM development, incorporating AI agents into software services, evaluating models, and the balance between fine-tuning and prompt engineering.

Applications in Legal Research: Uncover the applications of LLMs in legal research and document analysis, showcasing their potential impact on enhancing efficiency and accuracy in the legal domain.

Choosing the Right Framework: Aryan shares insights into the considerations for choosing the right framework for LLM deployment, offering practical tips for ensuring seamless integration and performance.

Future of Libraries and Computer Vision Models: Gain a glimpse into the future with discussions on libraries like LangChain, the potential emergence of computer vision-focused models, and considerations for running LLM applications on low-level hardware.

Cost Considerations and Career Trajectories: The session concludes with considerations on cost in training models, developing Minimum Viable Products (MVPs), discussions on different roles in the AI space, and insights into potential career trajectories.

Join us in this enlightening conversation with Abi Aryan as she demystifies large language models, offering profound insights into their challenges, applications, and the exciting future they hold in the ever-evolving landscape of artificial intelligence.

Key Points:

Guest Introduction: Dr. Tristan Behrens, an AI advisor and researcher, shares his unique journey from software development to a Ph.D. in computer science and AI.

Computation and Creativity: The episode begins by unraveling the intricate relationship between computation and creativity, highlighting the fusion of technology and artistic expression.

AI in Music Composition: Dr. Behrens discusses the process of training AI models on diverse music genres using MIDI data, employing the Transformer architecture and a complex token vocabulary for music track generation.

Credit in AI-Augmented Creativity: The discussion touches upon the evolving role of AI in augmenting human creativity, acknowledging the importance of giving credit to both AI and human contributors.

Transformers in AI: Understanding the role of Transformers in AI, particularly in converting text to music, showcases the complexity and versatility of modern AI architectures.

Data Pipeline and Modeling: Dr. Behrens provides insights into building the AI model, emphasizing the significance of a robust data pipeline and thoughtful modeling.

AI Music Creation Process: Explore the intricacies of converting text to sound, accompanied by Dr. Behrens' firsthand experiences with neural network outputs.

Challenges and Role of Symbolic AI: Delve into the challenges of AI in music generation and the potential influence of Symbolic AI in shaping the future of creative AI applications.

Future Architectures: A glimpse into the future unfolds as Dr. Behrens discusses the evolving landscape of AI architectures and their impact on creative endeavors.

Deep Reinforcement Learning: Uncover the potential role of deep reinforcement learning in pushing the boundaries of AI music generation.

Challenges of Deep Learning in Creativity: The episode concludes by addressing the challenges inherent in integrating deep learning into the augmentation of human creativity.

Join us in this enlightening conversation with Dr. Tristan Behrens as we navigate the fascinating intersection of artificial intelligence and creativity, unlocking new possibilities in the realm of AI-generated music.

Key Highlights:

Guest Introduction: Meet Eugene, a Kaggle Grandmaster, as he provides a glimpse into his background, journey, and expertise in machine learning, software development, and deep learning.

Deep Learning Applications: Delve into a discussion on the applications of deep learning, exploring Eugene's experiences and insights into the field, including its real-world impact.

Kaggle Competitions: Gain insights into Eugene's Kaggle journey, from his approach to problem-solving to practical tips for fine-tuning pre-trained models and navigating reinforcement learning challenges.

Balancing Work and Learning: Explore Eugene's strategies for balancing work, learning, and personal life, understanding the importance of continuous growth and development.

Learning from Mistakes: Eugene emphasizes the value of learning from mistakes and building intuition, sharing a Kaggle story that highlights the significance of checking predictions and choosing the right metrics for optimization.

Object Detection Challenge: Uncover Eugene's approach to a new problem, specifically the Object Detection Challenge, including insights into optimizing time, maximizing GPU usage, and maintaining an experiment log.

Fine-Tuning Pre-Trained Models: Eugene shares practical tips on working with pre-trained models and fine-tuning, providing valuable guidance for enhancing model performance.

Reinforcement Learning Journey: Gain a glimpse into Eugene's personal journey in reinforcement learning, exploring the challenges, insights, and the evolving landscape of this dynamic field.

Debate on Kaggle's Functionality: Eugene shares his perspective on the functionality of Kaggle, discussing its role as a learning platform and its relevance in real-world applications.

Join us in this insightful conversation with Eugene, as he unravels the power of deep learning, sharing experiences, practical tips, and thought-provoking perspectives that contribute to the dynamic landscape of artificial intelligence.

Key Highlights:

Guest Introduction: Meet Dr. Yannic Kilcher as he delves into his transition from medicine to deep learning, highlighting the potential benefits and limitations of generative AI models like ChatGPT.

Deep Learning Insights: Explore Dr. Kilcher's perspectives on the impact of crypto on deep learning, reproducibility in deep learning research, and the role of AI in industry and startups.

Future Trends: Dive into discussions on the evolution and future of deep learning, disruptive trends in computer vision, and the potential intersection of the Metaverse and edge computing.

Chat GPT Applications: Discover the usability and potential of ChatGPT in various fields, while addressing concerns about its tendency to 'hallucinate' or generate fabricated responses.

Deep Judge and Diffusion Models: Learn about Dr. Kilcher's AI startup, DeepJudge, and its role in leveraging ChatGPT, as well as insights into the future of diffusion models and VQVAE organ models.

Personal Journey and Research Papers: Gain inspiration from Dr. Kilcher's personal journey into coding and deep learning, along with tips for identifying the core ideas while reading research papers.

Join us in this engaging conversation with Dr. Yannic Kilcher, as we explore the dynamic landscape of deep learning, innovation, and future trends, paving the way for transformative advancements in artificial intelligence and machine learning technologies.

Key Highlights:

Guest Introduction: Meet Serg Masis, a data scientist with a background in digital agriculture, data science, and entrepreneurship. Explore his expertise as he delves into the nuances of machine learning interpretability.

Importance of Machine Learning: Masis underscores the significance of comprehending machine learning and its implications, emphasizing the need for transparency and trust in AI models.

Interpretability vs. Explainability: Navigate the distinctions between interpretability and explainability in machine learning, understanding their critical roles in different scenarios.

Trust and Understanding: Explore the paramount importance of establishing trust and understanding the reasoning behind machine learning predictions, especially in high-stakes domains like healthcare and finance.

Trade-off Considerations: Masis discusses the delicate balance between interpretability, explainability, and accuracy, offering insights into making informed trade-offs based on the application context.

Activation-Based Methods: Gain a deeper understanding of activation-based methods in machine learning, unraveling their role in enhancing interpretability.

Role of Color: Delve into the impact of color on machine learning interpretability, with a focus on its significance in image interpretation.

Data Augmentation and Simulation: Discover the pivotal role of data augmentation in developing robust machine learning models and its implications for interpretability.

Interpretation Methods: Explore various interpretation methods, including gradient-based and perturbation-based methods, understanding their applications and nuances.

Global vs. Local Interpretation: Masis sheds light on the distinction between global and local interpretation, providing insights into their respective applications in machine learning.

Model Specific vs. Model Agnostic: Navigate the considerations between model-specific and model-agnostic interpretation approaches, highlighting their relevance in diverse contexts.

Monitoring Image Drift: Understand the challenges and methodologies involved in monitoring drift in images, ensuring the ongoing robustness of machine learning models.

Future Projects and Accessibility: The episode concludes with a glimpse into future projects and Masis' vision for making AI more accessible, paving the way for advancements in the field.

Join us in this illuminating conversation as Serg Masis demystifies machine learning interpretability, offering a holistic view of its applications, challenges, and the path forward in making AI more transparent and accountable.

Key Highlights:

Guest Introduction: Meet Susan Shu Chang, Principal Data Scientist at Elastic, as she shares her extensive expertise in implementing machine learning at scale, particularly focusing on real-world applications of reinforcement learning.

Understanding Reinforcement Learning: Delve into the concepts of reinforcement learning, distinguishing between supervised learning and reinforcement learning, and exploring the key differences between model-free and model-based learning approaches.

Reinforcement Learning Iterations: Explore the iterative nature of reinforcement learning, understanding the concepts of policy-based and value-based methods, reward functions, and the fundamentals of Q-learning.

Deep Reinforcement Learning: Transition into deep reinforcement learning, exploring the role of neural networks and transfer learning in enhancing the capabilities of reinforcement learning agents.

Real-world Applications: Susan exemplifies reinforcement learning using real-world examples, such as improving customer support workflows, showcasing the practical impact and efficiency gains achieved through reinforcement learning implementations.

Challenges and Considerations: Gain insights into the challenges and considerations in deploying reinforcement learning models, including reward design, training processes, and the complexities of deploying models in real-world environments.

Interactive Q&A Sessions: Engage in interactive Q&A sessions covering topics such as reinforcement learning in natural language generation, further expanding on the practical applications and challenges in deploying reinforcement learning models.

Closing Remarks: Conclude the discussion with reflections on the role of reinforcement learning in driving innovation and efficiency in various industries, emphasizing the ongoing journey of exploration and adaptation in the dynamic field of artificial intelligence.

Join us in this insightful conversation with Susan Shu Chang, as we unpack the intricacies of reinforcement learning and its transformative impact on real-world applications, paving the way for future advancements in machine learning and AI technologies.

🧐 What’s in this edition?

1. 💻 NVIDIA GTC

2. 📊 What Do LLM Benchmarks Mean?

3. 🆕 The New Deci AI Inferece

4. 📰 The Deci Digest (Research and repositories)

5. 📹 New YouTube Tutorials

tl;dr: Deci just dropped a new model and an inference platform. Here’s a tutorial notebook for using the API and a notebook showing how to use it in LangChain! You can try the model directly in the playground here. It’s free, and no credit card is required to sign up!

Join 700+ Peers in the DLD Discord

Are you going to NVIDIA GTC?

The Deci AI team will be at Booth 1501 - come and say hi!

Deci AI CEO, Yonatan Geifman, is delivering a talk titled Can High Performance also be Cost-Efficient when it Comes to Generative AI? You can add this session to your schedule here.

Can’t make it to GTC but still want to watch sessions?

Below are some sessions that I find most interesting, which NVIDIA has made open for you. You have to register for the virtual event and add these to your schedule.

• Jensen Huang’s Keynote Speech

• XGBoost is All You Need

• Customizing Foundation Large Language Models in Diverse Languages With NVIDIA NeMo

• Rapid Application Development Using Large Language Models (LLMs)

• Retrieval Augmented Generation: Overview of Design Systems, Data, and Customization

• LLM Agent Fine-Tuning: Enhancing Task Automation

• Unveiling Transformer Learning for Trustworthy AI

You can explore more Generative AI sessions here.

📊 What Do LLM Benchmarks Mean?

Evaluating LLMs is hard. For several reasons:

• Difficulty assessing nuance, context, and reasoning

• Variability and inconsistency in outputs

• Lack of interpretability and explainability

• Resource-intensive evaluation

• Difficulty of evaluating open-ended generation

Overcoming these challenges is an active area of research involving developing better metrics, benchmarks, stress tests, human evaluation protocols, and transparency tools. However, evaluating LLMs remains fundamentally difficult due to their black-box nature and the open-ended nature of language generation.

Despite this, we still try because evaluating LLMs is important.

The way I see it, LLM evaluations can be divided into two categories:

1. Benchmarks

2. Vibe checks

Benchmarks gauge the LLMs overall performance on a dataset, while vibe checks are informal assessments performed manually by an AI engineer.

Vibe checks are subjective and difficult to compare across models, while benchmarks provide insights into the LLM's strengths, weaknesses, and performance compared to other models.

At least one pain in the ass about benchmarks is that…there are SO MANY benchmarks out there!

Some benchmarks evaluate the knowledge and capability of LLMs by rigorously assessing their strengths and limitations across a diverse range of tasks and datasets. Other benchmarks assess how well-aligned an LLM is - evaluating their ethics, bias, toxicity, and truthfulness. Some benchmarks evaluate the robustness of LLMs by measuring their stability when confronted with disruptions. There are risk evaluations that examine general-purpose LLMs behaviours and assess them as agents. There are even benchmarks for assessing an LLMs knowledge of domains as diverse as biology and medicine, education, legislation, computer science, and finance.

But, over the last year or so,  the community has seemed to converge around what I call the “big six”.

These benchmarks, used to assess base LLMs, include ARC, HellaSwag, MMLU, TruthfulQA, Winogrande, and GSM8K. These are the ones you’ve seen on the Hugging Face Open LLM Leaderboard - which is powered by Eluther AI’s Language Model Evaluation Harness.

But, what are these benchmarks? What do they measure?

Each of these benchmarks serves a unique purpose in assessing different aspects of LLMs. Everything from reasoning, commonsense understanding, and knowledge acquisition to truthfulness, logical deduction, and problem-solving abilities.

ARC (AI2 Reasoning Challenge)

• Released in 2018 by the Allen Institute for AI

• Contains 7,787 multiple-choice science questions for grades 3-9

• Measures an LLMs ability to reason and apply scientific knowledge

• Important because it tests higher-level reasoning and knowledge beyond just language understanding

• Pros: Challenging questions that require reasoning.

• Cons: Limited to multiple-choice format.

HellaSwag (Harder Endings, Longer contexts, and Low-shot Activities for Situations With Adversarial Generations)

• Released in 2019 by researchers from UW, AI2 and others

• Contains 70,000 multiple-choice questions

• Measures an AI system's ability to use commonsense reasoning to complete descriptions of situations

• Pros: Focuses specifically on commonsense reasoning.

• Cons: Contains some ambiguous or subjective questions.

MMLU (Massive Multitask Language Understanding)

• Released in 2021 by researchers from Stanford, DeepMind, Google and others

• Contains ~16,000 multiple-choice questions from 57 topics including STEM, social science, humanities

• Measures an LLMs multitask accuracy across a broad range of academic and professional subjects

• Important because it comprehensively evaluates the breadth of knowledge

• Pros: Very broad coverage of knowledge domains.

• Cons: Answers can be answered via information retrieval vs. reasoning

TruthfulQA

• Released in 2022 by researchers from UMass Amherst and Google

• Contains 817 questions designed to probe truthfulness and ability to avoid false or misleading answers

• Measures an AI system's factual accuracy and calibration

• Important because it evaluates truthfulness, which is critical for real-world applications

• Pros: Focuses on truthful answering, an important capability.

• Cons: Relatively small dataset.

Winogrande

• Large-scale dataset of 43,985 Winograd Schema Challenge (WSC) problems

• Introduced in 2020 to more rigorously evaluate machine commonsense reasoning

• Adversarially constructed to be robust against statistical biases in existing WSC datasets

• Highlights that models may be exploiting biases rather than achieving true commonsense understanding

GSM8K (Grade School Math)

• Released in 2021 by researchers from UC Berkeley, Google and others

• Contains 8,500 high-quality grade-school math word problems

• Important because it evaluates mathematical reasoning, a key component of intelligence

• Pros: High-quality problems that test mathematical reasoning. 

• Cons: Focused only on math word problems.

For chat and instruction-tuned models, we have the Holy Trinity of evals.

These are interesting because they’re evaluating models on open-ended generation…and they’re evaluating the models in some interesting ways! 

One of these interesting ways is the LLM-as-a-Judge approach. This leverages LLMs as judges to evaluate chat assistants based on open-ended questions.  MT-bench and Chatbot Arena benchmarks show that LLM judges like GPT-4 can match human preferences with over 80% agreement.  LLM judges complement traditional benchmarks and offer a cost-effective way to evaluate chat assistants.

LMSys Chatbot Arena

• Uses a pairwise comparison approach where users chat with two anonymous models side-by-side and vote for the better response

• Has collected over 240K votes across 45 models as of March 2024

• Computes an Elo rating for each model based on the pairwise votes to rank them on a leaderboard

• Pros: Crowdsourced diverse questions, tests real-world open-ended use cases, ranks models by human preference

• Cons: Votes may be noisy/biased, expensive to run

AlpacaEval/ AlpacaEval 2

• AlpacaFarm is a dataset of 52,000 instructions and demonstrations 

• Compares model outputs to a reference model using an LLM-based annotator

• Provides a leaderboard ranking model by win rate over the reference

• Pros: Fast, cheap, reliable proxy for human eval on instruction-following

• Cons: Biased towards verbose outputs, limited to simple instructions, not comprehensive

MT Bench (Multi-turn benchmark)

• Contains 80 high-quality multi-turn questions across 8 categories 

• Evaluates instruction-following, knowledge, reasoning, etc. over multiple turns

• Provides a score for each model and is used alongside Elo ratings in the Chatbot Arena leaderboard

• Pros: Tests challenging multi-turn abilities, provides category breakdowns, uses strong LLM judge

• Cons: GPT-4 judge can make errors, especially on math/reasoning, limited to 80 questions

Now that we’ve reviewed what these numbers mean, the table I will show you will make more sense!

We just released Deci Nano and the Deci Generative AI Development Platform!

You can read more about the model and the platform here. It has amazing scores across all benchmarks and is blazingly fast!

You can try it for yourself in the playground here. Signing up for the API is free; no credit card is required. Enjoy!

More of a code-first type of person? Me, too!

• Here’s a tutorial notebook which will walk you through using the API via cURL, requests, and the OpenAI SDK.

• Here’s a tutorial notebook showing how to use it in LangChain.

• And finally, a notebook showing you how to use the API along with LangChain and LangSmith

Happy hacking!

📰 The Deci Digest

👾 Google DeepMind has introduced Genie, which generates interactive playable environments from a single image prompt. The model has been trained on 2D games and robotic videos and shows potential for generalizability across domains.

🎬 Alibaba Research has published a paper on EMO, a framework for creating expressive videos from audio and image inputs. EMO uses a ReferenceNet network for feature extraction and a diffusion model for generating video frames.

📌 Pinterest engineers share lessons learned and best practices for unlocking AI-assisted development. Details include the opportunities, challenges, and successes the team encountered from the initial idea to the general availability stage.

🛠️ Microsoft has expanded Copilot with a wider range of Windows 11 settings adjustments and integrated plugins for services like OpenTable, Shopify, and Kayak.

🚗 As the development of autonomous driving continues to evolve, automotive developers are exploring ways to optimize their systems for better performance. One approach that has gained traction is tailoring smaller models to specific hardware. By doing so, developers can achieve greater efficiency and accuracy, which are crucial for successfully implementing autonomous driving technology.

New YouTube Videos!

A tutorial on decoding parameters and strategies. Showing how temperature, top-p, top-k, etc, impact the selected token

GPTQ Tutorial: Shrinking LLMs without Quality Loss

How does quantization impact model performance? An in-depth analysis

Direct Preference Optimization (DPO) Explained: A Comprehensive Tutorial

That’s it for this week!

Key Highlights:

Guest Introduction: Meet Dr. Mark Moyou, as he shares his journey from chemical engineering to systems engineering, driven by his passion for machine learning, and his transition from academia to industry.

Navigating PhD Research: Explore Mark's philosophy and skills acquired during his PhD journey, including valuable advice for prospective PhD students, emphasizing problem-solving skills regardless of the algorithm used.

Industrial Research and AI Deployments: Delve into the complexities of industrial research and AI deployments, covering hardware considerations such as GPUs and CPUs, balancing performance aspects, and advocating for a data-centric approach over developing better models.

Edge Inference and Deployment Frameworks: Gain insights into edge inference, quantization, and deployment frameworks, understanding the importance of hardware considerations, latency, and networking strategies for AI deployment on the edge.

Future of AI on the Edge: Explore the future of AI on the edge, discussing the role of hardware, video compression, data transfer, personalization, and feature stores in AI deployment, while addressing challenges and solutions in managing latency and throughput.

Closing Remarks: Conclude the discussion with reflections on the future of AI on the edge, emphasizing the importance of hardware in AI deployment and the ongoing journey of learning and adaptation in the dynamic field of artificial intelligence.

Join us in this enlightening conversation with Dr. Mark Moyou, as he provides valuable insights into the future of industrial research, AI deployments, and the evolving landscape of machine learning in real-world applications.

Key Highlights:

Guest Introduction: Meet Mark Ryan, the author of 'Deep Learning for Structured Data,' as he shares his expertise on machine learning with tabular data and his intriguing experiment translating COBOL to JavaScript using large language models.

Deep Learning and Tabular Data: Delve into the controversial topic of applying deep learning to tabular data, comparing its challenges and limitations to traditional approaches for image and text data.

Handling High Cardinality Categorical Columns: Explore the nuances of dealing with high cardinality categorical columns in deep learning, understanding the complexities and potential solutions.

XGBoost vs Deep Learning: Mark discusses the trade-offs between cost efficiency and simplicity, comparing XGBoost to deep learning for tabular data and shedding light on the considerations when choosing between them.

Feature Engineering: Understand the importance of feature engineering in deep learning for tabular data, exploring strategies to enhance model performance and interpretability.

Choosing the Right Framework: Navigate the landscape of frameworks for deep learning with tabular data, considering factors such as scalability and ease of use.

Scaling Deep Learning: Mark provides insights into scaling deep learning for tabular data, addressing challenges and considerations when dealing with large datasets.

Lisp and Reverse Polish Notation: Explore Mark's experiment with using Lisp for deep learning with tabular data, unraveling the intricacies of reverse Polish notation and its application.

Real-world Applications: Understand the practical applications of deep learning with tabular data, exploring its relevance in diverse industries and the nature of problem statements.

Challenges in Regulated Industries: Mark discusses the challenges of applying deep learning in regulated industries, highlighting considerations related to privacy, security, and compliance.

Pre-trained Models and Network Architectures: Gain insights into the use of pre-trained models, the design of network architectures, and the role they play in boosting the efficiency of deep learning for tabular data.

Translating COBOL to JavaScript: Explore Mark's unique experiment of translating COBOL to JavaScript using large language models, showcasing the interdisciplinary possibilities of deep learning.

Join us in this deep dive into the realm of applying deep learning to tabular data with Mark Ryan, as we uncover the impact, challenges, and innovations in this dynamic field.

It’s been a while since I published, and I have missed writing this newsletter!

But I’ve been gone for a good reason! I’ve been going hard all of January recording tutorials for the Deci YouTube channel.

We’ve already released a series where I hack around with Small Language Models (sub-13B parameters, which is odd to call small, but here we are) and a video showing you how to get started with GGUF models on the free tier of Google Colab.

In the coming weeks (and months), you’ll see videos on fine-tuning an LLM for chat, quantizing a model via GPTQ, DPO, and more! If there’s a topic you want me to create a video on, let me know in the comments!

Below is the first video of the Small Language Models series. Check it out, smash a like, hit the subscribe button, and ding the bell to get notified of new uploads (I’ve always wanted to say that)!

Join 1,300+ Members in Discord!

🧐 What’s in this edition?

1. 🗞️ Your Weekly AI Bulletin (News headlines)

2. 📰 The Deci Digest (Research and repositories)

3. 🚢How to Ship Computer Vision Models to Production Faster with Better Performance

4. 🎥 How to Efficiently Scale Video Analytics on Edge Devices: From Advanced Algorithms to Optimized Pipelines

🗞️ Your Weekly AI Bulletin

👩🏽‍🌾 Attentive.ai is leveraging AI and computer vision to transform the traditional, labour-intensive processes of landscaping and construction services. With a fresh injection of $7 million in funding, led by Vertex Ventures Southeast Asia and India, the startup is poised to broaden its AI-led offerings. This move follows a successful $5 million seed investment and marks a significant step forward in Attentive.ai's mission to automate and optimize outdoor service operations.

👩🏾‍🏫 How will the integration of artificial intelligence (AI) in classrooms shape the future of education and ethical decision-making among teachers? A recent study from the University of Southern California (USC) investigates this question. At the heart of this exploration is the USC Center for Generative AI and Society's report, "AI in K-12 Classrooms: Ethical Considerations and Lessons Learned." The study, led by Stephen Aguilar, sheds light on the complex interplay between educators' gender, their comfort with technology, and their ethical stances towards adopting AI in educational settings. It underscores the necessity of fostering critical thinking and ethical reasoning in students, preparing them for a future where AI's role is ever-expanding.

🌌 Dr. Sebastian Wolfschmidt and Christopher Straub are using deep neural networks to swiftly predict the long-term behaviour of galaxies. This AI-based method, rooted in Einstein's theory of relativity, marks a significant leap from traditional numerical simulations, offering predictions in mere seconds. Their research sheds light on the structure of galaxies and opens new avenues for verifying astrophysical hypotheses efficiently.

🏭 Daedalus, a startup emerging from the southwestern German city of Karlsruhe, is on a mission to redefine manufacturing with its AI-driven approach. Focusing on creating custom parts for industries like medical devices, aerospace, defense, and semiconductors, Daedalus is leveraging the power of automation to streamline the production process. Fresh off securing $21 million in Series A funding, the company is poised for expansion, aiming to open more factories and further its reach.

Join 1,300+ Members in Discord!

After an introduction session on RLHF, my friends at AI Makerspace are back with part two! This time, you’ll learn about Aligning LLMs via RLAIF!

This session will be on February 14th, 2024 at 10am PST.

Hit the “Notify Me 🔔” button on the video below so you don’t miss it!

🚢 How to Ship Computer Vision Models to Production Faster with Better Performance

Fast and efficient inference plays a key role in the success of computer vision-based applications, especially when performance is strictly required, such as in autonomous vehicles and IoT-enabled mobile devices.

In most cases, achieving real-time inference is necessary to deliver the best user experience. With inference acceleration in the spotlight, join our live webinar to learn about:

• The importance of inference performance

• Challenges in Computer Vision inference

• Factors that impact inference and how to improve them

• Tips and best practices for accelerating inference performance

Watch the Recording Here!

Check out the session I did with the LLMOps Space community on fine-tuning a model for chat

This session is basically a crash course on fine-tuning, quantization, peft, and evaluating your fine-tuning results.

In more detail, here's what I cover:

• Pre-training and fine-tuning

• The difference between a base model and a fine-tuned model

• Defining jargon (prompt engineering, RAG, fine-tuning, peft)

• Matrix Rank, quantization, LoRA, and QLoRA

• And a coding example where we take DeciLM-7B, fine-tune it for chat using a peft method (QLoRA) via Hugging Face SFT, and evaluate the results of the fine-tuning run.

Hope you enjoy it!

📰 The Deci Digest

📱 Researchers investigate methods to optimize powerful models for mobile devices, focusing on a tiny language model containing 1B parameters. The study presents an empirical analysis based on a 1B-parameter model, discussing the impact of neural architecture, parameter initialization, and optimization strategy. The paper demonstrates notable performance improvements through tokenizer compression, architecture modifications, parameter inheritance, and multi-round training, culminating in developing PanGu-π-1B Pro and PanGu-π-1.5B Pro models. These models show significant improvements over state-of-the-art models, especially in multilingual corpora, underscoring the effectiveness of the proposed methodologies.

🧑 An open-source framework for augmenting humans using AI. Fabric addresses the challenge of integrating generative AI functionalities into everyday lives. Its approach is to break problems into individual pieces and then apply AI to them one at a time. It’s got an awesome prompt library that’s also worth checking out!

🕒 Google introduces a decoder-only foundation model for time-series forecasting. The TimesFM model is based on pretraining a patched-decoder style attention model on a large time-series corpus. With 200 million parameters, this model is designed for time-series forecasting and demonstrates strong zero-shot performance on unseen datasets of different domains and temporal granularities. Unlike traditional deep learning architectures, TimesFM provides decent out-of-the-box forecasts on unseen time-series data with no additional training, allowing users to focus on refining forecasts for specific downstream tasks.

⚙️ AdeptAILabs launches Fuyu-Heavy, a novel multimodal model tailored for digital agents. It’s the world's third-most-capable multimodal model, excelling at multimodal reasoning and UI understanding. Despite having to devote some of its capacity to image modelling, it matches or exceeds the performance of models in the same compute class on standard text-based benchmarks.

🍎 Apple has introduced an open-source AI model named MLLM-Guided Image Editing (MGIE), a tool that stands out for its ability to understand and execute text-based commands for photo manipulation. Developed in collaboration with researchers from the University of California, Santa Barbara, this model leverages multimodal large language models (MLLMs) to transform vague or simple text prompts into precise editing instructions. Whether it's making a pizza look healthier or adjusting a person's appearance in a photo, MGIE seems to have a knack for intuitively understanding and implementing user requests.

Join 1,300+ Members in Discord!

🎥 How to Efficiently Scale Video Analytics on Edge Devices: From Advanced Algorithms to Optimized Pipelines

Join us on Thursday, February 15, 2024, at 11:00 a.m. PST for a webinar to guide you through this intricate process. We will delve into selecting efficient models tailored for object detection on edge hardware and enhancing throughput and hardware utilization with advanced tools and techniques. Join us to uncover the keys to achieving superior real-time analytics performance at the edge.

- 🚀 Dive into selecting top-notch models for object detection on edge devices & boosting performance!

- 💡 Optimized Models: Discover cutting-edge object detection & pose estimation models perfect for edge gadgets like NVIDIA Jetson Orin.

- ⚡ Throughput Enhancement Techniques: Uncover how tricks like compilation and quantization can majorly speed things up.

- 🛠 Maximizing Hardware Utilization: Learn to run multiple video streams on one device to get the most out of your hardware.

🌎 Real-World Applications: Learn about practical uses and case studies that show how these tactics work in various situations.

Register Here!

That’s it for this week!

If you enjoyed this edition, smash a like or leave a comment!

Cheers,

Harpreet

And now, I’m excited to announce two more: DeciCoder-6B and DeciDiffusion-v2.0.

👩🏾‍💻 𝐃𝐞𝐜𝐢𝐂𝐨𝐝𝐞𝐫-𝟔𝐁

DeciCoder-6B is a bigger and stronger version of DeciCoder-1B (the first model we released on Hugging Face in mid-2023). DeciCoder-1B was extremely well received by the community, having nearly 200k downloads last month.

Take a look at the blazingly fast generation speeds below 👇🏽

Details about DeciCoder-6B

👉🏽 Supports 𝟖 𝐥𝐚𝐧𝐠𝐮𝐚𝐠𝐞𝐬: C, C# C++, GO, Rust, Python, Java, and Javascript.

👉🏽 Released under the 𝐀𝐩𝐚𝐜𝐡𝐞 𝟐.𝟎 𝐥𝐢𝐜𝐞𝐧𝐬𝐞

🥊 𝐏𝐮𝐧𝐜𝐡𝐞𝐬 𝐚𝐛𝐨𝐯𝐞 𝐢𝐭𝐬 𝐰𝐞𝐢𝐠𝐡𝐭 𝐜𝐥𝐚𝐬𝐬 𝐨𝐧 𝐇𝐮𝐦𝐚𝐧𝐄𝐯𝐚𝐥: Beats out CodeGen 2.5 7B and StarCoder 7B on most supported languages. Has a 3-point lead over StarCoderBase 15.5B for Python

💻 Resources to get started

Your help in supporting this project by liking the model card on Hugging Face is much appreciated!

🃏 𝐌𝐨𝐝𝐞𝐥 𝐂𝐚𝐫𝐝: https://huggingface.co/Deci/DeciCoder-6B

📓 𝐍𝐨𝐭𝐞𝐛𝐨𝐨𝐤: https://colab.research.google.com/drive/1QRbuser0rfUiFmQbesQJLXVtBYZOlKpB

🪧 𝐇𝐮𝐠𝐠𝐢𝐧𝐠𝐅𝐚𝐜𝐞 𝐒𝐩𝐚𝐜𝐞: https://huggingface.co/spaces/Deci/DeciCoder-6B-Demo

Interested in more technical details? Read the full blog here.

🎨 𝐃𝐞𝐜𝐢𝐃𝐢𝐟𝐟𝐮𝐬𝐢𝐨𝐧 𝐯𝟐.𝟎

DeciDiffusion 2.0 is an advancement over our first text-to-image generation model, DeciDiffusion-v1.0. It includes the Variational Autoencoder and pre-trained Text Encoder CLIP. DeciDiffusion's U-Net component is optimized for cost-effective hardware, using AutoNAC-generated U-Net-NAS with 525 million parameters.

This makes DeciDiffusion highly efficient and effective for text-to-image generation tasks, check it out 👇🏽

Details about DeciDiffusion

👉🏽 Produces quality images on par with Stable Diffusion v1.5, but 𝟐.𝟔 𝐭𝐢𝐦𝐞𝐬 𝐟𝐚𝐬𝐭𝐞𝐫 𝐢𝐧 𝟒𝟎% 𝐟𝐞𝐰𝐞𝐫 𝐢𝐭𝐞𝐫𝐚𝐭𝐢𝐨𝐧𝐬

👉🏽 Employs a 𝐬𝐦𝐚𝐥𝐥𝐞𝐫 𝐚𝐧𝐝 𝐟𝐚𝐬𝐭𝐞𝐫 𝐔-𝐍𝐞𝐭 𝐜𝐨𝐦𝐩𝐨𝐧𝐞𝐧𝐭 𝐰𝐡𝐢𝐜𝐡 𝐡𝐚𝐬 𝟖𝟔𝟎 𝐦𝐢𝐥𝐥𝐢𝐨𝐧 𝐩𝐚𝐫𝐚𝐦𝐞𝐭𝐞𝐫𝐬.

👉🏽 Uses an optimized scheduler, 𝐒𝐪𝐮𝐞𝐞𝐳𝐞𝐝𝐃𝐏𝐌++, which 𝐜𝐮𝐭𝐬 𝐝𝐨𝐰𝐧 𝐭𝐡𝐞 𝐧𝐮𝐦𝐛𝐞𝐫 𝐨𝐟 𝐬𝐭𝐞𝐩𝐬 𝐧𝐞𝐞𝐝𝐞𝐝 𝐭𝐨 𝐠𝐞𝐧𝐞𝐫𝐚𝐭𝐞 𝐚 𝐪𝐮𝐚𝐥𝐢𝐭𝐲 𝐢𝐦𝐚𝐠𝐞 𝐟𝐫𝐨𝐦 𝟏𝟔 𝐭𝐨 𝟏𝟎.

👉🏽 Released under the 𝐂𝐫𝐞𝐚𝐭𝐢𝐯𝐞𝐌𝐋 𝐎𝐩𝐞𝐧 𝐑𝐀𝐈𝐋++-𝐌 𝐋𝐢𝐜𝐞𝐧𝐬𝐞.

💻 Resources to get started

Your help supporting this project by liking the model card on Hugging Face is much appreciated!

🃏 𝐌𝐨𝐝𝐞𝐥 𝐂𝐚𝐫𝐝: https://huggingface.co/Deci/DeciDiffusion-v2-0

📓 𝐍𝐨𝐭𝐞𝐛𝐨𝐨𝐤: https://colab.research.google.com/drive/11Ui_KRtK2DkLHLrW0aa11MiDciW4dTuB

🪧 𝐇𝐮𝐠𝐠𝐢𝐧𝐠𝐅𝐚𝐜𝐞 𝐒𝐩𝐚𝐜𝐞: https://huggingface.co/spaces/Deci/DeciDiffusion-v2-0

I'm pumped to see how you'll use these models in your projects. Your input and creativity are vital in pushing the boundaries of AI. Don't hesitate to share your experiences and join our discussions on our community forum.

Together, we're not just using technology; we're shaping the future of AI.

Cheers,

Harpreet

As we step into the brand new year, we're excited to bring you the latest updates and insights from the world of artificial intelligence. Happy New Year to you all!

In this edition, we've packed some fascinating AI content that you won't want to miss. So, without further ado, let's dive right in! 🚀

Join 1,200+ Practitioners in Discord

🧐 What’s in this edition?

1. 🗞️Your Weekly AI Bulletin

2. 🚀 How to Optimize LLMs for Production

3. 👥 Community Content

4. 📰 The Deci Digest

5. 👨🏽‍💻Open Source LLMs vs APIs: Pros, Cons & Everything in Between

6. 💬 Chat with PDFs using DeciLM-7B-instruct, LangChain, and FAISS

🗞️Your Weekly AI Bulletin

🔑 Are you ready to embrace the future of AI with just a keystroke? Microsoft seems to think so, as they're reshaping how we interact with PCs. In a bold stride, Microsoft has revamped the traditional PC keyboard by introducing a dedicated "Copilot" key, signalling their commitment to AI integration. This change, the first in about three decades, is set to make AI assistance more accessible than ever. But the question lingers: will users be on board with this new AI-centric approach?

🌀The recent upheaval at OpenAI is a saga worthy of a tech thriller. Sam Altman, the former Y Combinator president and a pivotal figure in OpenAI's rise, has been ousted as CEO in a move that's sent shockwaves through the AI community. But that's just the tip of the iceberg. In a gripping narrative that unfolded over several days, OpenAI's boardroom was the stage for dramatic exits and negotiations. The story is still evolving, and the details are as intricate as they are intriguing.

🚀 Are we on the brink of an AI revolution in industry-specific applications? The past year has been a rollercoaster for AI startups, with a flurry of activity that excited and rattled the tech world. Looking back on 2023's wild ride, it's clear that the landscape of artificial intelligence is evolving rapidly, and 2024 is poised to be a year of significant transformation. In a recent TechCrunch+ article, over 40 investors shared their insights on the future of AI startup investments. They painted a picture of an industry maturing beyond the initial hype, focusing on creating sustainable businesses. The buzz around AI tools was palpable last year, with everyone from tech enthusiasts to grandmothers giving them a whirl. Funding rounds were reminiscent of the heady days of 2021, despite some high-profile shutdowns and the drama around OpenAI and its legal tussles with the New York Times.

🤖 Could the future of healthcare be a silent AI partner in the room, meticulously documenting every word of your doctor's visit? Paris-based startup Nabla is betting on it, and investors seem to agree. Nabla has just secured a hefty $24 million in Series B funding, with tech giants like Cathay Innovation and ZEBOX Ventures showing their support. This AI-driven platform is not just a concept; it's already transforming how thousands of U.S. doctors interact with patients, thanks to a partnership with Permanente Medical Group. Imagine a world where doctors can focus solely on you during a consultation while an AI copilot takes care of all the administrative tasks.

Join 1,200+ Practitioners in Discord

🚀 How to Optimize LLMs for Production

Watch the webinar for an in-depth exploration into the forefront of model design and optimization techniques. Discover strategies to accelerate LLM inference speed without sacrificing quality or escalating operational expenses.

What you’ll learn:

• Explore efficient modelling techniques: Dive into techniques that enhance LLM efficiency while maintaining quality, including grouped query attention (GQA) and variable GQA.

• Understand recent LLMs: Discover why recent LLMs, such as Llama 2 7B and DeciLM 6B outperform older and significantly larger LLMs.

• Uncover advanced optimization techniques: Learn about advanced runtime optimization strategies like selective quantization, CUDA kernels, optimized batch search, and dynamic batching.

Watch the on-demand webinar here!

👥 Community Content

• Pradeep created a video on MobileVLM. MobileVLM is a multimodal vision language model designed for mobile devices, featuring language models with 1.4B and 2.7B parameters, pre-training in the CLIP style, and efficient cross-modality interaction. It performs comparably to larger models on various benchmarks. It achieves impressive inference speeds, with 21.5 tokens per second on a Qualcomm Snapdragon 888 CPU and 65.3 tokens per second on an NVIDIA Jeston Orin GPU, setting a new state-of-the-art standard.

• Jacob Marks, from our friends over at Voxel51, wrote an awesome blog about Why 2023 was the most exciting year in computer vision history (so far). In 2023, computer vision saw significant advancements, particularly in multimodal vision-language models (MMVLMs). These computer vision developments underscored the field's rapid evolution in 2023, with innovations spanning object detection, segmentation, self-supervised learning, view synthesis, text-to-image generation, fine-tuning techniques, video perception, and multimodal language models.

• Our friends at the LLMOps Space community are hosting a session on January 9th titled The Science of LLM Benchmarks: Methods, Metrics, and Meanings. They’re hosting Jonathan from Shujin AI, and they will talk about LLM benchmarks and their performance evaluation metrics. Register here

🗓️ Community Events

• How to Master Computer Vision Challenges in Advanced Driver Assistance Systems (ADAS) Development. Discover how we tackle the intricate challenges of achieving speed and accuracy on edge devices, ensuring optimal ADAS functionality, safety, and efficiency. Learn from Deci's experience in assisting automotive firms with deep learning model development and optimization, gaining insights into navigating diverse conditions, optimizing computing resources, and harnessing the power of Neural Architecture Search (NAS) for hardware-tailored computer vision models. Don't miss this opportunity to master the complexities of computer vision in the automotive industry.

• Our friends at AI Makerspace gave a great tutorial about Efficient Fine-Tuning of LLMs with LoRA. You can find the notebook here. They’re also hosting a session on January 10th, where they’ll teach you about Quantization of LLMs and Fine-Tuning with QLoRA.

• On January 10th, our friends Sage Elliot from Union AI and Yujian Tang from Zilliz are hosting a fireside discussion about The Essential Role of Vector Databases in LLMOps.

Join 1,200+ Practitioners in Discord

📰 The Deci Digest

✍️ Researchers introduce AnyText, a groundbreaking tool designed to seamlessly integrate texts into images, enhancing the aesthetics and utility of visual text manipulation. It uses a combination of latent and text embedding modules to integrate text with the image background seamlessly, supports multiple languages, and outperforms other approaches in evaluation experiments. Additionally, the project includes a large-scale multilingual text images dataset and will be open-sourced to foster the advancement of text generation technology.

📷 Nikon, Sony, and Canon are working on advancing camera technology to incorporate digital signatures into images, enabling easy differentiation from highly sophisticated fake images. These digital signatures, including date, time, location, and photographer information, aim to combat the proliferation of realistic fake images. Sony plans to release this technology for professional-grade mirrorless SLR cameras in 2024, and Canon will follow suit.

📄 JPMorgan AI Research presents DocLLM, a lightweight extension to traditional LLMs for reasoning over visual documents with rich layouts. DocLLM focuses on combining textual and spatial layout information without needing image encoders, utilizing disentangled attention matrices and a pre-training objective to handle complex layouts and content, leading to superior performance on various document intelligence tasks compared to state-of-the-art LLMs.

🏪 OpenAI intends to unveil the GPT Store next week, a platform for developers to showcase custom apps utilizing its text-generating AI models. The launch, initially announced last year, was delayed, and it's unclear if a revenue-sharing scheme will be in place. Still, more details are expected to be revealed in the coming week as OpenAI transitions from an AI model provider to a platform.

Join 1,200+ Practitioners in Discord

👨🏽‍💻Open Source LLMs vs APIs: Pros, Cons & Everything in Between

In this webinar, Yonatan Geifman, Co-Founder and CEO of Deci, navigates these complexities. Expect insights into advanced inference acceleration techniques, strategic deployment, and ways to optimize your workflow and enhance model performance.

Watch now to broaden your generative AI expertise:

• Gain a deep understanding of the generative AI inference stack and learn how to make informed decisions when selecting tools for optimal resource allocation and latency reduction.

• Discover strategies to accelerate LLM inference, including efficient batching techniques, multi-GPU utilization, selective quantization, and hybrid compilation.

• Become familiar with Deci’s high-performance SDK, designed to supercharge your models’ performance in on-premises deployments.

Watch the on-demand webinar here!

💬 Chat with PDFs using DeciLM-7B-instruct, LangChain, and FAISS

The DeciLM-7B-instruct text generation model offers top-notch accuracy with higher throughputs than other models in its class. This tutorial will comprehensively take you through how to use DeciLM-7B-instruct and LangChain to build a working chatbot to enable you to chat with multiple PDF files.

By the end of this tutorial, you will:

• Understand the capabilities and functionalities of Langchain in chatbot development.

• Learn how to load and process PDF documents for textual data extraction using LangChain.

• Understand the challenges encountered when chunking large documents like multi-page PDFs or books

• Grasp the concept of text embeddings and their role in information retrieval.

• Develop a retrieval-based question-answering system that can extract relevant information from documents.

• Recognize the benefits of using multiple documents as a knowledge base for chat with pdfs.

Read the full tutorial here

That’s it for this week!

Let me know how I’m doing.

Cheers,

Harpreet

Join 1,200+ Practitioners in Discord

1. 🗓️ 2023 Recap

2. 🗞️ Your Weekly AI Bulletin (News headlines)

3. ⛓️ DeciLM-7B vs Mistral 7B on Chain of Thought Tasks

4. 🥊 Small Giants: Best LLMs Under-13B in Open Source

5. 📰 The Deci Digest (Research and repositories)

6. 👨🏽‍🏫 LLM Evaluation and How Decoding Strategies Impact Instruction Following

7. 🦜 Evaluating RAG pipelines using LangChain and Ragas

What’s up, Community!

2023 is coming to a close….and wow, what a year it’s been!

I wanted to take a moment to summarize Deci's accomplishments this year:

• April: Launched the Deep Learning Daily Community Discord, which started with 0 members and has since grown to 1,300+!

  • I also launched this community newsletter you’re reading right now! Starting from 0 subscribers to over 2,300!

• May: We launched YOLO-NAS as part of Deci’s SuperGradients library. SG’s star count has increased from just under 500 stars before the YOLO-NAS launch to its current count of 4,000 stars 🌟!

  

• July: We launched Data Gradients, an open-source Python-based computer vision dataset analysis library.

• August: We released DeciCoder-1B under an Apache 2.0 License. This was the first generative model we released, and it’s received a warm welcome from the community! Just last month, it had nearly 340k downloads!

• September: We released DeciLM-6B, DeciLM-6B-Instruct, and DeciDiffusion. All three models were trending on Hugging Face for a minute before Mistral released their model and stole our moment in the sun 🥺.

• November: After we launched YOLO-NAS, one of the biggest requests we kept getting from the community was “Can we have YOLO-NAS-Pose?!” We heard you all loud and clear, and in November we released YOLO-NAS-Pose as part of the SuperGradients training library!

  December: After postponing the launch of this model a few times due to other huge announcements, we finally released DeciLM-7B, DeciLM-7B-Instruct, and DeciLM-7B-Instruct-GGUF. All are under the Apache 2.0 license!

  • DeciLM-7B is the most powerful 7B parameter LLM out there and took the top spot on the OpenLLM Leaderboard from Mistral-7B.

🗞️ Your Weekly AI Bulletin

🛡️ Shield AI, a defence technology startup, has just supercharged its Series F funding round with an additional $300 million, pushing the total to a hefty $500 million. This infusion of capital, a mix of equity and debt, fuels the company's ambitious project: an AI pilot system designed to transform aircraft into autonomous machines. Their cutting-edge product, Hivemind, aims to enable aircraft fleets to operate independently without needing remote control, communication, or even GPS.

🤥 Researchers have found that GPT-4 can exhibit deceptive behaviour when acting as an AI investor under stress in a fascinating turn of events. The study, detailed in a pre-print on arXiv, reveals that when faced with demanding performance expectations and insider information, the AI often engaged in illegal insider trading and lied to cover its tracks. This research raises the question: If AI can learn to deceive under certain conditions, how can we ensure that the technology remains trustworthy and aligned with legal and ethical standards?

🌤️ In a recent article reviewed by Science X, experts from Northeastern University discuss the evolving role of AI in predicting and managing weather and climate disasters. With 2023 witnessing significant weather-related destruction, including Hurricane Idalia and the Hawaii firestorm, the urgency for more advanced prediction tools is palpable. Integrating AI with traditional climate models is heralded as a game-changer for future disaster preparedness and response. With AI's potential to revolutionize our predictive capabilities, one must wonder: What will the landscape of disaster management look like when these advanced systems are fully operational? Will we see a significant reduction in the loss of life and property, and how will this shape our relationship with technology and nature?

💻 In a bold move, Microsoft has infused AI into the core of Windows, leveraging its hefty $13 billion investment in OpenAI. The tech giant has launched Bing Chat and Copilot, infusing AI capabilities into Edge, Microsoft 365, and even Windows 11. While the AI features are still evolving, Microsoft is not slowing down, with plans to upgrade Copilot with the more advanced GPT-4 Turbo and Dall-E 3 models. Will Microsoft's AI integration set a new standard for operating systems, or will it be a case of ambition outpacing practicality? The full article explores the unfolding AI narrative within one of the world's largest tech companies.

⛓️DeciLM-7B vs Mistral 7B on Chain of Thought Tasks

Explore a comparison between DeciLM-7B and Mistral 7B in my latest blog, highlighting their performances in Chain of Thought tasks.

Here’s what I cover:

• Comparative Analysis: Showcases a side-by-side comparison of DeciLM-7B and Mistral 7B on Chain of Thought tasks.

• Effectiveness in Complex Problem-Solving: Discusses how these models perform in detailed reasoning tasks.

• Methodological Approach: Describes zero, one, and three-shot regimes in evaluating the models.

• Performance Metrics: Highlights throughput, accuracy, and coherence as key comparison measures.

• The superiority of DeciLM-7B: Details how DeciLM-7B, especially its 'instruct' variant, outperforms Mistral 7B.

• Adaptability in Different Scenarios: Explores the consistent performance of DeciLM under various prompting conditions.

  Read the full blog here!

🥊 Small Giants: Best LLMs Under-13B in Open Source

2023 has not just been a year of technological strides in open large language models (LLMs). It’s been a year of redefining them.

In August, I shared my thoughts on the top 10 LLMs, but the subsequent months highlighted a significant landscape shift. One particularly striking trend is the emergence and impressive performance of smaller LLMs. Contrary to their unassuming size, these models have punched above their weight in performance, challenging our understanding of efficiency and capability in AI.

Initially intended to focus on the major LLM releases in the latter half of 2023, this post will instead concentrate on these smaller yet robust LLMs.

I will explore them in categories based on their parameter sizes, specifically those with 13 billion parameters or less, grouped by parameter size categories – 1 billion, 3 billion, 7 billion, and 13 billion. For each, I’ll provide a thorough introduction, details on their training data, current popularity metrics like likes and downloads, and a discussion on their unique contributions and use cases.

This exploration highlights how these smaller LLMs are reshaping the AI landscape, offering impressive capabilities despite their reduced scale.

Read the full blog here!

📰 The Deci Digest

🗣️ Myshell AI presents OpenVoice, an open-source instant voice cloning AI library. OpenVoice is an advanced technology that can accurately clone the reference tone color and generate speech in multiple languages and accents. With flexible voice style control, OpenVoice allows for granular control over voice styles, including emotions, accents, rhythm, pauses, and intonation. It also offers zero-shot cross-lingual voice cloning, which means that neither the language of the generated speech nor the language of the reference speech needs to be presented in the massive-speaker multi-lingual training dataset.

⚗️ Predibase has published a guide for engineers and machine learning practitioners keen on utilizing language model distillation for production applications. This guide is for engineers and ML practitioners interested in LLM distillation for production. It's assumed you have familiarity with deep learning and LLMs. The library focuses is on effectively distilling LLMs for production.

🖼️ BytedanceTalk releases DiffPortrait3D, a conditional diffusion model designed to generate 3D-consistent, photo-realistic views from just a single in-the-wild portrait. DiffPortrait3D  is fast and efficient and produces high-quality results for camera angles, facial expressions, and styles. They've achieved state-of-the-art results on in-the-wild and multi-view benchmarks.

🔮 As the year ends, one would ask, “Where is AI headed next?” VentureBeat surveyed leading VCs, uncovering their most daring predictions for 2024, encompassing topics like GPU shortages, AI regulation, and beyond. Venture capitalists (VCs) anticipate progress in AI ethics and governance, including fairness, transparency, and accountability. They predict AI expansion in healthcare, driving innovations in drug discovery, personalized medicine, and patient care. The AI industry is expected to shift its focus towards practical, industry-specific solutions, providing benefits in finance, retail, and manufacturing.

👨🏽‍🏫 LLM Evaluation and How Decoding Strategies Impact Instruction Following

You know, recently, my world’s been revolving around two things:

1. How do models generate text? (Trying to grok how various decoding strategies impact the resulting generations)

2. And how do we gauge how good they are at it? (The minefield known as LLM evaluation)

It’s not just idle curiosity. It’s my job.

I’ve been handed this cool yet daunting task: give the new kid on the block, DeciLM-7B, a thorough vibe-check. So, here’s the deal. It’s supposed to stand toe-to-toe with the big gun in town – Mistral-7B-v0.1. That’s our competitor’s ace.

And me?

I’m digging deep, comparing these two. Imagine late nights, endless papers, tons of coffee – the whole nine yards of a research binge. Then bam! Call it fate, serendipity, or just good timing – two pieces drop from the AI heavens. One is a deep dive into “Instruction-Following Evaluation for Large Language Models,” and the other is Damien Benveniste’s eye-opener on text generation techniques. Talk about perfect timing.

That lit a spark.

Why not mix these up? Why not put DeciLM-7B and Mistral-7B-v0.1 through their paces using these techniques? I’m a scientist at heart – experimenting, exploring, that’s my jam.

Read the full blog here!

🦜 Evaluating RAG pipelines using LangChain and Ragas

Here’s what you’ll learn in this blog:

• 🧪 Synthetic Data Creation: Understanding the process and importance of generating synthetic data for RAG model evaluation.

• 🛠️ Utilizing the Ragas Tool: Learning how to use ragas to assess RAG performance across various metrics comprehensively.

• 🔍 Impact of Retrieval Methods: Exploring how different retrieval approaches influence the effectiveness and accuracy of RAG models.

• 💡 Practical Application: Applying these concepts through examples and exercises to solidify understanding and skills in RAG pipeline evaluation.

Let’s begin our exploration, starting with the essentials of synthetic data creation, moving through the detailed process of evaluating RAG using ragas, and delving into the subtle influences of different retrieval methods.

Read the full blog here!

That’s it for this YEAR!

Cheers,

Harpreet

It’s been a while since we’ve connected, but I’m back with some exciting news to share👇🏽

I’ll keep this edition short so you can start experimenting with the model ASAP - here's what you need to get started:

📔DeciLM-7B Base Model Notebook: https://bit.ly/decilm-7b-notebook

📘DeciLM-7B Fine-tuning Notebook: https://bit.ly/decilm-7b-finetune

📗DeciLM-7-Instruct Notebook: https://bit.ly/declm-7b-instruct

Help us get trending on HuggingFace by ❤️ the model cards

🤗 Base model: https://huggingface.co/Deci/DeciLM-7B

🤗 Instruction-tuned model: https://huggingface.co/Deci/DeciLM-7B-instruct

🤗 Demo on HF Spaces: https://huggingface.co/spaces/Deci/DeciLM-7B-instruct

👀Check out DeciLM + Infery: https://hubs.ly/Q02cz_pB0

⭐️ Bonus resource: Evaluating DeciLM-7B vs Mistral-7B on Chain of Thought prompts

Note: I finished this notebook yesterday before learning about the news Mistral model. I will re-run this with the new Mistral instruct-tuned model later this week. If anyone from the community wants to re-run the notebook using the new Mistral model before then, please do.

📘 Check the notebook out here

🧐 What I'm doing

I sampled 30 random rows from the kaist-ai/CoT-Collection. This dataset is 1M+ rows, so it was infeasible for me to use them all. I chose 30 because, once upon a time, I was a clinical trials statistician, and 30 was always a magical number.

I then generated responses for each prompt under a zero, one, and three-shot setting for DeciLM-7B-Instruct and Mistral-7B-v01.

⚖️ 👩‍⚖️ Evaluations - LLM as Judge

I used LangChain string evaluators for the following, with GPT-4-Turbo as judge.

1. COT Evaluation (evaluate_cot): This grades answers to questions using chain of thought 'reasoning' based on a reference answer. It will return one of the following evaluation results: CORRECT or INCORRECT, evaluating whether the generation is correct, accurate, and factual.

2. Coherence Evaluation (evaluate_coherence): This gives a score between 1 and 10 to the generation, assessing whether it is coherent, well-structured, and organized based on a ground truth reference label. This is useful for assessing the quality of the generation's reasoning.

Faithfulness via ragas

I also used the ragas framework to measure faithfulness, which assesses how well a model's responses align with the given context or source material. This was also done using GPT-4-Turbo

🤷🏽‍♂️Why did I do this?

Mostly because I'm curious and thought it would be a cool project. I also work at Deci, and I'm skeptical of benchmarks and wanted to see if our model was as good as we claim.

All feedback is welcome. Enjoy!

I’ll get back to the usual Friday editions starting this week. Keep an eye out for our regularly scheduled newsletters!

Cheers,

Harpreet

🧐 What’s in this edition?

1. 🗞️ News headlines

2. 📰 The Deci Digest (Research and repositories)

3. How to fine-tune a base LLM with QLoRA

4. 🤨 (Opinion)

5. A poll - Let me know how I did this week

Join 700+ Peers in the DLD Discord

🗓️ Upcoming Community Events

Nov 20 - How to Fine-tune a Base LLM for Retrieval Augmented Generation (RAG): In a webinar, Deci and Ai Bloks will demonstrate the integration of Deci LM-6B LLM - fine-tuned for RAG - into a RAG workflow using Ai Bloks' open-source library, llmware. The webinar will focus on use cases in financial services, legal and compliance.

The webinar will provide hands-on experience with code samples, highlight every component of this state-of-the-art open-source RAG system, and show how to customize it for your workflows.

Nov 20 - How to Chat with Images Data Using New GPT-4 Vision API. Discover how GPT-4's Vision API transforms image data analysis with AI. Join a live demo and code-sharing session covering integration, strategies, and best practices.

🤖 Nov 22 - Agents: LangChain vs. OpenAI Assistants. Learn to develop complex LLM apps by our very own community member Chris Alexiuk at an event! LangChain develops reasoning apps using Chain-of-Thought for LLM. Combined with ReAct, it creates complex LLM apps. OpenAI's Assistants API simplifies creating agent-like apps. Ideal for LLM Ops practitioners and builders wanting to develop agent-like systems.

🎥 Community Newsletter Exclusive: Darwin Bautista on Scene Test Recognition with Permuted Autoregressie Sequence Models

The following interview is a conversation I had with Darwin Bautista at ECCV (European Conference on Computer Vision) about his paper "Scene Text Recognition with Permuted Autoregressive Sequence Models."

Bautista's work addresses scene text recognition (STR), which is challenging due to the text's variable font styles, orientations, shapes, illumination, and occlusions in natural scenes.

His paper presents a novel approach called PARSeq (Permuted Autoregressive Sequence models). This method is notable for its unified structure, which combines context-free non-autoregressive (NAR) and context-aware autoregressive (AR) inference and iterative refinement using bidirectional context.

This contrasts with previous methods that used separate language and fusion models.

Here are some highlights from our conversation and his research.

🏋🏼‍♀️ Challenges in STR: He highlighted the challenges in STR, such as recognizing partially obscured or incomplete text. Incorporating language context or modeling helps understand such text, similar to human perception.

🙆🏽 Approach to STR: His approach combines vision and language models. Initially, a vision model makes predictions which a language model then refines. He discovered that using a mixture of autoregressive and non-autoregressive techniques from NLP (Natural Language Processing) improves STR.

🔍 Findings and Innovations: Bautista's research showed that training models on real data containing misoriented or vertical text improved performance significantly compared to models trained only on synthetic data. He proposed establishing more challenging benchmarks for STR as current methods perform well on standard benchmarks.

🤖 Transformer-Based Model: Learn about the Transformer architecture utilized in PARSeq, showcasing a 12-layer Vision Transformer (ViT) encoder and a single-layer decoder, offering a new perspective in deep learning model design.

📈 State-of-the-Art Results: Learn how PARSeq achieved remarkable accuracy on STR benchmarks, excelling in recognizing arbitrarily oriented text, a key challenge in real-world applications.

🌐 Practical Applications: Find out how Bautista's research paves the way for practical applications, particularly in augmented reality and assistive technologies, highlighting the real-world impact of this research.

💡 Future Directions and Accessibility: Get insights into the potential future developments and the availability of this research for further exploration, with code and data accessible to the public.

🗞️ Your Weekly AI Bulletin

🧠 Have you ever imagined a workspace where artificial intelligence streamlines every search and simplifies your workflow? Dropbox and Nvidia are joining forces to turn this vision into reality. In a recent announcement, Dropbox revealed its collaboration with Nvidia to enhance its platform's productivity using advanced AI tools. This partnership is set to revolutionize how Dropbox customers interact with their cloud content by integrating Nvidia's cutting-edge AI technology.

🤖 Ever wondered if the cold precision of AI can be warmed up with a touch of human empathy in customer service? The founders of Siena AI are betting on it, redefining the customer service game for merchants. In a recent TechCrunch article, we dive into how Siena AI, co-founded by Andrei Negrau and Lisa Popovici, is tackling the notorious reputation of chatbots in customer service. With their background in e-commerce and software development for Shopify merchants, they've crafted an AI-powered solution that promises a machine's efficiency but with a human's understanding and empathy. This innovative approach aims to streamline customer service interactions without sacrificing the personal touch that brands have worked hard to cultivate.

🌱 Have you ever wondered how urban farming could revolutionize our cities and our plates? A recent article delves into the burgeoning world of high-tech urban agriculture, painting a picture of a future where our food grows up, quite literally, around us. The article explores the innovative approaches to urban farming, where technology and agriculture meet to create sustainable food systems within city landscapes. It highlights how these green initiatives are reshaping unused urban spaces and bringing fresh produce closer to consumers, reducing food miles and potentially lowering carbon footprints.

👁️ Could the eyes be a window to the heart's health? Ehsan Vaghefi, the CEO of Toku, seems to think so, and his personal story is as compelling as the technology his company is developing. Dive into how a childhood surrounded by the visually impaired led Vaghefi to innovate in the field of ocular imaging, aiming to revolutionize how we detect cardiovascular diseases. In a heartfelt narrative, we learn that Vaghefi's inspiration stems from his father's blindness due to congenital glaucoma. Rather than becoming a clinician, Vaghefi chose the technology path, founding Toku to leverage AI in diagnosing health conditions through the eye. Toku's flagship product, CLAiR, is an AI-powered retina scan that can non-invasively assess cardiovascular risks within seconds, potentially integrating into routine eye exams.

🤖 What does a change in leadership mean for a pioneering AI company like OpenAI? Today's news might give us a glimpse into the future of AI innovation. OpenAI has significantly shifted at the top in an unexpected turn of events. Sam Altman, the CEO and board member, has been dismissed, and Mira Murati stepped in as the interim CEO. Murati, with a rich background in engineering and product development, has been a critical player in the tech industry, with stints at Tesla and Leap Motion before joining OpenAI. Her promotion is critical for the company, which is known for its groundbreaking AI tools like ChatGPT and DALL-E.

Join 700+ Peers in the DLD Discord

🧘🏽YOLO-NAS Pose Has Arrived!

⭐️ Go and star SuperGradients, the official home of YOLO-NAS-Pose on GitHub

📓 Go and try it yourself with the quickstart notebook

🧑🏽‍💻 Train the model on custom data with the fine-tuning notebook

🤗 Try the demo on Hugging Face Spaces

Join 700+ Peers in the DLD Discord

📰 The Deci Digest

🖼️ Meta reveals various advancements regarding Emu, its first foundation model for image generation. New tools enable more control over image editing via text instructions and a new method for text-to-video generation.

🎼 Also, from Meta AI, researchers present a model that can produce 3D spatial audio for full human bodies. The system takes audio data from headset microphones and body positioning, generating a 3D audio environment around the individual. Data and code will be available by Dec 10th, 2023.

📔 An Adobe Research and Australian National University team introduces the Large Reconstruction Model (LRM). This pioneering model predicts the 3D structure of an object using only one image within just 5 seconds.

🔉 Qwen-Audio, the multimodal iteration of Alibaba Cloud's Qwen large model series, accepts various audio formats such as human speech, natural sounds, music, and text inputs, generating text as its output.

👔 Google’s AI-powered Search Generative Experience (SGE) is slowly rolling out to more countries. To help users find unique products, its shopping features use AI to generate gift ideas or fashion items that users can explore and purchase.

👨🏻‍🏫 How to Fine-Tune a Base LLM Using QLoRA

Are you ready to code? Because you’ll be doing a lot of that in this hands-on tutorial! Here’s what you’ll learn:

• How the QLoRA magic works

• The innovations that QLoRA introduced

• How to set up your config files for QLoRA and peft

• Hyperparameters for QLoRA

• How to prepare the model for training

• Training the model using SFTTrainer

• Overcoming deployment challenges

Read the full blog here

👩🏾‍💻 The Webinar Version: Join me on December 5th

Read the blog and want to go deeper? Join me for a webinar on December 5th, where I’ll fine-tune for a different use case, talk about evaluating the fine-tuned model, go into more depth, and answer your questions.

• Specialized Fine-Tuning: Adapt LLMs for niche tasks using labelled data.

• Introduction to Instruction Tuning: Enhance LLM capabilities and controllability.

• Dataset Preparation: Format datasets for effective instruction tuning.

• BitsAndBytes & Model Quantization: Optimize memory and speed with the BitsAndBytes library.

• PEFT & LoRA: Understand the benefits of the PEFT library from HuggingFace and the role of LoRA in fine-tuning.

• TRL Library Overview: Delve into the TRL (Transformers Reinforcement Learning) library's functionalities.

• SFTTrainer Explained: Navigate the SFTTrainer class by TRL for efficient supervised fine-tuning.

Sign up for the webinar here!

That’s it for this week!

Let me know how I’m doing.

Cheers,

Harpreet

I’m happy to announce the release of YOLO-NAS-Pose! See it in action 👇🏽

🧑🏽‍💻 Want to get right into some code? I've got you covered!

• 🏁 Quickstart guide

• 📻 Fine-tuning notebook

• 🪧 Demo on HuggingFace

⭐️ Go and star SuperGradients, the official home of YOLO-NAS-Pose on GitHub

The community has created some fantastic content about the model!

• What Advantages Does YOLO-NAS Pose Offer Over YOLOv8? by Ritesh Kanjee

• Joel Nader with a quick video showing off the HuggingFace space

• A new contender to the battle of pose estimation: Unveiling YOLO-NAS Pose by Henry Navarro

• Introducing YOLO-NAS Pose: A Leap in Pose Estimation Technology by the team at OpenCV

Check out Nicolai Nielsen’s video about the model 👇🏽

⭐️ Go and star SuperGradients, the official home of YOLO-NAS-Pose on GitHub

Read the full technical blog

Computer vision has witnessed remarkable strides, and the latest leap comes from YOLO-NAS Pose. This model isn't just an iteration; it's a redefinition of pose estimation's potential.

👤 Takes the foundational brilliance of YOLOv8 Pose and propels it to new heights. Focusing on real-time performance, it offers a unique blend of precision and speed, critical for applications in healthcare diagnostics, athletic performance analytics, and vigilant security systems.

🏗️ At its core, YOLO-NAS Pose is engineered using a state-of-the-art NAS framework, AutoNAC, which meticulously optimizes the architecture for unparalleled efficiency. This process has birthed a model with an ingenious pose estimation head seamlessly integrated into the YOLO-NAS structure.

📈 The training regimen of YOLO-NAS Pose deserves a spotlight – refined loss functions, strategic data augmentation, and a meticulously planned training schedule.

The result?

A robust model tailored for diverse computational demands and crowd densities without compromising accuracy.

🛠️ Deployment-wise, YOLO-NAS Pose stands as a versatile juggernaut.

Whether it's low-latency applications or scenarios where accuracy can't be traded off, this model adapts. It simplifies post-processing by unifying detection and pose prediction, giving us consistently reliable outputs.

🌐 And the best part? It's open-sourced. Deci has provided YOLO-NAS Pose under an open-source license with pre-trained weights for non-commercial research purposes.

This isn't just another model; it's a testament to where the field is heading. YOLO-NAS Pose is here to elevate our work, from experimental tinkering to deploying large-scale solutions.

Let's harness this technological marvel and see where it takes us. The future of pose estimation is here, looking incredibly precise and efficient.

Nov 9th: Optimizing Pose Estimation for Real-Time Performance

Join Eugene Khvedchenya, a Kaggle Grandmaster and deep learning engineer at Deci, for a live webinar that delves deep into the intricacies of pose estimation and offers unparalleled insights on optimizing it for better accuracy and real-time performance.

⭐️ Go and star SuperGradients, the official home of YOLO-NAS-Pose on GitHub

That’s it for this week!

Cheers,

Harpreet