#19 Good or bad? Open source AI models are breaking grounds
In this week’s LiveWall innovation substack we check out Meta’s open source ai model, realtime generations with steam-diffusion and we take a look at the new object removal tool in Adobe’s Premiere Pro. Lets’s go!
Open Source AI language model LLama 3 is outperforming OpenAI and Google.
Meta has just launched their most impressive range of models to date: the Llama 3 family. This collection includes an 8B parameter mini model designed for lighter, on-device tasks and a 70B parameter model crafted for general knowledge tasks. If the benchmarks are to be believed, these models have set a new standard not only for open-source models but for Gen-AI models as a whole.
The release of these new Llama models is set to have a significant impact on the market. Enterprises are anticipated to triple their investment in Large Language Models (LLMs); however, they have faced a major challenge up until now. The highest-performing models have been closed-source, limiting access and customisation.
With the launch of these Llama models, enterprises now have access to state-of-the-art models that they can own, manage, and improve upon independently.
The open source nature of these models can lead to widespread benefits, such as increased collaboration, rapid advancements, and accessibility to technology across different parts of the world. However, it also raises concerns like security risks, potential misuse, ethical issues, and challenges in regulation.
Play around with these new models on Groq or Hugging Face.
Realtime AI art generation with StreamDiffusion
A few months ago, a small team of developers launched a new repository on GitHub called StreamDiffusion. This open-source project enables real-time image generation and is built upon earlier open-source models. The best part? It's fully accessible for anyone to use.
Realtime image generation using some basic drawing skills.
This innovative technology will be particularly advantageous in scenarios demanding continuous input, such as the Metaverse, live video streaming, and broadcasting, where high throughput is essential. Although it is open-source and accessible to everyone, it is computationally intensive. To generate images using the StreamDiffusion pipeline, you would need the following setup: an RTX 4090 GPU, a Core i9-13900K CPU, and Ubuntu 22.04.3 LTS as the operating system.
Realtime Generative AI filters built with simple prompting.
Adobe Gen-AI upscaling and video object removal
Adobe took a bit longer to get up to speed, but they are catching up and launching many new generative AI tools that integrate with their services. In the past few weeks, they launched two new features that intrigued us the most: VideoGigaGAN and Generative AI in Premiere Pro.
Generative Fill has become one of the most useful Generative AI tools around. Its ease of use and simple integration in the Adobe suite makes it accessible for anyone to use on their creative projects. This tool will now also be accessible in video format inside Premiere Pro.
In addition to Generative Fill, upscalers are also being introduced to video tools like Premiere Pro. Adobe employees published a paper demonstrating how they achieved 8x upsampling with some simple video formats. This would be an invaluable tool for enhancing existing content and utilizing previously "unusable" content.