SOSS Community Day Europe 2024

GenAI is defining a new industry. It uses different types of data to generate new content. It requires access to large volumes of data and generates even more data. Organisations are eager to adopt genAI projects due to their clear benefits and many use cases. GenAI initiatives often work with sensitive data such as sales data or customer behaviour patterns. Professionals working on these projects need many access points to organisations’ infrastructure, which can easily become a risk. Whether we’re thinking of the infrastructure where models are built or optimised or we focus on the edge devices where they run, there is a need to ensure the security of the entire stack. Open source tooling is widely used in AI projects due to its scalability and portability. Securing the entire stack will enable organisations to focus on genAI projects, without worrying about the security risks. This talk will walk the audience through all the layers of the stack, from the operating system to the MLOps platform, covering data centres where models are built and edge devices. It will present key considerations for security, best practices and opportunities for highly regulated industries.