WP2 – Federated Learning Platform Development
Main Purpose and Objectives
Work Package 2 (WP2) is dedicated to designing and implementing the Federated Learning (FL) platform infrastructure for the SYNTHEMA project. This infrastructure connects health data sites, leveraging harmonized data sources from WP1 for federated training and validation of anonymization and Synthetic Data Generation (SDG) pipelines, as well as privacy-preserving model aggregation from WP3. The primary focus areas for WP2 include developing the FL infrastructure with clinical centers as distributed nodes, enhancing Secure Multi-Party Computation (SMPC) protocols for privacy-preserving model aggregation, and improving Differential Privacy (DP) protocols to protect computational outputs.
Task Overview
T2.1 – System Requirements, Components, and Specification
Leader: UPM
Participants: DW, SBA, UNIBO, UoS, VICOM, i~HD, INTRA
Duration: M1-M12
The primary objectives of T2.1 were to define the infrastructure requirements and analyze previous projects to inform the design and implementation of the FL platform.
T2.2 – FL Framework Organization and Deployment
Leader: UPM
Participants: DW, SBA, UNIBO, UoS, VICOM, i~HD, INTRA, VHIR, ICH, UMCU, APHP, CHA, UNIPD
Duration: M7-M18, M34
This task focuses on organizing the framework, implementing additional features, deploying the FL architecture, and implementing the platform.
T2.5 – Integration of Platform Components and Interactions
Leader: INTRA
Participants: UPM, DW, SBA, UNIBO, UoS, VICOM
Duration: M18-M24, M6-M48
The goals of T2.5 include integrating all platform components and interactions into the FL platform, using Continuous Integration/Continuous Deployment (CI/CD) methods and DevOps best practices, developing a unified API and user interface, and conducting technical validation tests to ensure the integrated workflow functions correctly.
Latest Activity (M14-M18)
From months 14 to 18, the team focused on several key activities:
- Monitoring hardware acquisition by clinical partners.
- Providing consultancy support to the IT departments of clinical partners.
- Initiating collaboration with Flower developers.
- Reaching sprint 5 in the development process.
- Developing auxiliary components for data storage and inter-component communications.
- Finalizing the development of data cataloguing and ingestion, pending integration and testing.
- Redefining the design of federated training components with the help of Flower developers.
- Deploying additional Kubernetes services to support existing CI/CD environments, including centralized NFS storage, Kubernetes dashboard, and Helm package manager.
- Monitoring the development, deployment, and integration of platform components.
- Developing Jenkins pipelines for various components.
- Deploying platform components to the dedicated staging environment.
- Supporting the technical team by showcasing the use of Helm charts.
Planning and Action Items (M19-M24)
For the upcoming months (M19-M24), WP2 will focus on:
- Progressing through agile sprints to implement all defined components.
- Continuously revisiting the design to align with partner needs.
- Collaborating with WP3 to properly develop and integrate SDG and anonymization pipelines.
- Delivering the first version of the FL system (Milestone 1).
- Deploying the initial version of the FL system, ensuring servers are ready by July (M20).
- Preparing the FL data platform deliverable (D2.2) with a Table of Contents to be shared by July (M20).
Open Issues
Several open issues need to be addressed to ensure the project’s success:
- Ensuring timely acquisition and deployment of necessary hardware and software infrastructure.
- Aligning the development of federated training components with the redefined design from Flower developers.
- Addressing any integration challenges that arise from combining various platform components and ensuring smooth interoperability.
- Maintaining robust communication and collaboration among all participants to align on technical requirements and solutions.
Conclusion
WP2 is making significant strides in developing a federated learning platform that will play a crucial role in the Genomed4ALL project. By focusing on developing and integrating the FL infrastructure, enhancing privacy-preserving protocols, and ensuring continuous alignment with partner needs, WP2 aims to create a robust, scalable, and secure platform. The progress made so far, along with the detailed planning for the upcoming months, underscores the team’s commitment to delivering a state-of-the-art federated learning system that will facilitate advanced research and personalized healthcare solutions.