AI Serving Society: A heart for the non-profit STXBP1 e.V.

At Blibla, we've always believed in using our AI expertise to make a positive impact on society. This belief led us to a unique collaboration with STXBP1 e.V., a non-profit foundation close to our hearts, quite literally. Tim Pietrusky, one of our co-founders, has a deep personal connection with the foundation, having helped establish it in response to his daughter's diagnosis with a rare DNA mutation known as STXBP1. This collaboration wasn't just about technology; it was about bringing together a community and harnessing the creative spirit of some extraordinary children.

From Scribbles to Symbols of Love

The journey began with a simple yet profound idea: to create a new T-shirt design for STXBP1 e.V.'s e-commerce shop, inspired by the children affected by the STXBP1 mutation. These children face a spectrum of neurodevelopmental challenges due to STXBP1-related disorders, which can include early-onset epilepsy, developmental delays, and intellectual disabilities, making activities like drawing exceptionally challenging​​​​. We wanted this design to be more than just a motif; it needed to capture the essence of these children's spirits, their resilience, and the sheer determination they show every day.

In collaboration with STXBP1 e.V., we reached out to the families within the community, inviting them to contribute drawings made by their children and siblings. Despite the difficulties posed by their condition, the response was overwhelming, with 157 drawings from 17 different families. This incredible participation not only highlights the creativity and strength of these children but also underscores the importance of supporting such meaningful causes.

Crafting the Dataset

The foundation of any AI project lies in the quality and preparation of its dataset. For the STXBP1 e.V. collaboration, our initial step was to gather a diverse collection of drawings from the community. These 157 pieces of art from 17 families were not just drawings; they were expressions of the unique perspectives of children affected by the STXBP1 mutation and their siblings.

Dataset Preparation

Preparing these drawings for AI training involved several steps:

• Digitization: All physical drawings received via post were scanned at high resolution to ensure digital quality was on par with those received via email.
• Image Cleaning: Each drawing underwent a cleaning process. This step was crucial to remove any background noise or irrelevant details, such as the texture of the paper or marks from the scanning process. We utilized image processing techniques to enhance the clarity of the drawings, ensuring that the focus was solely on the artwork.
• Normalization: To maintain consistency across the dataset, all images were resized to a uniform dimension. This step is vital for neural networks to process the data efficiently.
• Annotation: Perhaps the most intricate part of the process was annotating each drawing with a descriptive caption. These captions included key phrases like 'scribble in STXBP1 style' or 'scribble heart in STXBP1 style on a dark fabric background'. The annotation process was not just about labeling; it was about imbuing the dataset with context, enabling the AI to understand the essence of 'STXBP1 style'.

Cleaned up dataset

Ethical Considerations

Given the sensitive nature of the artwork and the privacy of the families involved, we ensured that all data handling was done with the utmost respect and confidentiality. Each family's contribution was anonymized and securely stored, adhering to GDPR standards and ethical AI practices.

AI at Work: Training the LoRA

Training an AI model, especially a LoRA, requires a nuanced approach. Our goal was to create a model that could generate heart designs in the unique 'STXBP1 style' derived from the children's drawings.

We chose StabilityAI's Stable Diffusion XL 1.0 (SDXL) as our foundational model due to its robustness and versatility in generating high-quality images. This decision was pivotal, as the base model significantly influences the training outcome and the quality of generated images.

Customization through LoRA

The essence of LoRA training lies in its ability to fine-tune a pre-existing model to adopt new styles or patterns without extensive retraining from scratch. This approach was ideal for our project, as it allowed us to infuse the base model with the unique artistic expressions found in the STXBP1 community's drawings.

Training Process

• Utilizing Kohya_ss: We initiated our training process with kohya_ss, a web UI for managing and customizing AI model training
• Environment Setup: Utilizing RunPod's infrastructure, we set up a training environment equipped with a NVIDIA 4090 GPU. This high-performance setup was crucial for handling the computational demands of training a LoRA model.
• Parameter Tuning: We applied a custom configuration, refined through previous projects, to balance the learning rate, epoch count, and other hyperparameters. This careful tuning ensures that the model learns effectively without overfitting or underfitting.
• Keyword Association: A critical step in the training was associating the 'STXBP1 style' keyword with the visual patterns in the dataset. This association enabled the model to generate new images that reflect the style when prompted with the keyword.

Outcome and Iterations

After training, we evaluated various checkpoints to identify the version that best captured the 'STXBP1 style'. Version 3 emerged as the most effective, striking the right balance between artistic fidelity and creative expression.

Through this detailed and thoughtful approach, we managed to bridge the gap between technology and human creativity, leading to the creation of a heart design that truly represents the spirit of the STXBP1 e.V. community.

The Heart of the Project: Generating the Design

When it came time to bring the heart designs to life, we dove into the creative process with a clear vision and a tool that felt like an extension of our own hands, ComfyUI.

This wasn't about crafting just any heart design; it was about creating something that resonated deeply with the essence and stories of the children and families involved.

Through an iterative process, we refined our prompts based on the feedback loop that included not just our team but the STXBP1 e.V. creative team as well. This collaboration was key, making sure the heart designs we were developing felt right and held a personal connection to the community's experiences and the children's original artwork.

The final selection process was a testament to the project's collaborative spirit, where the community's vote led to the choice of a heart design that best captured their collective journey. This chosen heart symbolized more than just a motif for a T-shirt; it represented love, resilience, and the shared experiences of all the families involved, showcasing the incredible synergy of technology, creativity, and community in creating something meaningful.

The winning heart was then prepared for print, ensuring the highest quality for the T-shirts. We again used ComfyUI with RealESGRAN to upscale the image from a resolution of 1024px x 1024px to 4k.

Conclusion: A Tapestry of Technology and Humanity

This project stands as a beacon of what's possible when technology meets humanity. The final T-shirt design is not just a piece of apparel; it's a symbol of love, unity, and the boundless potential of AI to serve society.

Join us in supporting STXBP1 e.V. by purchasing a T-shirt and wearing it as a badge of honor, celebrating the incredible journey of these children and the power of community-driven innovation.