Integrating Large Language Models and Video-based Design Analysis for Enhanced Design Insight Generation

Contact: Tianhao He (T.He-1@tudelft.nl)

With the advent of Large Language Models (LLMs) like GPT-4, there is an unprecedented opportunity for the design community to leverage the extensive knowledge and cognitive capabilities of these AI systems for innovative design thinking and problem-solving. The project, VisDesignLLM, endeavors to explore and develop a mechanism that synergistically integrates LLMs, design case databases, and video analysis, aiming to generate comprehensive and insightful design ideas. VisDesignLLM seeks to tap into the LLM's vast knowledge base, aiding designers in understanding intricate design cases, spotting potential problems, and devising effective intervention strategies. It envisions incorporating advanced video analysis techniques to interpret visual cues and patterns present in design-related videos. This approach provides a richer, more dynamic context for the LLM, allowing it to process and generate nuanced design insights with videos effectively, which can significantly enhance the designer's creative process and decision-making. In this initiative, the exploration and development phase will be crucial. The project will involve rigorous experimentation, testing, and validation using real-world design cases and videos, ensuring that the final mechanism is robust and effective. Continuous refinement and optimization based on feedback and performance evaluation will be integral to the project’s success. Through VisDesignLLM, the aim is to expand the horizons of design thinking, fostering a culture of innovation and excellence within the design community by providing a mechanism that encourages more insightful and creative design ideas.