Searching for the ImageNet in the financial field | Real-time recording of Qifu Technology's live broadcast: How does multimodal AI in credit establish standards?

Recently, Qifu Technology, in collaboration with researchers from Fudan University and South China University of Technology, jointly launched a live discussion with the theme "How to Set Standards for Multi-Modal AI in Credit". The live stream deeply analyzed the first multi-modal evaluation benchmark for credit scenarios, FCMBench-V1.0. This benchmark is designed to evaluate tasks focusing on multi-modal perception, reasoning, and decision-making key processes, and simultaneously open sources datasets and evaluation tools in an attempt to establish a widely accepted "ruler" for financial AI. The entire sharing session lasted for 1 hour, integrating cutting-edge academia with industry practice, providing professional references and development insights for financial institutions, research institutions, and industry professionals. The following is a summary of the core content of this live stream. Industry Practice Perspective: FCMBench provides a unified measuring standard for the capabilities of financial AI models Yang Yehui, the head of multi-modal at Qifu Technology, first analyzed the pain points of the development of financial AI from an industry practice perspective and the original intention and core design logic of FCMBench-V1.0. He vividly compared AI to a tool "hoe", with the financial and medical industries, high-threshold industries with development potential, being the "fertile soil". The high requirements of the financial industry for privacy, security, and compliance naturally determine that the verification of model capabilities cannot rely on "self-proclamation", but must establish an objective and unified evaluation system. The birth of FCMBench-V1.0 is aimed at solving the core dilemma of financial institutions in model selection. Yang Yehui pointed out that the current financial industry has the problem of different models each claiming high scores but lacking a unified comparison standard. The core value of FCMBench is to create a "unified ruler" to measure model capabilities, pulling different models to the same starting line, and allowing their capabilities to be tested in real business conditions. Regarding the design of this "ruler", Yang Yehui proposed three core principles that FCMBench adheres to: fairness, scientificity, and practicality. Fairness eliminates "self-proclamation" and establishes a unified evaluation baseline; scientificity is reflected in reasonable data distribution, task difficulty settings, and effective distinction of algorithm differences; practicality is the core, aiming for outstanding performance of models on the benchmark to directly adapt to real business scenarios. To make the evaluation more practical, FCMBench simulates more than a dozen real shooting disturbances, sets reasonable judgments on document information, multiple document comparisons, and other reasoning tasks to recreate various risk scenarios in credit business. For example, if a user provides an annual income accumulated over 500,000 but the tax rate is less than 10%, this obvious risk point is included in FCMBench's reasoning difficult problems. This tests the model's risk identification and anti-fraud judgment capabilities, ensuring that the actual value of the evaluation tasks is set. In Yang Yehui's view, FCMBench is not created "just for the sake of creating", its core goal is to give back to the business and the industry itself. Its positioning is a public resource for the financial industry, aiming to achieve a deep binding of AI capabilities with business value through unified standards. At the same time, FCMBench also serves as a communication bridge between academic research on large financial models and industry applications. On the technical level, it will continue to expand tasks, data types, languages, and modalities, achieving full-scenario coverage for credit AI; on the industry level, it will collaborate with universities to tackle technical challenges, invite banks and various financial institutions to participate deeply in co-construction, enrich real business data and scenarios, and promote its upgrade to an industry-recognized evaluation standard and even a group standard, becoming a practical threshold for model selection and cooperation in financial institutions. Academic Research Perspective: The "ImageNet Moment" of Financial AI urgently needs to come If the industry focuses on how the "ruler" is used, the academic community is more concerned with why the "ruler" is missing and how to create a truly credible "benchmark". Professor Chen Tao from Fudan University started from the history of AI development, addressing the root of the problem: "The development of large AI models highly relies on the open-source ecosystem, while the financial field currently lacks a unified evaluation dataset and standard recognized both domestically and internationally. Without a unified 'ruler', it is difficult for enterprises and academia to collaborate in research and cannot form a strong development ecosystem, fundamentally limiting the birth of large financial models." He turned his attention to the milestone of deep learning - ImageNet. "The ImageNet dataset has driven the explosion of deep learning, becoming the unified benchmark for image recognition, similar evaluation standards are the key to the breakthrough of the AI industry." Chen Tao believes that the financial field currently lacks this kind of unified, comprehensive evaluation dataset, making it difficult to form a collaborative development ecosystem, urgently needing to create its own "ImageNet". Regarding the introduction of FCMBench-V1.0 by Qifu Technology, Chen Tao praised it as one of the large and authoritative unified evaluation benchmarks in the field of domestic and international financial credit. Compared to other scattered evaluation datasets in the industry, FCMBench-V1.0 has achieved mode unification for the first time, covering multiple core tasks such as credit and risk control, and fully designed for real business scenarios. With the characteristics first launched by Qifu Technology and the industry, it combines comprehensiveness and practicality, making it an important exploration in the financial field to create its exclusive "ImageNet". Industry-Academia-Research Integration Perspective: Significant advantages of financial AI implementation, FCMBench connects industry needs with talent cultivation Professor Xu Jianwu from South China University of Technology interpreted the actual application status and implementation advantages of financial AI from the perspective of industry-academia-research integration, as well as elaborated on the important value of FCMBench in industry talent cultivation. He first clarified a common misunderstanding: "Many people intuitively feel that AI has a weak 'presence' in the financial field, which is not accurate. AI has long been deeply involved in core scenarios such as insurance pricing, asset evaluation, and quantitative trading, it's just that these values do not directly appear in consumer products, so they are 'invisible'." At the same time, Xu Jianwu pointed out that compared to other high-threshold industries such as healthcare, financial AI has significant advantages in landing efficiency, with implementation efficiency reaching tens or even hundreds of times. This advantage stems from the fact that the financial credit field can quickly verify model actual effects through historical data backtesting and parallel testing of dual models, with a very short model adjustment cycle; while in the medical field, if changing algorithms, one needs to complete the full process verification from pre-clinical experiments, which can take three to five years, showing a huge gap in practical costs between the two. For the construction of financial datasets, Xu Jianwu proposed three core elements: value-driven, comprehensive and meticulous, fair and inclusive. He believes that high-quality financial datasets should first have valuable and innovative topics that can truly solve industry practical problems; secondly, the design should be comprehensive and meticulous, taking into account the multidimensional application needs of the industry; and finally, the evaluation method should be fair and just, with a focus on industry public value creation rather than being profit-oriented. The launch of FCMBench-V1.0 perfectly fits these three core elements, while also playing an important role in industry talent cultivation. Xu Jianwu stated that FCMBench is an important link connecting talent cultivation with industry needs, improving the industry's talent pool. It can provide real industrial practice scenarios for students studying AI in finance and related fields, enhancing their competitiveness in employment. It can also provide algorithm students with practical scenarios in the financial industry, helping them quickly adapt to the job demands in the financial field, and continue to supply high-quality talent to the financial industry, improving the industry's talent pool. In this live broadcast, the three guests each discussed the standard construction of multi-modal AI in credit from three different perspectives: industry practice, academic research, and industry-academia-research integration, providing the industry with a clearer understanding of the current status, pain points, and future direction of financial AI development. In the future, with the continuous operation and co-construction of FCMBench-V1.0, as well as the participation of more financial institutions and research institutions, the financial field is expected to gradually form an open-source ecosystem similar to ImageNet, allowing a deeper integration of AI technology with financial business, promoting the development of financial AI towards standardization and normalization, ultimately realizing a two-way empowerment of technological breakthrough and industrial implementation.