In the past decade, the data-driven method utilizing deep neural networks has driven artificial intelligence success in various challenging applications across different fields. These advancements address multiple issues; however, existing methodologies face the challenge in data science applications, especially in fields such as biology, healthcare, and business due to the requirement for deep expertise and advanced coding skills. Moreover, a significant barrier in this field is the lack of communication between domain experts and advanced artificial intelligence models.
In recent years, the fast progress in Large Language Models (LLMs) has opened up many possibilities in artificial intelligence. Some well-known LLMs are GPT-3, GPT-4, PaLM, LLaMA, and Qwen. These models have great potential to understand, generate, and apply natural language. These advancements have created a medium for LLM-powered agents that are now being developed to solve problems in search engines, software engineering, gaming, recommendation systems, and scientific experiments. These agents are often guided by a chain of thought (CoT) like ReAct and can use tools such as APIs, code interpreters, and retrievers. The methods discussed in this paper include (a) Enhancing LLMs with Function Calling, and (b) Powering LLMs by Code Interpreter.
A team of researchers from Hong Kong Polytechnic University has introduced LAMBDA, a new open-source and code-free multi-agent data analysis system developed to overcome the lack of effective communication between domain experts and advanced AI models. LAMBDA provides an essential medium that allows smooth interaction between domain knowledge and AI capabilities in data science. This method solves numerous problems like removing coding barriers, integrating human intelligence with AI, and reshaping data science education, promising reliability and portability. Reliability means LAMBDA can address the tasks of data analysis stably and correctly. Portability means it is compatible with various LLMs, allowing it to be enhanced by the latest state-of-the-art models.
The proposed method, LAMBDA, a multi-agent data analysis system, contains two agents that work together to solve data analysis tasks using natural language. The process starts with writing code based on user instructions and then executing that code. The two main roles of LAMBDA are the “programmer” and the “inspector.” The programmer writes code according to the user’s instructions and dataset. This code is then run on the host system. If the code encounters any errors during execution, the inspector plays the role of suggesting improvements. The programmer uses these suggestions to fix the code and submit it for re-evaluation.
The results of the experiments show that LAMBDA performs well in machine learning tasks. It achieved the highest accuracy rates of 89.67%, 100%, 98.07%, and 98.89% for the AIDS, NHANES, Breast Cancer, and Wine datasets, respectively for classification tasks. For regression tasks, it achieved the lowest MSE (Mean Squared Error) of 0.2749, 0.0315, 0.4542, and 0.2528, respectively. These results highlight its effectiveness in handling various models of data science applications. Moreover, LAMBDA successfully overcame the coding barrier without any human involvement in the entire process of these experiments, and connected data science with human experts who lack coding skills,
In this paper, a team of researchers from Hong Kong Polytechnic University has proposed a new open-source, code-free multi-agent data analysis system called LAMBDA that combines human intelligence with AI. The experimental results show that it performs well in data analysis tasks. In the future, it can be improved with planning and reasoning techniques. It bridged the gap between data science and humans with no coding skills, successfully connecting them without human involvement. By bridging the gap between human expertise and AI capabilities, LAMBDA aims to make data science and analysis more accessible, encouraging more innovation and discovery in the future.
Check out the Paper, Project, and GitHub. All credit for this research goes to the researchers of this project. Also, don’t forget to follow us on Twitter and join our Telegram Channel and LinkedIn Group. If you like our work, you will love our newsletter..
Don’t Forget to join our 47k+ ML SubReddit
Find Upcoming AI Webinars here
Sajjad Ansari is a final year undergraduate from IIT Kharagpur. As a Tech enthusiast, he delves into the practical applications of AI with a focus on understanding the impact of AI technologies and their real-world implications. He aims to articulate complex AI concepts in a clear and accessible manner.
