In this series of blog posts, we'll share our experiences in building an AI-driven data mapping system for composable commerce. Starting with this first post, we delve into our journey of developing a model capable of generating mappings from arbitrary JSON data between different formats. Subsequent posts will explore each component of this process in detail.
In the dynamic realm of composable commerce, integrating multiple applications like commerce engines (e.g. commercetools, commerce layer), content systems (e.g. sanity, contentful), or search engines (e.g. algolia, constructor) is crucial. A key challenge is ensuring seamless integration between these systems, primarily through effortless data mapping. To tackle this, we embarked on creating a mapping framework capable of handling complex data structures. While this framework was effective for technical users, it was less so for the average user.
Our vision is a tool that required minimal domain knowledge, one that could intuitively generate mappings by understanding our API format and the target data format. This tool, building on our existing mapping framework, would use arbitrary user-provided JSON data to significantly simplify the mapping process for users less familiar with data formats and APIs.
Terminology
Before diving deeper, let's clarify some terms we'll use in this series:
Single-shot prompt: Ideal for tasks requiring a specific format or context with a single guiding example.
Few-shot prompt: Best suited for tasks needing multiple examples to provide broader context or handle more nuanced responses.
Fine-tuning: Ability to build your model based on pre-existing model with medium to large number of examples sent to large language model (LLM)
Objective
Our goal was to develop a system that could use any arbitrary user JSON data to generate mappings in our specified format and have a schema on the receiving end for the data we intended to map. We aimed for the model to perform this task with minimal human intervention and to keep the process cost-effective.
Solutions and Observations
Single-shot prompt with OpenAPI Specification
In our first attempt, we presented the model with a simple instruction and our OpenAPI specification, hoping it would autonomously generate the required mappings. Unfortunately, the model struggled to comprehend the OpenAPI specification, failed to generate the necessary mappings, and this approach was resource-intensive, both in terms of cost and time.
Few-shot prompt with OpenAPI Specification
Next, to give the model more context, we supplemented the OpenAPI specification with example mappings. This time, the model showed improved understanding but often confused different APIs within the same mapping. Despite its slight improvements, this approach remained costly and time-consuming, indicating a need for further refinement.
Few-shot prompt without OpenAPI Specification
We then decided to eliminate the OpenAPI documentation and rely solely on example mappings. The model's performance improved, demonstrating a better grasp of the expected format and result. However, as examples grew more complex, the model's performance became inconsistent, and the method was faster but still presented challenges in debugging and cost-efficiency.
Fine-Tuning the Model
After exploring the above methods, we opted to fine-tune the model, starting with a small set of examples and gradually increasing their number. We created a Colab template notebook for this process, beginning with 10 examples and incrementally increasing to 50. Significant improvements were observed even with a limited number of examples. The fine-tuning approach proved to be more cost-effective in the long term and yielded more reliable results than previous methods, being faster and more efficient.
As we increased the number of examples, covering various data structures and application types, our confidence grew in training the model to map data between different formats and composable commerce applications with minimal user intervention. This development marks a significant step forward in our mission to make composable commerce fast to build and accessible to everyone, not just technical users.
Conclusion
Our journey in integrating composable commerce with AI-driven data mapping has been a revealing one. Starting small and iteratively building the model's capabilities proved to be a wise strategy. We realised the importance of treating the model's responses as we would a human user's—by validating and continually enhancing them. This approach has not only improved our system's efficiency but also enriched our understanding of AI's potential in complex data mapping.
In our next blog post, we will delve into the single-shot prompt in more detail, examining how it can be used to generate mappings between different formats.
Useful Resources for Your Journey:
Kommentare