Optimizing Frontier’s Code Generation for Speed and Quality

Introduction

Creating Frontier, our generative front-end coding assistant, posed a significant challenge. Developers demand both fast response times and high-quality code from AI code generators. This dual requirement necessitates using the “smartest” language models (LLMs), which are often slower. While GPT-4 turbo is faster than GPT-4, it doesn’t meet our specific needs for generating TypeScript and JavaScript code snippets.

Challenges

1. Balancing Speed and Intelligence:

   • Developers expect rapid responses, but achieving high-quality code requires more advanced LLMs, typically slower in processing.

2. Code Isolation and Assembly:

   • We need to generate numerous code snippets while keeping them isolated. This helps us identify each snippet’s purpose and manage their imports and integration.

3. Browser Limitations:

   • Operating from a browser environment introduces challenges in parallelizing network requests, as Chromium browsers restrict the number of concurrent fetches.

Solutions

To address these challenges, we implemented a batching system and optimized LLM latency. Here’s how:

Batching System

1. Request Collection:

   • We gather as many snippet requests as possible and batch them together.

2. Microservice Architecture:

   • These batches are sent to a microservice that authenticates and isolates the front-end code from the LLM, ensuring secure and efficient processing.

3. Parallel Request Handling:

   • The microservice disassembles the batch into individual requests, processes them through our regular Retrieval-Augmented Generation (RAG), multi-shot, and prompt template mechanisms, and issues them in parallel to the LLM.

4. Validation and Retries:

   • Each response is analyzed and validated via a guardrail system. If a response is invalid or absent, the LLM is prompted again. Unsuccessful requests are retried, and valid snippets are eventually batched and returned to the front end.

Micro-Caching

We implemented micro-caching to enhance efficiency further. By hashing each request and storing responses, we can quickly reference and reuse previously generated snippets or batches. This reduces the load on the LLM and speeds up response times.

Conclusion

The impact of parallelization and micro-caching is substantial, allowing us to use a more intelligent LLM without sacrificing performance. Despite slower individual response times, the combination of smart batching and caching compensates for this, delivering high-quality, rapid code generation.

The post Minimizing LLM latency in code generation appeared first on Anima Blog.

Implementing Guard Rails for LLMs

Large Language Models (LLMs) have made a profound leap over the last few years, and with each iteration, companies like OpenAI, Meta, Anthropic and Mistral have been leapfrogging one another in general usability and, more recently, with the ability of these AI models to produce useful code. One of the critical challenges in using LLMs, is ensuring the output is reliable and functional. This is where guard rails for LLM become crucial.

Challenges in Code Generation with LLMs

However, as they are trained on a wide variety of code techniques, libraries and frameworks, trying to get them to produce a unique piece of code that would run as expected is still quite hard. Our first attempt at this was with our Anima Figma plugin, which has multiple AI features. In some cases, we intended to expand our ability to address new language variations and new styling mechanisms without having to create inefficient heuristic conversions that would be simply unscalable. Additionally, we wanted users to personalize the code we produce and have the capability of adding state, logic and more capabilities to the code that we produce from Figma designs. This proved much more difficult than originally anticipated. LLMs hallucinate, a lot.

Fine-tuning helps, but only to some degree – it reinforces languages, frameworks, and techniques that the LLM is already familiar with, but that doesn’t mean that the LLM won’t suddenly turn “lazy” (putting comments with /* todo */ instructions rather than implementing or even repeating the code that we wanted to mutate or augment). It’s also difficult to avoid just plain hallucinations where the LLM invents its own instructions and alters the developer’s original intent.

But as the industry progresses, LLM laziness goes up and down and we can use techniques like multishot and emotional blackmail to ensure that the LLM sticks to the original plan. But in our case, we are measured by how well the code we produce is usable and visually represents the original design. We had to create a build tool that evaluated the differences and fed any build and visual errors back to the LLM. If the LLM hallucinates a file or instructions, the build process catches it and the error is fed back to the LLM to correct, just like a normal loop” that a human developer would implement. By setting this as a target, we could also measure how well we optimized our prompt engineering, Retrieval-Augmented Generation (RAG) operations and which model is ideally suited for each task.

Strategies for Implementing Guard Rails

Conclusion: The Necessity of Guard Rails for LLMs

The post Guard rails for LLMs appeared first on Anima Blog.

Design to code is a difficult problem to crack, there are so many variations to consider. On the Figma side, we have to consider auto layouts, design tokens, component sets, instances and Figma variables. On the code side, we have to assume that the codebase could contain both local and external components that could come from anywhere.

That’s why, when we created Frontier, we didn’t want to stick to just one coding design system. MUI, for example, is a very popular React Design System, but it’s one of <very> many design systems that are rising and falling. Ant Design is still extremely popular, as is the TailwindCSS library. We’re seeing the rapid rise of Radix-based component libraries like ShadCN as are Chakra and NextUI. However, we knew that if we wanted to reach a wide audience we could not rely on a limited subset of Design Systems, we had to create a “pluggable design system”.

Key Challenges in Implementing a Pluggable Design System

There are a few challenges to accomplishing this:

• 1. Existing Project Integration:

     You have an existing project that already uses a design system. In this case, we are expected to scan the codebase, and understand and reuse the design system. We do this when Frontier starts, it looks through your codebase for local and external components (you can restrict where it actually scans and also control how deeply it looks at the code) for your code components and usages of those code components.

  2. Design and Code Component Mismatch:

     When we look at the Figma design, we don’t assume that the designer has a clear understanding of which component system will be utilized to implement the design. Typically, if this is an Enterprise with a Design System Team, the components in the design will match in design. Still, not necessarily in their name, variants nor have a 1:1 match between the Figma and code component counterparts. In fact, the same design could be used with different Design Systems code components and fully expected to match and work.

  3. Flexible Implementation:

     Once applied, components could have multiple ways to implement overrides and children:

     1. Props / variants
     2. Component children
     3. Named slots

  4. The “Cold start” problem

     Even if you solve scanning the project’s repo, what happens when you encounter a brand new project and want to use a new library with it? In this case, you would have zero code usage examples and zero components that you are aware of…

To overcome these problems we started with a few assumptions:

• 1. Leverage Usage Examples:

     the project has a robust set of usage examples, we can take inspiration from them and understand how this particular project utilizes those components, which will help us solve the prop/overrides/children/named-slots issue.

  2. Custom Matching Model

     We had to create a custom model that understands how designers in design systems implement their components and how developers code the code components. This matching model was trained on a large set of open source Design System repos and open Figma design systems. It reached a surprisingly high matching rate on all our tests. Looks like many designers and many developers think in similar ways despite using very different conventions and actual designs.

  3. Cross-System Matching

     Once we were able to match within the same design system, the next challenge was to make the model more robust with matching across design systems – take a design that relies on AntD components and train the model to implement it using MUI components, or vice versa. This made the model much more versatile.

  4. Local Storage for Privacy and Security

     For security and privacy purposes, we have to encode and store our RAG embeddings database locally, on the user’s machine. This allows us to perform much of the work locally without having to send the user’s code to the cloud for processing.

      

Interestingly, the fact that we can store bits and pieces of this databases, also opens up possibilities with cold starts. An empty project can now easily state that it wants to use MUI and simply download and use the embeddings. That gives the usage LLMs all the context that’s needed to produce much more robust results, even when the codebase is completely empty from any actual context.

The result is that Frontier can now generate code components in projects, even if the Design System doesn’t actually match the code design library and even when the codebase is completely devoid of any actual examples.

The post Pluggable design system – Figma to your design system code appeared first on Anima Blog.

Short answer: Yes!

Long answer:

NextJS is an extremely popular framework for ReactJS that provides quite a few benefits, one of which is the mix between server and client-side components. 

Server-only components are components that do not use/need state and can pull their data from external APIs without worrying about credentials falling into the wrong hands. They can only be rendered on the server. Server components may contain server and/or client components.

Client-only components are components that have the “use client” directive defined. A component that uses state and other React APIs needs to be a client component, but a client component doesn’t require state to function.

In Next.js, components are server components. This ensures that fully-formed HTML is sent to the user on page load. It’s up to the developer’s discretion to set the client boundaries. If components are not using state and are not making outward API calls, they can be implemented both as client and server, which is ideal. 

Since it can be quite complex to determine, which type of component a particular React component is (Server only, Client only, agnostic), Frontier will generate client components by default, when it detects NextJS. This is done by adding the ‘use-client’ statement at the beginning of the component declaration. 

This issue arises because it can be challenging to identify if the rendered component tree includes descendants that must be rendered on the client side. Without a ‘use client’ directive for those components, runtime errors may occur.

If you remove the ‘use-client’ and the code still builds with no errors, this means that the client boundaries have been set correctly, and you can let Next.js determine whether the component is rendered on the client or the server. If, on the other hand, removing it causes a build error, it means that one or more of the descendants uses client-only APIs, but hasn’t declared itself as a client component. In this case, you can add the ‘use-client’ statement in the code we’ve created, or add the directive directly inside of the offending descendant.

So, what’s the bottom line?

Short answer: Yes, Frontier supports NextJS!

Start here!

The post Does Frontier support NextJS? appeared first on Anima Blog.

The post Generative code: how Frontier solves the LLM Security and Privacy issues appeared first on Anima Blog.

LLMs Don’t Get Front-end Code

Enter: Useful generative coding

The post LLMs Don’t Get Front-end Code appeared first on Anima Blog.