Using AI to make user interfaces better, faster, and simpler
AI and Large Language Models give us the power to make our interfaces highly personalized, easier to use, and faster to develop. Learn what the hype about Generative User Interfaces (GenUIs) is, how it works, and how it can be implemented in this guide.
The growth of AI technology and models has led to a lot of discussion on how we can use generative AI to create better UI designs. There’s even predictions about AI completely replacing UX designers. But what if instead we used AI as a part of our interfaces, not a designer of them?
This article is the first in my series of GenUI and covers a brief introduction on what, how and why. Part two focuses on the GenUI Framework and how to practically get started (coming soon!)
What is a Generative UI?
A ‘generative’ interface displaying different layouts and components to different users. (Source: Image by the author)
A Generative User Interface (GenUI) is an interface that adapts to, or processes, context such as inputs, instructions, behaviors, and preferences through the use of generative AI models (e.g. LLMs) in order to enhance the user experience.
Put simply, a GenUI interface displays different components, information, layouts, or styles, based on who’s using it and what they need at that moment.
Adopting the principles of a Generative UI system promises to give your users better personalization and usability, while also simplifying many aspects of development and design. It’s still in its early days, and not without its own limitations and challenges, however it’s an exciting concept to look in to.
How it Works
A Generative UI system uses a combination of generative AI models (e.g. LLMs) to render components, layouts, and views in response to the data it is given. Their features can be implemented using a variety of approaches, data sources, and UI components.
Types of Implementations:
An interface may contain only a single generative element (e.g. dynamic dashboard sections), or be an entirely generated front end application (through API definitions and configuration). Here are some examples of types of generative interface features:
Dynamic Views and Components: Views, components, and screens built and ordered based on a specific user’s actions or preferences (e.g. A user that primarily uses ‘reports’ is shown a carousel of available reports on their dashboard as the first item).Generic Instruction Handling: An interface that handles user instructions or requests through plain text or voice input (e.g. an AI command box that executes actions without the need for specific screens or flows).Personalized Styling: Colors, styles, typography, and layouts adapted on user preferences or accessibility needs (e.g. larger text and higher contrast colors for a visually impaired user).Configuration Based Designs: Component type and layouts are generated at build time from configuration and application code (e.g. user settings page and site menus automatically built based off endpoints and API definitions).A sample GenUI framework. (Source: Image by the author)
A Generative UI Framework:
A GenUI system can contain some or all of the following data sources and parts:
Contextual Data: User data, user preferences, formats, instructions, session data, device data, etc.Application Knowledge: API definitions (actions), site structure (pages/routes), knowledge base (e.g. company data), etc.External Data Sources: external APIs (e.g. Product API), external Databases, etc.Components: Dynamic menus, data containers, inputs, forms, chat interfaces, themes, etc.Model Network: A collection of generative models with specific roles that decide what data to fetch, display, ask for, or send.
Generative vs. Traditional UIs
An interface with generative elements can provide new, and faster ways for your application user to get things done. Let’s see some side-by-side examples of a Generative versus a traditional UI.
The scenario: You have received your new new credit card which needs to be activated through your banking app before using it.
Traditional UI Example — A Context-less Journey
A traditional UI flow for activating a new credit card with a PIN. (Source: Image by the author)
There is no context-awareness of you receiving a new card, and it makes no difference to your home screen, menus, or messaging. You see the same screen you always do and are required to find your own way towards the activation screen.
Generative UI Example 1 — Dynamic Dashboard
GenUI input components are shown for a predicted action. Timely informational components are displayed after activation. (Source: Image by the author)
Meanwhile in GenUI land, the home dashboard is rendered based on your needs and account status.
It identifies that you have a new card in ‘pending activation’.It adds an input for activation action at the top and pre-fills what data it can.Once activated, it displays related data (card rewards and details) as it predicts you would seek out this information once you have a new card.
Generative UI Example 2 — Instruction Following
Instructions and parameters are submitted in the interface without the need for specific screens or input components. (Source: Image by the author)
Another generative approach is to allow plain text instructions in a chat-like interface, which can be handled by the interface. This type of interaction gives an interface the ability to all application actions without coding specific screens or forms.
Why use Generative AI for UI?
The world of UI is at a level of maturity and predictable-similarity that feels solved. A hamburger means menu, a floppy disk means save to the cloud, and every developer tool landing page must contain the word ‘seamless’ and ‘AI powered’ at least twice. Why fix what isn’t broken?
Outside of chatbots and girlfriend-substitutes, LLMs also have huge potential in the world of UI and UX. That’s because these models are:
Adept at handling natural language, multi-modal, and ambiguous data,Able to classify and decide without upfront training or fine-tuning to specific use cases, and;Accessible and affordable to almost everyone.
This means they are a great tool for making interface decisions and classifications. Or basically, they can do more for us.
Doing More For Us
(from gifer.com)
Improvements in a classical UI typically means better layouts, CTAs, or flows to guide us do things more effectively. But they still require us humans to do everything.
Conversely, GenUI interfaces can be designed to educate, predict, or do as much as they can for us. This means preempting and fetching what data we need, suggesting suitable next actions, and performing tasks for us where possible.
LLMs + Generative Interfaces make our interfaces smarter and better at working for us. Adapted from a maturity model for manufacturing systems. (Source: Image by the author)
Benefits of a Generative User Interface
Adopting one or many Generative UI concepts could provide a range of benefits for both users and developers of an application, depending on your use case.
Benefits for Users
Improved usability and experience: People get things done faster and in a way that suits them.Better accessibility: Adaptable styles ensures that anybody with visual, learning, or ergonomic impairments can be catered for.Improved understanding: An interactive interface can respond to plain language to cater to all types of communication and terminology.Smaller user learning curve: A complex product means a complex UI with a host of screens, views, and nomenclature for a user to learn. GenUI can suggest, guide and adapt to users, instead of relying on people reading documentation.
Benefits for Companies
Improved conversion metrics: 80% of people are more likely to purchase when given a personalized experience. Hyper-personalized experiences and messaging leads to improved conversion rates (~+20%) on signups and free-to-paid conversions.Reduced churn rate: 55% of customers would stop using a product or service they don’t understand. A GenUI avoids this risk by being able to infer or guide people in a language they understand.Reduced cart abandonment: You may be missing out on potential sales without proper accessibility. 16% of the population have a disability that requires assistive tools, and 70% of these shoppers will abandon a website they find difficult to navigate (link).
Benefits for Developers
Reduced UI development time: An adaptable UI frees up front-end development time spent on coding repetitive layout, display, options, and preferences.Scalable UI development: GenUI will shift the focus from screen and layout CSS and JS, to a configuration and component. A generative interface will choose and adapt UI elements to match the inputs and outputs of a function. As they change, so will the UI, without reworking the code.
Who benefits the most from GenUI?
While many GenUI implementations may bring advantages to almost any application, there are a sweet spot of products that may benefit the most from a generative approach:
Highly transactional: Apps where user satisfaction improves when it is faster and easier to get things done. Think banking, government services, HR applications. You log on to perform a series of (tedious) tasks and need them done as quickly as possible.Highly personal: Products that cater to a large variety of users, preferences, and needs, where a tailored experience would greatly improve satisfaction, usability, or conversions.Highly functional: Applications that have a large amount of distinct user actions, options, settings, preferences, where users may be confused or overwhelmed.
Generative UI Examples
It’s hard to justify any new technology as a step forward unless you can demonstrate that real customers can get real value from it (*cough* blockchain *cough*). So, I have compiled a variety of use cases and examples where a GenUI approach may be better than a traditional UI and let you be the judge.
Dynamic Components and Screens
A dynamic GenUI system will decide on the most appropriate component to render the data or request it has been provided. This may include how much data its given, and what its attributes are, previous user preferences, or the request itself.
In the below example, a user’s bank accounts is displayed in a column of large cards, or a sorted table. There is no right way to show accounts, instead there are optimal ways which will depend on user preferences (e.g. more/less detail), the request (e.g. ‘When did I last put money in each of my accounts?’), which part of the site application (home vs. accounts), or other contextual data.
A user’s bank account data displayed in different ways based on who, why, where they are looking at it. (Source: Image by the author)
Instruction Handling
Most applications, especially in business, provide services that we want to do as fast as possible. They’re boring but necessary. Any change that helps your customers do their work faster and easier is one they’re are going to love.
A GenUI can infer and carry out user requests, or apparent needs, in an efficient manner. It makes chaining, or combining actions, very straightforward.
Following Plain Text Instructions:
Completing plain text instructions without any navigation or form inputs. (Source: Image by the author)
Performing Three Tasks at Once:
Completing multiple plain text instructions without any navigation or form inputs. (Source: Image by the author)
Personalized Messaging
A component’s text and data can be tailored according to a users behavior and usage. This can provide a range of benefits for marketing and conversion optimization.
Tag lines and benefits adapting to different user behaviors and histories. (Source: Image by the author)
Understanding Your Customer
Fortunately, as their name suggests, language models are adept at interpreting language. They can infer what a user needs despite them not using your terminology.
The problem with traditional UIs is that there’s only way to signify actions and it’s up to us designers and developers to produce layouts, microcopy, tooltips, and documentation to guide people to do things. A GenUI interface that takes plain-text instruction doesn’t have this limitation and can understand and infer your user’s language.
Understanding the user’s goal without them needing to know the terminology or being able to find the right menu option. (Source: Image by the author).
Better Accessibility
A system that can update your theme and style variables can add accessibility through a range of features:
Style and color adaption: Automatically adjust CSS variables for font sizing, color, and contrast for every visual impairment.Reduce unnecessary navigation and clicking: A predictive interface will reduce the amount of navigation, scrolling, and clicking required.
Enhanced guidance and voice interaction: An interactive interface can provide feedback and guidance, through text or voice inputs.
A Simpler Developer Experience
Most of this article has been focused on the advantages of an interface adapting to a user’s needs however its worth noting that the same concept can be used at build time to make the development easier.
Rather than hard-coding the component(s) and input(s) for each specific user settings, all that’s required is a configuration with the details — e.g. API endpoints and definitions — and it can decide how to display and capture the data.
This can be especially useful for streamlining the generating of low-user-value screens such as user preferences and profile settings.
Limitations and Challenges
LLMs are Unpredictable
People want consistent and predictable interfaces but LLMs are inherently unpredictable. A UI that changes every time you log in will lead to frustration and confusion. A well-designed GenUI system has to ensure that same context and inputs should produce the same output.
Privacy Considerations
If you’re using personal information in your system then it’s not advisable to send this information to a third party LLM provider (e.g. OpenAI) without getting customer permission. On premise, or secure LLM services may be needed to ensure personal data doesn’t get into the wrong hands.
High Complexity and Upfront Effort
Until someone — like your author here — develops a framework to host and manage a GenUI service, it may be too resource intensive for smaller companies to build the required infrastructure and services.
As your application grows and evolves, a GenUI setup may scale better better than a traditional approach but the upfront effort may be prohibitive.
There are non-AI alternatives
You can rightfully say that an AI model is not necessarily needed to achieve the above examples. Custom styles, suggested actions, and personalization existed well before LLMs. GenUI excels as complexity grows — e.g. # of actions, variations, or user preferences — beyond what designers and developers have the time and awareness of. Until an application reaches that level, it may be a overkill to use an AI-based setup.
It’s Expensive
Using generative models will be more costly given the additional need to pay for tokens, GPU server time, and/or infrastructure. Caching, falling token costs, and other enhancements will reduce this cost over time but it will still be more than a traditional setup.
Conclusion
Thanks for reading! I will be writing more in-depth guides in this series as I follow its growth and I hope you continue follow and read along. I’d also love to hear any comments or questions you might have.
I think the concepts of a Generative UI (GenUI) has a lot of advantages and innovations to unlock. As the ecosystem matures, I hope many of these ideas evolve and become common in the interfaces we use each day. Anything that promises to make things easier and faster is worth pursuing.
Get involved & reach out
I’m working on more material and tools for GenUI frameworks so please reach out to me if you want to talk more about the article or are interested in building something in this space.
An introduction to Generative UIs was originally published in UX Collective on Medium, where people are continuing the conversation by highlighting and responding to this story.
